[Bf-committers] Python!

[Kevin Seghetti]

--=-SSHlKI3gHbNNtY8gBtF9
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Thu, 2003-01-09 at 03:56, nlin@nlin.net wrote:
> On Thu, Jan 09, 2003 at 09:39:06AM +0100, Laurence Bourn wrote:
> > One of the problems, I think was associated with past python
> > implementtations is that they were built directly onto weak, ill though=
t out
> > C,C++ interfaces. It becomes more and more difficult to refactor, impro=
ve
> > existing C,C++ code because you have essentially exposed it's internal
> > working (through the python API) to end-users.
>=20
> My thoughts exactly. One interesting approach to scripting is that taken
> by World Foundry (http://worldfoundry.org, or http://wf-gdk.sourceforge.n=
et).
> It supports 3 scripting languages, with others being looked at.
>=20
> The way World Foundry abstracts the engine from the scripting language
> is through the concept of a mailbox. It's very similar to memory-mapped
> I/O. For simplicity I will use the analogy of the peek and poke commands
> in BASIC. Peek reads a memory address; poke writes to a memory address.
> With memory-mapped I/O, reading certain addresses that are mapped to
> hardware I/O doesn't return you a memory value, but instead returns to
> you the state of a certain hardware device. Similarly, writing to a memor=
y
> address which is mapped to I/O doesn't store a value in memory, but
> instead passes the value on to some hardware device. We can
> use this memory-mapped I/O idea to read/write engine state through a
> similar interface.
>=20
> The idea is that the engine exposes its state only through "mailboxes",
> i.e., only through memory addresses or slots. For instance, in a script, =
if
> you said "a =3D peek(1000)", you would get the value that the engine is
> exposing in slot 1000. The engine intercepts this call, then returns some
> state. For instance, you might have a convention that slot 1000 represent=
s
> the integer ID of the currently active object. Slot 1001 might represent
> the total number of objects in the scene. And so on and so forth.
>=20
> Writing to a mailbox is like writing to a hardware port. For instance,
> if you say "poke(1000,10)" you could set the current object ID to be 10.
> The engine intercepts the call, realizes that mailbox slot 1000 is the
> object ID, then calls the internal functions to change the active object
> ID.
>=20
> Furthermore there are system-wide mailboxes (which reflect global
> system state) and system-local mailboxes (which reflect state specific to
> a particular object). For instance, system local mailbox 2000 could
> represent the first element of an object's matrix.=20
>=20
> The engine publishes a list of constants defining which mailboxes
> have which meaning so you don't need to refer to them by name.
>=20
> In other words, the engine only exposes its state through numbered
> addresses (which can be given names through constants). The engine
> needs only implement two commands to support a new scripting language:
> set_mailbox and get_mailbox. The architectural decision made at the engin=
e
> level is "what state do I need to expose?" and (trivial) "what address
> do I want to use to expose this state"?
>=20
> The scripting language (e.g. Python, TCL, Lua) can then declare its own
> helper wrapper functions which access the engine through mailboxes, but
> which provide a more convenient interface to the scripting language
> programmer. For instance, to retrieve a 3x3 rotation matrix, you need
> to read 9 values, and read 9 maiboxes. A python script could declare
> a help function to read 9 mailbox values and pack them into a python obje=
ct.
> The script progrmamer doesn't need to deal with the mailboxes directly.
> And importantly, the Python API is thus defined ONLY in Python - it
> has nothing to do with Blender. Indeed, the Python "API" is a Python
> convention of using Python structures to wrap access to mailboxes in
> an easier fashion. Thus, defining new scripting API's has absolutely
> nothing whatsoever to do with the engine code. A script API function is
> just a wrapper to access engine mailboxes.
>=20
> This makes adding new API's in one scripting language, or new scripting
> languages entirely, easy. The scripting API has zero, nada, zilch to
> do with engine internals. The only interface between the engine and the
> scripting language is two functions: get_mailbox and set_mailbox.
> Everything else is a helper function in the scripting language, in a
> separate file totally unknown to the engine.
>=20
> I have used this scheme with success on a commercial coin-op game
> project. I had to change the scripting language mid-project due to
> performance reasons. With the mailbox approach, this was no problem.
>=20
> See http://worldfoundry.org for more information on mailboxes in World
> Foundry.
>=20
> -Norman

I would like to add that while the mailbox system works really well for
quickly supporting a new scripting language, there are some
disadvantages which make it best used as the first of 2 steps:

There is no type safety since all mailboxes contain the same type of
variable, so a function written in the scripting language which reads a
matrix from the engine cannot tell if the mailbox index you give it is
the first value in a array of mailboxes containing a matrix or not.
The other disadvantage is it can be a bit slow doing all engine IO one
variable at a time.
So I propose that scripting language support be done in stages depending
on the popularity of the scripting language in the application:

(assuming the application already has a mailbox interface)

1. Add the new scripting language by implementing ReadMailbox and
WriteMailbox.
2. Write wrapper functions in the scripting language which abstract the
most common data constructs (in Blender these would probably be thinks
like Vector, Matrix, ObjectList, etc.)
3. At this point it is possible to write useful scripts in the new
scripting language, leave it at this stage and let the wrapper functions
mature and a body of useful scripts accumulate.
4. If/when the language is popular enough and the interface wrappers
mature enough, then add direct support in the application for the most
popular wrappers (implement/replace the wrapper function directly in the
application). This will enable type checking, and be faster than the
individual mailbox accessors, and be completely transparent to all of
the existing scripts.

Step 4 need never be taken if it is fast enough as is, but I like having
some type safety, I have considered enhancing the mailbox system in
World Foundry to support various types, but that would make the simple
case more complex).=20

Of course the variables/functions you attach to mailboxes BECOME your
published interface, so you still have to be careful about what you
reveal.=20


I use this simple -> complex approach in other areas as well, for
example World Foundry has a software 3D renderer which only needs 2D
polygon rendering, so it is quick and easy to port it to a new graphics
platform, then over time I can implement a renderer which leverages the
hardware for that platform (so the software renderer would likely never
be used in a shipping product, but it makes a great test bed/stepping
stone).

--=20
Kevin Seghetti: E-Mail: kts@tenetti.org, HTTP: www.tenetti.org
GPG public key: http://tenetti.org/phpwiki/index.php/KevinSeghettiGPGKey
Check out www.worldfoundry.org for my GPL'ed 3D video game engine

--=-SSHlKI3gHbNNtY8gBtF9
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA+HdPTzHFdenxERVsRAjF0AJoCPCO6ES3qr7dMZlhSaJrWqn/BQACgpHHC
vqIJ74P5Ns6G0fTCI5w2T2w=
=+Cgd
-----END PGP SIGNATURE-----

--=-SSHlKI3gHbNNtY8gBtF9--