[Bf-committers] Proposing a unique ID for Blender objects system, for use with game engines.

[Lissanro Rayen]

On 16/11/20 7:00 pm, Brecht Van Lommel via Bf-committers wrote:
> To me a UUID makes sense when you want to make a connection between two
> completely different databases. Like Blender and a game engine, or an asset
> manager that manages data from multiple applications. Any time you have to
> do that kind of syncing and no longer have a single source of truth, it's
> not an ideal situation in the first place, but UUIDs do help.

I recently encountered a need for UUID when writing an addon. In my case 
I did not need to manage data between different applications, but I 
needed a way to keep track of objects/materials/nodes between sessions 
(even if they were renamed), so pointers were not an option. Besides, 
pointers can become invalid after undo/redo, even though the object 
itself still exists and was not changed. Names may not be unique either 
- for example, top-level compositor node and a node within a group may 
have exactly the same name. Even if it was not so, the user can change 
names at any moment.

I was not able to find any other solution, and I ended up implementing 
UIDs for objects, materials and nodes. For my purposes, 32-bit UIDs 
(integer hashes) were sufficient, with a check to guarantee uniqueness 
within single .blend. I also implemented functions to get a pointer to 
object/material/node by its UID, and each object has .uid_get() 
function. UID is generated for the first time when this function is 
called (there is a check to make sure that new UID does not collide with 
already existing ones).

128-bit UUIDs could be implemented in similar way. It would be much more 
efficient and cleaner if UUIDs were natively supported.

> It can be misused as well. For example I've heard requests for this so
> add-ons can store relations to Blender datablocks at runtime, or saved in
> .blend files. To me that would be a mistake, as references to datablock
> should be done explicitly with pointers that are managed by Blender, that
> can be properly referenced counted, cleared, replaced, etc. Also for
> library linking datablocks between .blend files this is tempting, but again
> it also comes with it own set of issues.

In practice objects/materials are often referenced by their names, at 
least this was very common when I was looking for examples how to save a 
list of objects/materials/nodes, or how to keep track of an object 
chosen by the user between sessions. I agree that pointers should be 
used when possible, but I could not find a way to reliably get them 
without unique identifiers.

In my addon I have a list of pointers. Occasionally (for example, 
because of undo/redo) some or even all of them become invalid, then at 
the moment I hit invalid pointer, I can get new pointer by UID, and then 
the rest of the code works as expected even if name of 
object/material/node was changed.

> If this were to be implemented, it does raise some questions:
> * Would we want to do this for every datablock? Presumably not because the
> memory overhead and cost of generating a UUID, so I guess it would be
> something that is generated on demand.
In my specific case I implemented UIDs only for objects, materials and 
compositor nodes, but in general case, there could a use for unique 
identifier for any datablock. Indeed, generating UUID on demand is the 
best and most efficient way.
> * Are UUIDs supposed to be unique within one .blend file, or globally
> unique across all .blend files? What happens when you copy or rename a
> .blend file, do the UUIDs change?

It would be impossible task to make them truly unique across all .blend 
files, there are always a possibility of collision. But according to 
https://en.wikipedia.org/wiki/Birthday_problem, for 128-bit hash 
probability of collision is relatively low.

If UUID supposed to uniquely identify a datablock, I think it should not 
change if .blend file is renamed, since renaming the file does not 
regenerate the datablock, it stays exactly the same. However, if there 
are two copies of the same .blend file (they may even have the same name 
and even the same full path on two different PCs) and each was edited 
independently, and then objects from both .blend files are appended to a 
single .blend file, there may be a lot of UUID collisions. This could be 
solved in a simple way - if imported datablock already has UUID, make 
sure it is unique when appending to current .blend. If it is not unique 
then drop UUID (new one will be generated on demand). The same is true 
for duplicated objects (or other datablocks). This way datablocks which 
already existed in .blend file will keep their UUIDs for their lifetime 
(or since the moment UUID was first generated, if this was done on demand).

> * Do overrides copy the UUID or generate a new one? I'm guessing it should
> generate a new one.

Yes, I also think that it would be appropriate to generate new one in 
this case.

> * What about datablocks that are the result of geometry nodes evaluation?
> How does this UUID remain stable if geometry nodes can output an arbitrary
> number of objects, that might even vary over time?

In this case UUIDs may not be "stable" - I think if an object was 
generated again (and this was not because of redo), then it is a 
different object, so it may not have UUID initially at all - new UUID 
can be generated on demand. Then UUID may end up having lifetime of a 
pointer. I did not try geometry nodes myself, but I guess at least 
parent geometry node itself could have stable UUID.

UUIDs have limitations. The way I see them - they are sort of like 
unique name which cannot be changed by the user (only automatically 
generated and once generated, will not change unless there is a collision).

> * How does this relate to the asset manager design? There was some
> discussion of having UUIDs for that though no conclusion. If it is needed
> there, would that UUID be the same?

Few weeks ago when I was googling for existing solutions, I noticed that 
idea of UUID is not new and was mentioned in context of asset manager 
but as far as I understand it was not actually implemented. I'm 
unfamiliar with asset manager design, but I think it is better to keep 
UUIDs as unique as possible. The simplest way to do that is to generate 
them randomly. For example, in my implementation I generate unique 
random hashes to minimize chance of collisions (so even if two nodes 
have the same name and .get_uid() was called for the first time for both 
of them almost in the same moment, they will end up with different 
unique hashes as UID).