[Bf-committers] FM 2.8 Proposal : Fracturing, Cache, Rigidbody

[Brecht Van Lommel]

Thanks for the explanation. Overall I agree with the approach.

It would be very helpful if this kind of functionality was submitted for
review and integration in smaller parts. For example:
* Change rigid body simulation to work as a modifier.
* Add Alembic support to point caches for physics systems. (could even
replace custom point cache format)
* Add fracture modifier that fractures a mesh, without any physics.
* Integrate fracture modifier with rigid body physics.

But let's see what developers say about the design.

On Mon, Feb 26, 2018 at 5:59 PM, Martin Felke <martin.felke at t-online.de>
wrote:

> Yes, I want to have something like a collection of mesh islands over
> either one shattered Mesh Datablock, or even outsourcing the "generated"
> data to a geometry cache, which is written  to an Alembic file for
> example and being read back by something similar like the meshsequence
> cache modifier.


I would expect a (Derived)Mesh to be in memory, which can then the
read/written to/from a cache file. The cache (Derived)Mesh would not be
saved in the .blend if it's backed by a cache I think, since other physics
systems don't do it either.


> Each of those islands should be an "Item" in the cache.
> And the rigidbody data should be "attached" per island like custom data
> is with vertices, edges, loops, polys. I find the custom data concept
> quite interesting for that usecase as well.

If it is ok to keep one regular (DNA) stored Meshdatablock as fracture
> result in the background, being "overwritten" by new fractures,
> also each poly could carry a custom data layer with an mesh island
> index, means each poly knows to which island it "belongs".
> With the help of this minimal storage one could at load time build a
> runtime collection of mesh islands in the modifier or more generally
> spoken, in a geometry cache component.
> Additionally to that, since constraint buildup is relatively fast, those
> dont need to be stored at all anymore, but being rebuilt at load time.
>

This all seems like an internal implementation detail of how the mesh and
island data is written to the Alembic file. If it's simpler to make it
generic then ok, if not no need to make it more complicated.

Alternatively, to having all in one Object, is it planned to have

something like Collection Modifiers ? where a collection of objects is
> being treated as an object itself and could be fed into the modifier
> stack and where the modifier emits a modified collection
> (which is treated as one object / ID again ?)

Something similar to this could be some kind of "Pack Geometry"
> functionality, which not just joins all into one mesh, but keeps
> separate object's settings separate and individually changeable ?
> Like some "container" object ?
>

No concrete plans as far as I know, that would be something to consider
when adding node based modifiers or procedural object/geometry generation.


> But i guess just making a fractured mesh with island info as poly layer
> could be the most straightforward approach.
> The "Convert" button in fact exists already for the FM, making each
> meshisland a separate object. (to simulate the objects in other blender
> builds for example)


Let's focus on this first.


> Sorry for the confusion about "cross - object constraints". Those are a
> relatively new FM feature and allow to "glue" islands together across
> nearby / adjacent fractured objects. Currently an extra object
> referencing a group of to-be-glued objects thru the FM is responsible to
> just maintain constraints between islands of the fractured objects
> within that group. I mentioned this for the sake of completeness only,
> this is advanced functionality. (Here one object respectively
> one FM can hold many islands and many constraints, in the current
> design.) The "dedicated" modifier is just a "normal" FM with special
> settings in this case.
> I think I should further go into more advanced functionality only after
> the basic concepts are clear / being agreed on. But OTOH i also wanted
> to make sure the "basic" design is "powerful" enough to be able to deal
> with the more advanced things which will build upon it.


Ok, makes sense.

Ok, agreed :) It is a very complicated topic. I just tried to think out
> of the box since the existing Pointcache cannot dynamically grow and
> shrink (to my understanding, the total number of "Points" is fixed atm)
> and this would allow other advanced features like dynamic fracturing
> for example.
>

Right, fracture would require adding support for Alembic as a file format
in the point cache system. Smoke sim already has support for OpenVDB and a
custom cache file format as well, I think Alembic could be similar, keeping
the system and UI mostly the same but adding Alembic.


> I have a "customized" solution for this currently in the FM. The FM
> basically stores a sequence of meshisland lists currently, which is
> extended at each fracture event. Furthermore each meshisland stores its
> own "motion history" (locations, rotations) per frame (as long as the
> island exists, e.g. it disappears if it is fractured further.). The
> problem with this is that i currently duplicate unchanged mesh islands
> across several fracture "states" and this eats a lot of RAM.
> It might not be absolutely necessary to integrate everything in one go,
> but again I wanted to make sure not to "forget" something important in
> the initial design, which may backfire later.


Writing this data to a cache file seems like a good solution yes, Alembic
can do smart deduplicating of data, though I'm not familiar enough with it
to know how islands would work.

Regards,
Brecht.


> On Mon, Feb 26, 2018 at 1:20 PM, Martin Felke <martin.felke at t-online.de>
> > wrote:
> >
> > > Hi, first attempt to get my thoughts / ideas sorted on how to move on
> > > with FM in 2.8.
> > >
> > >
> > > FM 2.8 Proposal : Fracturing, Cache, Rigidbody
> > > ==============================================
> > >
> > > Fracturing:
> > > ----------
> > > Main Goal is to avoid 1000s of real objects, this still causes serious
> > > lags in blenders viewport. Should this be already fixed we could just
> > > use those and we are almost done.
> > > We need to decompose the  base or previous modifier mesh to islands,
> > > with an Operator for example. Now we have an Execute operator which
> > > “invokes” the modifier and suppresses modifier eval afterwards again.
> > > Result of the fracture operator is a non-persistent Mesh
> (pre-fractured)
> > > or a non-persistent Mesh Sequence (dynamic). We have the possibility to
> > > apply to a single mesh object or to convert to multiple Mesh Objects.
> > > It should be kept as a modifier for (nondestructive ?) interaction with
> > > the stack or nodes, else it would be “nailed” to the top of the stack,
> > > no modifiers before could be placed before it.
> > > Optionally we could  have a  more general pack operator which packs a
> > > group of objects and constraints (similarly to how it happens already
> > > with FM external mode)
> > >
> > >
> > > Cache:
> > > -----
> > > The fracture result should be kept in the RAM (Geometry Cache). It also
> > > could directly be written to a persistent Cache File.
> > > Alembic could be used as cache file / disk cache for geometry. Those
> > > per-modifier cache files could be packed into the blend, so the user
> > > doesn’t need to take care not to “forget” files when sharing the
> > > blend.
> > > It would be needed to feed the alembic file backend differently with
> > > data. It needs to be avoided that mesh gets fully dumped again and
> again
> > > each frame, because this will unnecessarily bloat the file size.
> > > Instead, we should treat each island like an object and store only
> > > transforms over time.
> > >
> > > A Fracture Modifier should hold a cache data block similar to the mesh
> > > sequence cache modifier. So each of those objects and / or modifiers
> > > would be linked to a disk cache file.
> > > During fracturing, the initial state will be written to a geometry
> cache
> > > file.
> > > During simulation, another file will be created and the initial state
> > > from simulation will be modified, by writing transform info.
> > > For the case of dynamic fracture, the cache needs to be able to extend
> > > or shrink (modify) the initial state. At each fracture event we need to
> > > add a new state to some event sequence. But it needs to be avoided to
> > > repeat redundant info as in writing the same geometry twice.
> > >
> > > Generally, we need to make the storage more dynamic, so the cache can
> > > grow and shrink, so instead of just storing an array of "points" per
> > > frame,the cache could be holding just a sequence of items where each
> has
> > > its own frame range.
> > > To ensure quick lookup then (state of cache at frame x) we could keep
> an
> > > interface similar to the old point cache.
> > > Going over all elements and linear search is just too slow with high
> > > element count, so jumping in cache and even sequential playback could
> be
> > > too, if we don’t know what is the “next” changeset.
> > > We could build up a lookup structure which works like an array of
> > > frames. This references the items according to whether this frame is
> > > part of their frame range.
> > > Building that structure could be done at sim start and at addition /
> > > removal of items in dynamic sim too. This helps to ensure fast playback
> > > and jumping at the expense of some overhead in dynamic simulations. But
> > > simulation generally is expected to be slower than cache playback, so
> > > the overhead should not be the main cause of slow performance.
> > > Each cache item should be able to hold any type of “custom” data. so
> the
> > > same cache structure could be used for any type of simulation purpose.
> > > Blenders Custom data Layer system could be applied to each of those
> > > cache items. It allows to append already “simple” datalayers like int,
> > > float, string, bool. Optionally we could attempt to extend it to hold
> > > pointers to arbitrary structs. We should be able to register new custom
> > > data types at runtime too, and expose the cache to python. (so addons
> > > could read / write from / into it)
> > > For example, the Rigidbody Object structure should be treated as custom
> > > data layer for the cache. Each island would be an item and index into
> > > the rigidbody layer.
> > >
> > >
> > > Rigidbody:
> > > ---------
> > > The rigidbody world would reference all objects which have caches with
> > > rigidbody custom data in it, with Fracture Modifiers referencing their
> > > cache datablocks, and their items.
> > > Access would be like : World -> Objects -> Caches per Object -> Items
> ->
> > > RigidBodyObjectLayer[Item.Index]
> > >
> > > The constraints should be managed by the modifiers as well, where we
> > > have “inner” and “outer” constraints then, the latter are cross-object.
> > > The cross-object constraints will be managed by a dedicated
> > > constraint-only Modifier. Each of those constraints should hold
> > > references to rigidbodies (or cache items, since a rigidbody is just a
> > > layer then) of the same or of the other participating objects.
> > > The old "regular" Object based rigidbodies shall be removed, they can
> be
> > > replaced by single-island modifiers.
> > > Rigidbody as modifier becomes movable in the stack / among nodes, and
> is
> > > not nailed to final / deform / base.
> > >
> > > Additional functionality like autohide and automerge could become
> > > separate modifiers, unless they also need to access cache details.
> > >
> > >
> > > Further Thoughts
> > > ----------------
> > >
> > > Should EACH modifier have a cache component, which is either being
> > > passed down the stack, and can be accessed externally too ?
> > > like ob.modifiers[“mod”].cache.layers.xxx ?
> > >
> > > A generic script modifier would be nice (where a certain python class
> > > could be attached to), could cooperate with the exposed cache component
> > > (read from input cache, write to output cache)
> > >
> > > Caches per modifier would also allow to stack multiple fracture
> > > modifiers without performance penalty (of continuous refractures)
> > > (would work in the old style DNA / DM storage too, but many places in
> > > the code assume there is only one FM, like the duplicated rigidbody
> > > code)
> > >
> > > _______________________________________________
> > > Bf-committers mailing list
> > > Bf-committers at blender.org
> > > https://lists.blender.org/mailman/listinfo/bf-committers
> > >
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > https://lists.blender.org/mailman/listinfo/bf-committers
>
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> https://lists.blender.org/mailman/listinfo/bf-committers
>