[Bf-animsys] BlenRig 5 needs your help :)

[Juan Pablo Bouza]

Wohoo! With Sergey's patch the 20 minutes went down to 3 seconds!! :D

I'll be looking forwards to seeing this applied on master if it doesn't cause any problem!

Ton, well, about the amount of constraints used... I should start by saying that this is because of the total amount of bones used. Victor and Franck have almost 2000 bones each, 1000 for the body, 1000 for the face rig. Surprisingly, it is not so hard to achieve that amount, as many workarounds for the constraint system require bone duplicates. Perhaps when we have everything nodified, Blendini as Bassam and Zanqdo call it, haha, you could just have less than half of those bones.

Rigify has 450 bones, it has a lot of cool features, but if you start adding little extra things you can end up doubling that amount. PitchiPoy's rigify version has 1200 bones. Tara has like 700, the Caterpillar has 900 just for the body, plus 1000 with my face rig...

So with these numbers, if you use 2 constraints per bone, you'll most likely have 2k to 4k constraints in a rig, which luckily, still run at 60 fps in blender :), and hopefully will soo run at the same speed in the  depsgraph refactor branch :D



Date: Mon, 16 Mar 2015 13:45:33 +0500
From: sergey.vfx at gmail.com
To: bf-animsys at blender.org
Subject: Re: [Bf-animsys] BlenRig 5 needs your help :)

- ED_object_constraint_update is not only used by GE now, it's now also used to make viewport aware what color bone should be colored with (IK bone, target bones all sort of jazz). Plus it's also used to make it so constraint list shows constraints without targets as RED_ALERT. Not saying it's really needed to do all actual stuff from that function on every property update, just saying things get's diffused deeper into blender.
- partial updates up depsgraph are quite really nasty and tricky to do. And it's not even a case of slowdown -- depsgraph is rebuilt once after the script is finished. All other operations just tags it for rebuild.- "depsgraph tags itself for rebuild" is somewhat me and Joshua were avoiding. It's just can of worms and things are to be solved elsewhere.
- py calls never forces animation/object/relations update, it's only tagging for update.

Main slowdown here is object_test_constraint() and BKE_pose_update_constraint_flags() being called on every property update (via ED_object_constraint_update). Those functions makes it so for every property update all the bones are traversed, all the constraints of each bone is traversed, plus for each of the bone constraints object list is traversed. That's just crazy stuff which should be just fixed, and for that you don't need new fancy depsgraph even.
All the stuff happening in those functions shouldn't be a part of DAG evaluation since all those flags are corresponding to the relations state, not the scene state and they more belong to DAG_relations_update() routines. After the scene graph is up to date flags set by those functions are never changed anyway. This (setting pose/constraint flags) we can easily do right now with the current depsgraph and new depsgraph will just call the same utility function. It should be pretty straightforward to do based on the patch i've submitted earlier this morning.
Can do it later today/tomorrow if nobody else wraps it up :)
On Mon, Mar 16, 2015 at 1:31 PM, Ton Roosendaal <ton at blender.org> wrote:
Hi,



This is really strange...



Bastien:

- ED_object_constraint_update() was added for the game engine by Benoit, it's code for his robotics IK solver (remove fake constraints). I am not sure if this so nice to keep.

- I really hope no other updates are done by constraint adding code in Python...



Sergey:

- The new despgraph could allow partial rebuilds (add/remove node)

- depsgraph can also tag itself invalid, so it rebuilds on first use.

- py calls should not force animation updates ever btw.



JP:

- explain why you need so many constraints?

- please next time use the bf-animsys list.



-Ton-



--------------------------------------------------------

Ton Roosendaal  -  ton at blender.org   -   www.blender.org

Chairman Blender Foundation - Producer Blender Institute

Entrepotdok 57A  -  1018AD Amsterdam  -  The Netherlands







On 15 Mar, 2015, at 23:03, Bastien Montagne wrote:



> Time explode because until you assign a target, the constraint is invalid (and hence, disabled), so update process is shorter, I guess.

>

> Actually, you could try to set constraints' targets in an ultimate, separated pass, this could makes things a bit quicker…

>

> Le 15/03/2015 22:21, Juan Pablo Bouza a écrit :

>> Thanks for the reply Mont!

>>

>> Yes, I didn't know about the TIMEIT thing, but I tried adding cProfile in every line, hehe. I got the same result as you did, it started in 0.000 and it increased up to 0.025 seconds.... so for the 80k lines of code, that's a big problem.

>>

>> About the amount of constraints, they are about 4000.

>>

>> To add a little bit more of info, the other day with slikdigit, we tried object unlink, then add the constraint, then object link, but it was the same. I also tried running the script with the armature in a layer that was not visible, but again, no change.

>>

>> One thing I do notice is that if I add only the constraints with their names, targets and the slider influence, it takes only 20 seconds. Once the constraints are added, if you start filling in their options, timing is about 0.004 seconds per line, and then it starts increasing and increasing. So my question, if it is of any help, is... why does performance decrease when you start adding info to the constraints, when the constraints are already added. If the 4000 constraints are already present, why does performance decrease with each option you fill in? shouldn't it take a constant amount of time per line, let's say 0.004 seconds, as it updates 4000 constraints each time? What difference does it make that constraints have options in them or not?

>>

>> This rig doesn't have many more constraints than rigify for instance, I wonder why rigify loads in 20 seconds, and my rig in 20 minutes :D

>>

>> 2015-03-15 16:55 GMT-03:00 Bastien Montagne <montagne29 at wanadoo.fr>:

>> Hi Pablo,

>>

>> Issue is actually quite simple, a simple TIMEIT in C code handling constraints update callback (ED_object_constraint_update()) shows that *at the beginning* each run takes about 0.1ms (on my machine)… After only ten constraints have been added, we are already at 0.3ms - imagine what it becomes after 20k constraints!

>>

>> There is nothing you can do from python side (at least, I see nothing).

>>

>> Campbell, Sergey, this is a bit similar to issue we have with adding tens of thousands of objects in a scene from py - update callbacks ran on each and every step end up taking way too much time. Seems like time to think about a way to disable (temporarily) updates from python? Though this could be tricky, guess we'd need some kind of 'stack' to stash all pending updates… :/

>>

>> Or maybe it’s just that constraints update (especially for bones/armatures) are doing way too much things, things that should be deferred to actual DAG execution?

>>

>> Le 15/03/2015 18:19, Juan Pablo Bouza a écrit :

>>

>> Hi guys!

>>

>> After 2 weeks of work and a lot of learning I've successfully managed to translate every single                   component of my rig into python.

>>

>> My idea is to finally be able to release BlenRig as an addon, so I needed to be able to create the rig in a procedural way with python, in order for the user to just press a button and make the rig appear.

>>

>> What I did was to create a rig parser that translates everything into python, some of the benefits I find by doing this are:

>>

>> 1) Being able to update the addon rig by just modifying my rig in blender and then running the rig parser over it (so no need to manually update the python code for creating the rig).

>> 2) Being able to easily add more rig types, like male, female, quadruped, different setups for face rigs, etc

>> 3) Being able to eventually create a system where you can add more limbs, heads and other stuff to the rig by just parsing the body parts.

>> 4) Being able to release BlenRig-Vehicles, with a whole feature set for cars and motorcycles I did for Kiribati.

>>

>> With that said, I've found a really disturbing bottleneck in the creation of constraints. Ideasman told me to use cProfile, but I wasn't able to figure out how to use it more than for obtaining the overall time of execution (you know I'm a programming ignorant :))

>>

>> Well, I'm attaching a blend with the rig in it and a script called "add_bone_constraints.py" in it. On my i7 computer, the whole process of adding the constraints take 20 minutes!!!!!!!!!

>>

>> The code is pretty basic as you will see... I can't do much more. But well, constraints are the only bottleneck I found, cause creating the whole bone rig, adding bone properties, ID properties, custom bone shapes and drivers, are all operations that take less than a second...

>>

>> >From some tests I did, I noticed that only creating the constraints doesn't take long, 3 seconds or so. The problem comes when you start adding the different constraint options and parameters. I don't really know what blender is doing here, maybe you find some kind of refresh bug, or maybe you'll be able to give me a magical line of code that will increase speed! :D

>>

>> Just in case you want to see where my constraint code comes from, in the blend file I also included a script called "parse_bone_constraints.py". If you run it on the armature that is in Layer 10, you'll obtain the lines for constraints creation.

>>

>> Well, I hope someone can help me out with this, I'm not in a big hurry, as I have a couple of months ahead before I can do a proper release, but addressing this early show stopper would be ideal :)

>>

>> Cheers! And sorry for the inconvenience!

>>

>>

>



_______________________________________________

Bf-animsys mailing list

Bf-animsys at blender.org

http://lists.blender.org/mailman/listinfo/bf-animsys



-- 
With best regards, Sergey Sharybin


_______________________________________________
Bf-animsys mailing list
Bf-animsys at blender.org
http://lists.blender.org/mailman/listinfo/bf-animsys 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/bf-animsys/attachments/20150316/5ccef34b/attachment.html>