[Bf-committers] GSoC 201 - Automatic Skeleton Extraction

Patrick Shirkey pshirkey at boosthardware.com
Tue Apr 30 21:18:05 CEST 2013


On Wed, May 1, 2013 3:48 am, Nathan Vegdahl wrote:
>> It would look like makehuman
>
> Eh... I think I may be a bit confused.  The proposal is about
> automatically generating skeletons for an arbitrary model, but it
> sounds like the workflow you outlined doesn't automate that step at
> all.

The skeletons in MH are applied automatically to the mesh. The same
principal can be used for any mesh.

> What you've outlined is still cool, but it sounds orthogonal to
> what is currently being discussed, unless I'm misunderstanding
> something...?
>
> Moreover, unless I'm mistaken, Makehuman is specific to bipedal
> characters with roughly human characteristics.  Whereas the intent of
> the original proposal (unless I'm misunderstanding it) is to function
> for any morphology.
>

That was not clear to me. If it is for any shape then it will be a more
complex procedure to automate.

Either way the makehuman UI deals with the problem pretty damn efficiently.

1: Create mesh
2: Select skeleton from list of several options

Can it be made easier than that?

Or maybe the question should be is it worth the time and effort to develop
the code to automatically create a rig for any shape possible? I don't
know so I am genuinely interested in hearing of reasons for the original
poster to use his valuable time on such a task.

Creating an open archive of rigs for specific mesh shapes and using the
"makehuman" method to apply them to a model may be a better use of time
with a higher ROI.



> --Nathan
>
>
>
> On Tue, Apr 30, 2013 at 12:18 AM, Patrick Shirkey
> <pshirkey at boosthardware.com> wrote:
>>
>> On Tue, April 30, 2013 12:54 pm, Nathan Vegdahl wrote:
>>> For something like this to be really useful, I think it needs to be
>>> user-guide-able.  And even then, it would mostly be useful for
>>> background characters, not hero characters (which need fine tuning by
>>> an artist).
>>>
>>> I can imagine this being useful, for example, for fully automatic
>>> rigging of a crowd's worth of background characters.  But it would
>>> need to take guidance from the user regarding the style and morphology
>>> of the characters, and the topology, naming scheme, and alignment of
>>> the skeleton to be generated.  And honestly, I haven't a clue what
>>> that would look like.
>>>
>>
>> It would look like makehuman
>>
>>> In general, building base skeletons is not the bottleneck of rigging
>>> characters anyway.  And even if you want to make that faster, just
>>> having better/faster tools for doing it manually would get you more
>>> bang for your buck than automatic generation, I suspect.
>>>
>>
>> Animating models is the current bottle neck. This is how I have achieved
>> it with a limited amount of success:
>>
>>
>> 1: Create mesh and apply xonotic rig in makehuman
>> 2: export model and rig to mhx with "Feet on Ground"
>> 3: open blender
>> 4: install iqe/qim exporter
>> 5: install makehuman tools and mhx importer
>> 6: import mhx to blender
>> 7: export iqe of model/rig from blender
>> 8: find and download original official xonotic model blend file
>> 9: Open with blender
>> 10: export model/rig/anims to iqe
>> 11: open both iqe files in a text editor
>> 12: extract anims from official xonotic models iqe file
>> 13: add anims from official xonotic model to makehuman xonotic model
>> 14: save mh iqe file
>> 15: build iqm convertor tool from source
>> 17: run iqm convertor tool on mh iqe file to create animated iqm file
>> 18: import iqm file to external 3d game engine (cube2)
>>
>> cube2 import steps
>> 19: Place xxx.iqm file in packages/models/xxx  folder where xxx is the
>> name of the model
>> 20: add iqm.cfg file and edit with correct commands for loading the iqm
>> 21: add the model name and location to the map.cfg file
>> 22: load the map in coop edit mode
>> 23: load the model with the "newent mapmodel i" where i is the numerical
>> position of the model in the model list
>>
>> additional steps for using animated model in cube2
>>
>> 24: make changes to some hardcoded function variables to allow loading
>> new
>> animated playermodels
>> 25: add new playermodel to the game config
>> 26: load animated model as a playermodel or bot in coop edit mode
>>
>>
>> End result:
>>
>> http://iguanapro.com/cube/mh4-2.png
>> http://iguanapro.com/cube/mh4-3.png
>> http://djcj.org/video/goblin-demo.mpeg
>>
>>
>> It's a very manual process and not very workable if building a crowd of
>> models in the thousands or hundreds of thousands like for ex. Massive
>> from
>> Weta can do
>>
>> There are also issues with scaling and position of the center of origin
>> between models. In this specific case cube2 doesn't (obviously) support
>> changing the center of origin of a model so it has to be handled before
>> exporting from mh/blender. I could not find an obvious method or
>> tutorial
>> for changing the center of origin on a blender model. MH allows changing
>> it at export from origin at hips to feet on ground.
>>
>> To cutdown on a lot of manual steps it looks like the way to go is to
>> add
>> an iqe exporter to MH and
>> export the blend animations/rigs to bvh for inclusion in MH assets
>> archive. Then everything can be applied in MH before exporting to iqe
>> cutting out around 10 steps in the process. FYI, I am working on the MH
>> iqe exporter at the moment.
>>
>>
>> --
>> Patrick Shirkey
>> Boost Hardware Ltd
>>
>>
>>
>>> --Nathan
>>>
>>>
>>> On Thu, Apr 25, 2013 at 9:02 AM, Alexander Pinzon Fernandez
>>> <apinzonf at gmail.com> wrote:
>>>> Automatic Skeleton Extraction
>>>>
>>>> Alexander Pinzon Fernandez
>>>>
>>>> Email: apinzonf at gmail.com
>>>>
>>>>
>>>> Short description:
>>>>
>>>> The character animation is generally done with skeletons that control
>>>> each
>>>> joint in the character which drastically simplifies the animation
>>>> process.
>>>> The construction of the skeleton given a polygon mesh is a task
>>>> typically
>>>> performed by an artist. This project proposes to implement a method
>>>> for
>>>> extracting the skeleton automatically given a polygon mesh.
>>>>
>>>>
>>>>
>>>> *Name*: Alexander Pinzon Fernandez
>>>>
>>>> *Email*: apinzonf at gmail.com
>>>>
>>>> *IRC*: apinzonf
>>>>
>>>> *www*: http://apinzonf.wordpress.com/,
>>>> http://wiki.blender.org/index.php/User:Apinzonf
>>>>
>>>> *Additional Contact Info*
>>>>
>>>> *Phone*: +57 300 2 92 74 30
>>>>
>>>> *Physical Address*: Cr 81 H No 42a-58 sur Bogota-Colombia
>>>>
>>>>
>>>> Synopsis
>>>>
>>>> The character animation is generally done with skeletons that control
>>>> each
>>>> joint in the character which drastically simplifies the animation
>>>> process.
>>>> The construction of the skeleton given a polygon mesh is a task
>>>> typically
>>>> performed by an artist. This project proposes to implement a method
>>>> for
>>>> extracting the skeleton automatically given a polygon mesh.
>>>>
>>>> This method iteratively contracts the shape in the reverse direction
>>>> of
>>>> the
>>>> curvature flow, with a constraint that forces the contracted model
>>>> will
>>>> closely match the original. In each iteration eliminates details.
>>>> After
>>>> the
>>>> model volume is close to zero is passed to the next stage in which
>>>> through
>>>> a special surface simplification method removes vertices to obtain a
>>>> 1D
>>>> structure.
>>>>
>>>> Paper used:
>>>>
>>>> AU, Oscar Kin-Chung, et al. Skeleton extraction by mesh contraction.
>>>> ACM
>>>> Transactions on Graphics (TOG). ACM, 2008. p. 44.
>>>>
>>>>
>>>> Benefits to Blender
>>>>
>>>> This project proposes a new tool for blender user that requires
>>>> extract
>>>> the
>>>> skeleton of polygon meshes.
>>>>
>>>> This method allows to obtain the skeleton of a polygon mesh quickly
>>>> and
>>>> automatically.
>>>>
>>>> This method is insensitive to noise because it uses a smoothing
>>>> algorithm
>>>> for contraction which removes the noise and detail in each iteration.
>>>>
>>>>
>>>> Deliverables
>>>>
>>>>    1. A new skeleton extraction tool for Blender.
>>>>    2. Some pages of documentation to be included in the manual
>>>>    3. A technical document for developers to improve the method in the
>>>>    future.
>>>>    4. A tutorial explaining the use of the tool.
>>>>
>>>>
>>>> Project Details
>>>>
>>>> The project would divide into seven parts:
>>>>
>>>>    1. Iteratively contracting the mesh until the volume is close to
>>>> zero.
>>>>    2. Simplifying polygon mesh to obtain a 1D structure (skeleton).
>>>>    3. Relocating skeleton nodes in the center of each segment.
>>>>    4. Create vertex groups for each node of the skeleton.
>>>>    5. Converting a 1D polygon mesh into an armature.
>>>>    6. Link vertex groups to bones of the armature.
>>>>    7. Generation of the documentation and tutorials.
>>>>
>>>>
>>>> Project Schedule
>>>>
>>>>    - *2 Weeks:-           May 27 – Jun 09:* Iteratively contracting
>>>> the
>>>>    mesh until the volume is close to zero.
>>>>    - *3 Weeks:-           Jun 10 – Jun 31:* Simplifying polygon mesh
>>>> to
>>>>    obtain a 1D structure (skeleton).
>>>>    - *2 Weeks:-           Jul 01 – Jul 14:* Relocating skeleton nodes
>>>> in
>>>>    the center of each segment.
>>>>    - *2 Weeks:-           Jul 15 – Jul 28:* Create vertex groups for
>>>> each
>>>>    node of the skeleton.
>>>>    - *3 Weeks:-           Jul 29 – Aug 18:* Converting a 1D polygon
>>>> mesh
>>>>    into an armature.
>>>>    - *2 Weeks:-           Aug 19 – Sep 01:* Link vertex groups to
>>>> bones
>>>> of
>>>>    the armature..
>>>>    - *1 Weeks:-           Sep 02 – Sep 08:* Testing the tool and
>>>> Define
>>>> and
>>>>    implement graphical user integration.
>>>>    - *2 Weeks:-           Sep 09 – Sep 22:* Generation of the
>>>> documentation
>>>>    and tutorials.
>>>>
>>>>
>>>> Bio
>>>>
>>>> I graduated as systems engineer in Colombia in 2007.
>>>>
>>>> I am a MSc computer science student at National University of
>>>> Colombia.
>>>>
>>>> Skeleton extraction and mesh smoothing are the research topics of my
>>>> MSc. I
>>>> am use CGAL, Graphite and Qt libraries.
>>>>
>>>> Since 2007 I am a member of the Bioingenium Research Group of National
>>>> University of Colombia.
>>>>
>>>> I have been using Blender for about 10 years. I was working on the
>>>> develop
>>>> of a operator to remove noise from a mesh at GSOC 2012 – Blender suchi
>>>> branch “Mesh Smoothing for 3d Scan Data”.
>>>>
>>>>
>>>> WORK EXPERIENCE:
>>>>
>>>> 2012 Google Summer of Code – Blender Foundation: “Mesh Smoothing for
>>>> 3d
>>>> Scan Data”.
>>>>
>>>> 2010 - Scire Foundation, Software Architect:
>>>>
>>>> Design and development of web services data, hibernate and oracle.
>>>>
>>>> Programming GUI with RichFaces.
>>>>
>>>> Library of generic manipulation of trees, based on Java reflection
>>>> techniques.
>>>>
>>>> 2005 - 2007  Sigtech, LTDA, Developer Engineer:
>>>>
>>>> SIGC  System  Development  for  the  "Departamento  Administrativo  de
>>>> Catastro  Distrital DACD".
>>>>
>>>> Software used in development: Oracle 9i, Visual Basic 6, PL / SQL
>>>> Developer.
>>>>
>>>> 2004 -2005  IIE and Colciencias, Student Monitor research:
>>>>
>>>> Project: " Theater of Memory in Virtual Worlds:"
>>>>
>>>> Architecture for handling streaming video that comes from a digital
>>>> camcorder and put it as a texture of a 3D plane in a virtual setting,
>>>> using
>>>> the Java software, J3D, JMF, VRML, X3D.
>>>>
>>>>
>>>> SOFTWARE DEVELOPED:
>>>>
>>>> Nukak3D  http://sourceforge.net/projects/nukak3d
>>>>
>>>> 3D medical image platform for visualization and image processing.
>>>>
>>>> Nukak3d  is a  flexible architecture  that  integrates general-purpose
>>>> graphics  libraries such as VTK, ITK, VTKInria3D, OpenGL, under a
>>>> graphical
>>>> user interface (wxWidgets)  for three-dimensional visualization and
>>>> processing of medical images. On C++.
>>>>
>>>>  JNukak3D  http://sourceforge.net/projects/jnukak3d
>>>>
>>>> Software for three-dimensional visualization of DICOM images based on
>>>> java.
>>>>
>>>>   JVC1394  http://sourceforge.net/projects/jvc1394/
>>>>
>>>> Wrapper for the library Libdc1394 to enable communication with the
>>>> Java
>>>> programming language. Java Video Capture for IEEE 1394 cameras. On C,
>>>> C++,
>>>> Java, Swig.
>>>>
>>>>
>>>> PUBLICATIONS
>>>>
>>>> León J., Pinzón A., Sánchez C., Romero E. A distributed plugin based
>>>> architecture for medical image processing. Proc. SPIE 8674, Medical
>>>> Imaging
>>>> 2013: Advanced PACS-based Imaging Informatics and Therapeutic
>>>> Applications,
>>>> 867403 (March 29, 2013);doi:10.1117/12.2007180.
>>>>
>>>> Pinzón,   A., Leon, J.C., Romero E.,   Diseño y desarrollo de una
>>>> aplicación web para  la visualización eficiente de imágenes medicas,
>>>> con
>>>> procesamiento 3D por software o por hardware. En Seminario
>>>> Internacional
>>>> de
>>>> Procesamiento y Análisis de Imágenes Médicas SIPAIM 2009, (Noviembre
>>>> 27-27,
>>>> 2009), Bogotá Colombia.
>>>>
>>>> Pinzón  A.,  Romero  E.,  Visualización  3D  de  imágenes  médicas:
>>>> una
>>>> herramienta  Open Source,    En Seminario Internacional de
>>>> Procesamiento
>>>> y
>>>> Análisis de Imágenes Médicas SIPAIM 2008, (Noviembre 24-27, 2008),
>>>> Bogotá –
>>>> Colombia.
>>>>
>>>> Mendoza,  B.  U.,  Ramos,  G.  A.,  Mendez,  L.  M.,  Santamaria,  W.,
>>>> and  Pinzón,  A.  2006.  Camera  Motion  Control from  a  Java  3D
>>>> Environment:  Virtual  Studio  Application  in Decorative  Arts Museum
>>>> Collections.  In Proceedings of  the 2006  international Conference on
>>>> Cyberworlds  (November 28  - 29, 2006). CW. IEEE Computer Society,
>>>> Washington, DC, 58-64.
>>>> _______________________________________________
>>>> Bf-committers mailing list
>>>> Bf-committers at blender.org
>>>> http://lists.blender.org/mailman/listinfo/bf-committers
>>> _______________________________________________
>>> Bf-committers mailing list
>>> Bf-committers at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-committers
>>>
>>
>>
>> --
>> Patrick Shirkey
>> Boost Hardware Ltd
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>


--
Patrick Shirkey
Boost Hardware Ltd


More information about the Bf-committers mailing list