[Bf-vfx] Blender data, big picture

[Keir Mierle]

On Mon, May 9, 2011 at 10:47 AM, Ton Roosendaal <ton at blender.org> wrote:

> Hi,
>
> Here's notes I wrote in gsoc review days, suggestion for Blender work
> Sergey can do. There's no libmv specific info, this is (was) unknown
> for now. Dark matter, but fits in design :)
>

Most of the blender matchmoving code should be libmv-independent. That's
just good engineering!


> (Note: in Blender we separate ID data from 'indirect' data. The first
> has option to be linked and used everywhere, appended/linked from
> files, used in nodes, etc).
>
> Tracker integration
>
> 1) New ID: MovieClip
>

Sorry to be stupid, but what's a "New ID"? Is this a new object type like
Camera or Mesh?


> - better definition of how sequence of images works
>

I'm not sure what this means. Are you suggesting fleshing out the concept of
a MovieClip is? For example, resolution, number of frames, file name, etc?


> - ffmpg movie file support
>

Doesn't blender already have a nice system for accessing movie sequences? I
remember using this some 4 years ago for my thesis.


> - proxy level support (like 4k -> HD)
> - proxy level with files as well as making levels in memory
>

While proxying is great, I'm not sure we'll need it for the first version. I
will ensure the design does not preclude adding this in the future.


> - optimized for real time playback
>

 We definitely want realtime playback!


> - Here tracking points (markers) can be stored, track paths, fcurves,
> whatever structs needed for libmv refining as well
>

Yes, this is reasonable.


> - 3d camera info stored? Lens (un)deform info
>

Sounds reasonable. We likely don't want to support solving for a single
camera across multiple shots.


> - Need good mapping from marker results to blender scene data
> (objects, drivers?)
>

This is where an hour or so of quality design time from the core blender
devs will save us time. Can you suggest either a reasonable approach here or
at least a couple paths to investigate? There are some challenges here since
the libmv side has limited experience with the blender internals. If you can
be overly explicit in your descriptions of blender internals that would be
fantastic!


> - Solve how to work with masks here
>

I propose entirely punting on masks for now. Secretly I hope we will deliver
the most-basic-tracker-ever ahead of schedule (October?), and in that case,
we can work on masking. However, I think it will be a distraction otherwise.


> (reasoning why markers are not "ID": mostly because it's fully tied to
> a movie clip itself, no sharing is required. Via MovieClip ID markers
> can be addressed and used).
>

I'm not entirely convinced about this. How will the markers positions be
stored? I would prefer if they used blender's internal fcurves system, but
doesn't that already suggest that they should be full objects themselves?

Does blender have a way of grouping objects such that the MovieClip "owns"
the markers / bundles but that they are still full fledged objects?


> Future:
> - allow MovieClips in sequencer
> - MovieClip node in compositor
>

This is required. At the least, there must be a way to render out the
composited scene and render. It's hard for me to imagine a practical system
that doesn't have this feature.


> - Allow type "From composite tree"
>   (stores or links node tree)
>

Sounds great, for next year :)


> 2) New editor: SpaceClip
>
> - links to MovieClips
>

By links do you mean has a dropdown that lets you select a movieclip? What
about a "add movie..." button?


> - allow realtime playback of clips
>   (could become "Render -> Play Anim" as well)
>

This will require a "scrubber" so that you can scrub back and forth. This is
a simple but necessary feature for practical semi-manual tracking.


> - Design region UI for libmv options, track point editing, solving,
> filters, etc
>

Yes, and this will be some work.


> - Manage & edit markers, masks
>

Let's punt on masks.


> - Various display options, like to see 3d scene in back blended/
> overlayed


The view-3D-scene-over-clip is required. Otherwise it will be difficult to
asses the quality of a solve within the matchmoving window.


> Future:
> - Color grading options!
>   - extract pixel-masks from movieclip (various color formulas)
>   - use such masks for effects... via nodes? layers?
>
> To think over:
> - Painting here, or in Image window?
>   How to manage paint-on-sequence (fast)
>   Tracking paint strokes...
>
> (reasoning why new editor: more practical for this experimental stage,
> also allows to create a full tracker-workflow unique UI layout without
> ugly modes in other editors).
>

I agree that we want a new editor. Gluing tracking functionality into
existing ones will just make a mess of both.


> 3) New Image options:
> - Procedural image type "video stream" -> allow playback of connected
> video device (video texture)
>   (no paint allowed, not for textures in render)
> - Allow image type "From composite tree"
>   (stores or links node tree)
>
> 4) new ID type CurveMask
> - 2D in unified 0-1 space, so it maps everywhere
> - editing allowed in compositor, spaceClip, etc
> - curveMask compo nodes, texture, sequence strips...
>

Can you expand on what this CurveMask is and is for? Perhaps I'm not
understanding.


> 5) Depsgraph
> - need a way to solve all dependencies on nodes, images, movies,
> sequence.
>

Can you expand on what this means? I know what solve means in the context of
a 3D reconstruction, but it's less clear in this case.

Keir


>
>
> -Ton-
>
> ------------------------------------------------------------------------
> Ton Roosendaal  Blender Foundation   ton at blender.org    www.blender.org
> Blender Institute   Entrepotdok 57A  1018AD Amsterdam   The Netherlands
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110510/0b1bce87/attachment-0001.htm