[Bf-vfx] Blender data, big picture

[Sergey I. Sharybin]

  Hi everybody,

If somebody still not sure, i'm exactly that student who'll be working 
under integrating libmv to Blender.

After all discussions, i could imagine workflow we want to reach (all 
this pre-/post-processing and so on). But let's be sequentially. We 
still got nothing implemented but arguing about place of workflow where 
undistortion should happen.

I'd prefer to stick to initial proposal with new editor type and new 
data ID named MovieClip. I'll describe why i prefer this a bit later.

So, proposal with working under tracker is the following:

- Switch to "Tracking" scene layout where there'll be SpaceClip editor 
opened. Also, there'll be timeline for searching/bplayback-ing image 
sequence.
Initially, implement tools-ish approach of working with this 
sequence/tracker data. No nodes at all for first time and working with 
such editor would be "iterative" -- you'll be working under 
clearing/refining/resolving sequence until you've got good result.
- In this SpaceClip you'll open (or choose already opened) video/image 
sequence and start working with it.
- There'll be Toolshelf with panels like "Lens tools", "Marker tools", 
"Solver tools" (i hope meaning of this panels is clear)roperties region 
to set such parameters, as marker's search area and so on.
   So to change marker's properties, i.e. you'll simply select it in 
SpaceClip, open Peoperties panel (if it's stil lclosed) and input some 
parameters there. The same approach could be user for other objects in 
clip (camera or whatever else).
- It was useful for me to see tracker points (i could be a bit wrong 
here with terminology, but i mean representation of markers in 3d 
world). I think displaying them (as set of empties or bones, i.e.) in 3d 
view would be what we need. Ofcourse we'll need selection 
synchronization between markers and trackers. I don't think it's an issue.

This should be enough for first-generation tracker in Blender. It'll be 
already big step forward if it'll be working. Ofcource more cool things 
would be implemenetd after this.

Now a bit about internal side.

Data structure overview. There were some discussions about where all 
this markers/lens/etc data would be stored and how it'll be applied. One 
of variants was to use modifiers' like approach -- modifier for 
tracking, modifier for solving.. Well, it's a bit complicated due to 
there'll be almost always auto tracking+manual tracks+solving, But i 
like idea of separating data from VideoClip and keeping data outside of 
VideoClip could help with mosing to node-based design. I can't give 
exact image of this structures, but i'd prefer keep it separatelly (not 
"hardcoded" to clip). Something like

     struct VideoClip*{
         ...
         struct TrackerData *tracker_data;
         ...
     }
     struct TrackerData {
         ...
         struct MarkersData *markers_data;
         struct LensData *lens_data;
         struct WhateverelseData *...;
         ...
     }

This MarkersData/LensData/etc could be easily stored in nodes in the 
future. And ofcourse all this things related to RNA..

But there's some issues i can't solve here myself. They are related to 
data types and API which are required/acceptable by libmv. I hope Keir 
would help me with this issues. So, questions are:

- I suppose we should sotre some kind of libmv context, right?
- How/where markers should be stored? How their "animation" should be 
stored?
- The same question about trackers.
- How would we pass image sequence to libmv? Also it'll be cool to know 
which API Blender should use (functions and so on).
- Not sure if it's time for such questions, but it'll be nice if 
tracking/solving woudl happen in separated thread and displaying kind 
progress bar. Some kind of "special" libmv API could be needed for this, 
not sure.
- And ofcourse issues related to cache. Would be nice to know if there's 
some cache in libmv already and whar it supports/would support.

If we'll decide that there would be nicer to have tracker ID as initial 
entity (this could happen because of different issues -- tracking from 
multiplie sources, i.e.), there wouldn't be much changes in DNA -- we'll 
just move VIdeoClip inside TrackerData and make TrackerData "root" of 
tracking stuff. Also, after such switch we coudl use Image instead of 
Clip (we've been discussing it today in IRC).

Note: do not treat this data structures as something finished, i just 
wanted to show it'll be better to keep data "separated" and not keeping 
all markers/lens/trackers data in the single structure. I've got feeling 
it'll help to make design more easily adaptable for nodes.

A bit about tools. Tools (like auto-track, solve, so on) would be 
implemented in SpaceClip to deal with very basic tracking. In the 
future, new tools would be added or even moved to other editors -- it 
shouldn't be difficult. I think we could start from auto 
track/markers/solving tools and extend them when very basic workflow 
would work.

Now a bit about that new ID and editor we've been arguing about this 
afternoon. Yep, i'l lagree it could be some duplicated work, but we'll 
never know which of changes are really necessery and which not (before 
we'll try ofcourse) and such kind of "experiment" could have negative 
result on existing tools. And i'm also warrying about further support of 
code -- fixing bug in tracker could break texture stuff (and main reason 
of it would be that there're different developers who works for textures 
and tarcker and they can't predict if their change would breac something 
in other place). It will also help to keep UI clear and logical. 
There'll be no paiting/UV tools while match-moving. This looks like an 
advantage. And.. never of us was really working under tracking software, 
so why not accept this plan as initial? It could be easult changed in 
the future (if we'll decide it's necessery).

And one more question mostly to Keir. Last time we've been talking in 
IRC he proposed to contentrate on 2d stuff without touching 3d/solving. 
But i've been able to use SfM from libmv (reconstruct_video) which gave 
me quite good result on some sequences from lessons for other tracker's 
software. So maybe it's not too much to be finished there to be useful 
from Bledner?

-- 
With best regards, Sergey I. Sharybin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110510/a2aacccd/attachment.htm