[Bf-committers] Is there a parallel pipeline in blender?

[Ruan Beihong]

I've heard Scala. and the "future" to me is just an explicitly way to
do lazy evaluation that exists in many functional programming
language.
See http://en.wikipedia.org/wiki/Lazy_evaluation .
And I think I understand your idea. But laziness is in running
strategy. The actually execution couldn't avoid locks and thread
waits, if anyone knows how to avoid those. We can introduce the lazy
evaluation in to composite node. But the feature I meant to talk about
has nothing about it. This feature focusing to the execution of actual
work. To the outside, its just simply a loop execution a set of
chained function wrapped in one "pipeline", and looks the similar as
chained functions are running in one thread, but they are run in more
than one thread.
Think about 4 piece of data d[0:3] need processing by function f1,f2:
The series version:
thread0:
f1(d[0])
f2(d[0])
f1(d[1])
f2(d[1])
f1(d[2])
f2(d[2])
f1(d[3])
f2(d[3])
The pipelined version:
thread0: thread1:
f1(d[0])
f2(d[0])   f1(d[1])
f1(d[2])   f2(d[1])
f2(d[2])   f1(d[3])
             f2(d[3])

2009/8/11 Jonathan Merritt <merritt at unimelb.edu.au>:
> Couldn't you simply use a "future" model for results of nodes:
>     http://en.wikipedia.org/wiki/Futures_and_promises
> In abstract terms, the "future" construct is like a place-holder for a
> result that will eventually be computed, but is not necessarily
> available just yet.  When the result of the "future" is demanded
> absolutely, execution on other threads can wait until the "future"'s
> result is available.  However, by chaining "futures" together, you can
> often avoid having to set many explicit points at which your "start"
> and "stop" events occur.  That approach can reduce thread waits in
> systems like a network of nodes.
>
> I know this sounds very abstract and complicated, and I can't point
> you to any examples in C.  It's an approach that works like a dream in
> Scala though... even I can manage it! :-)
>
> Jonathan Merritt.
>
> On 11/08/2009, at 12:37 PM, Ruan Beihong wrote:
>> Hi,
>> I use CPU pipeline as an example of how this should work.
>> This feature is not very generic, but it do solve problems that data
>> can be processed in parallel but should be start and stop the process
>> in order. Many operations could benefit from this feature for example
>> the composite node: each frame is individual and operation in each
>> composite node is pure functional, i.e. thread-safe, but they must be
>> done in order of frame and composite nodes (from input to output).
>> With this feature, more than one frame can actually composite in
>> pipeline but the are returned in order.
>>
>> The BLI_thread may serve as an under layer of this feature.
>>
>> 2009/8/11 Brecht Van Lommel <brecht at blender.org>:
>>> Hi,
>>>
>>> As mentioned, Blender already does multithreading in various places.
>>> Some using the BLI_thread functions, others using OpenMP. I'm not
>>> sure
>>> why you are talking about CPU pipelining, to me it seems what you are
>>> proposing is standard multithreading, so it may be better to talk in
>>> that terminology. There's more places in Blender that could benefit
>>> from
>>> multithreading, and they can share some code, but basically need to
>>> be
>>> handled on a case by case basis. What you're proposing is very
>>> generic,
>>> so it's unclear to me what it is really about.
>>>
>>> Brecht.
>>>
>>> On Fri, 2009-08-07 at 09:14 +0800, Ruan Beihong wrote:
>>>> Thanks for the reply.
>>>> Maybe I didn't put it right. I mean the execution of the tasks in
>>>> pipeline goes in similar way as the pipeline in CPU, which always
>>>> starts tasks in order, and end it in order but between them tasks
>>>> run
>>>> in parallel. This feature can be put into use when some series of
>>>> tasks done in order to more than one piece of data.
>>>> I don't know much about OpenMP, but it seems OpenMP doesn't provide
>>>> this feature directly, isn't it?
>>>>
>>>> 2009/8/7 GSR <gsr.b3d at infernal-iceberg.com>:
>>>>> Hi,
>>>>> ruanbeihong at gmail.com (2009-08-06 at 2311.39 +0800):
>>>>>> Hi there,
>>>>>> I wonder if there is a parallel pipeline in blender. I mean the
>>>>>> pipeline as that in CPU which increase IPC (instructions per
>>>>>> cycle).
>>>>>
>>>>> Sorry, but I am unable to see how instructions per cycle matter for
>>>>> this. Maybe you mean Symmetric Multi Processing?
>>>>>
>>>>>> I'm considering implement that feature.
>>>>>
>>>>> Some parts of code are using OpenMP to accomodate systems with more
>>>>> than one logical CPU, libelbeem for example.
>>>>>
>>>>> GSR
>>>>>
>>>>> _______________________________________________
>>>>> Bf-committers mailing list
>>>>> Bf-committers at blender.org
>>>>> http://lists.blender.org/mailman/listinfo/bf-committers
>>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Bf-committers mailing list
>>> Bf-committers at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-committers
>>>
>>
>>
>>
>> --
>> James Ruan
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 
James Ruan