[Bf-committers] Understanding the dev process

Wed Jul 10 09:26:26 CEST 2019

Hi!

Here is my perspective of the development process as a relative outsider.
I wrote this before reading the other replies so it would be unbiased.

I am full-time employed as a software engineer, working within a team which
develops a package of creative tools that includes a 3D Viewport.
I was very close to Blender development while finishing my MSc about 5
years ago. Nowadays, I try to contribute a bit when I have the time, which
could mean a few hours during the weekend.
I try to help out for fun, technical interest, and mostly because I truly
believe that Blender is a very important project.

So, what is the experience for a developer with some background on the
topic, but only a few hours to spare?

* A few hours go into catching up with the latest news. The weekly meeting
notes are the most important, but there is also mailing lists,
developer.blender.org and devtalk.blender.org.
During the development of 2.8 the amount of changes and discussions was so
high that all I could do was catch up.

* Picking up a task is tricky. I'm usually out of touch with whom is
patrolling the bug tracker or who is touching which area.
I need to spend some time making sure that there are no conflicts before
starting to do something.
For me it is difficult to choose between an actively developed area or
something more abandoned.
For the first, there needs to be constant coordination. Working on an
active area requires full-time availability which is not possible if you
already have a job. The typical outcome of this situation is that my work
gets out of date faster than I can work on it.
If I pick something a bit more out of the way, often there won't be a
module owner to coordinate with. Not having my work approved beforehand
makes me nervous.

* Working on UI is really tricky.
I love working on UI/UX and it is something I do daily at my job. However,
this a very emotional process at Blender, on top of a codebase that is very
hard to work with.
Sometimes UI-related discussions are constructive, but mostly it's just
stressful because of vocal opposing opinions. When working on UI, it is
quite likely to end up re-doing things slightly different again and again,
mostly because the communication wasn't good enough.
I believe that the codebase reflects this uncertainty. Features are hacked
into corners where they shouldn't be, maybe because they're "experimental",
but that's how they end up staying. I see hacks on top of hacks and it's
really kind of a miracle that the UI code still holds together.

* Codebase quality.
In my opinion, this is a point where Blender is really falling behind, and
I believe it is affecting the development process.

  1) Documentation. The code has a very low ratio of comments. The comments
are confusing, obscure, obsolete or unprofessional.
  There is nearly no high level documentation describing important systems
like data format or the undo/redo stack. (Not WIP design proposals, but
actual documentation to introduce new developers to the core design of
Blender)

  2) Naming. Unintelligible variables with a maximum of 3 letters are a
thing in Blender.
  I believe that variables should read like English, because it takes time
and effort to figure out the meaning of ot, ob, C, a, em, eve, selme, but,
kb, clog or co.
  I also see variables being re-used with a different meaning, which
therefore get a name like "a1". Readability should take more priority.

  3) Encapsulation. Blender is not very object oriented (which is fine),
and it has a tendency to have no encapsulation of concepts whatsoever
(which is not fine).
  One consequence is that modifying a piece of code will have really hard
to foresee repercussions in seemingly unrelated areas.
  Another consequence is that the cognitive load for a reader is very high.
It is hard to consider many things at the same time which have lots of
ramifications.
  For example, looking at the transform code in transform.c: this is a file
with 10k lines.
  It has all sorts of specific editing of meshes, vertex sliding, keymap,
shape keys, NLA tweak editing, curve shrink flatten...
  Even picking up a single function that looks like a "general" apply
rotation, I see specific domain knowledge of grease pencil falloff and pose
mode bone keying.
  My point here is that all these concepts mixed together are daunting. It
is unlikely that a new developer will be confident in all of these
different areas.

  4) Language/Idiom. I am not sure if this is a problem of C vs C++ or just
of how C is used in Blender.
  In Blender there are a lot of char and int being used where it would be
much better to see a typed enum.
  Variables are declared at the top of the function instead of at the place
where they're used.
  I really miss RAII and the convenience of high level containers like
vector and string instead of fumbling around with plain C-style arrays.

Due to all of the above reasons: I waste a lot more time than needed
understanding things and I often feel overwhelmed by complexity and
concepts I'm not familiar with.
For example, I start by wanting to do a simple thing, I find where the code
is (awesome!), but then I don't fully understand it, there is a comment
saying something like "// hack" next to some variable called 'c' that gets
initialized to '-2'. Also there is something about... bundles? What are
bundles even? Oooh.. they're something from tracking of which I know
nothing about. What are they doing here anyway? Right... now I'm pretty
sure that changing this code will break something that I didn't even know
existed.
I don’t feel confident to touch the code, I'm afraid to break things and I
feel a bit stupid. Maybe I should just back away slowly and figure out
something else to do. :(

* Lack of review process and unit tests.
Automated tests would go a long way to avoid breakage and give quick
feedback to developers.
Enforced code reviews would also reduce breakage and ensure that at least 2
people are knowledgeable about a commit. It is a great opportunity to teach
new developers good practices and it is essential to keep the quality of
the code, architecture, vision and roadmap.

* Go-to people.
There isn't always a module owner or stakeholder artist for an area. Asking
around can give you no useful reply as I sense that there is a general fear
of looking something less than an absolute master of the Blender code.
Asking something to a core developer will always result in loads of good
information, but I feel bad doing this because those people are already
overloaded.

The good things:
* Iteration time. So fast! I can compile in less than a minute. Usually, I
can compile *and* launch Blender in just seconds.
This is really, really good!

* Phabricator is not perfect, but it's quite nice! On the negative side:
I'd like a better dashboard, burndown charts and the like. On the positive
side: tasks are well tracked, I get notifications and it is feature-packed
of utilities such as file uploading and review tool.

* blender.chat really makes a difference. I can now easily post screenshots
and paste full blocks of code with nice formatting. I can ask something, go
offline and check the reply later.

* Culture and Community. I consider the people around Blender mostly
positive, however, there are some people who seem a bit stuck in a 90s
self-ego-inflating bro culture.
These people seem very focused in their own opinion and can quite sour your
mood by telling "rtfm" or "only the real smart people can do that. don't
waste your time".
Besides these very few personalities, people are generally very positive
and all the core developers are friendly, caring and helpful. Community is
one of the core strengths of Blender. ^^

Hope it helps,
Inês Almeida

On Sat, 6 Jul 2019 at 18:47, Jacques Lucke <mail at jlucke.com> wrote:

>
>
> Hi Nathan,
>
> It's really interesting to see all these statements, so thanks for
> asking. I agree with the things said already, and just wanted to add one
> thing regarding code quality.
>
> It is well known by now, that in the beginning of the Blender 2.8
> project, a lot of technical dept was created. That's the past and cannot
> really be changed anymore. What I find more alarming that most commits I
> see still create more technical dept (even if they fix bugs). So many
> commits add new conditions or extend existing ones, making it more
> difficult to reason about the code every time.
>
> I'd love to see more commits not only fixing bugs but also improving the
> quality of code by making it simpler to reason about (that usually boils
> down to having small functions with mostly no conditions). Larger
> refactors are definitely helpful and I totally support doing more of
> those. However, by themselves, they won't make any code simpler.
>
> I believe that pretty much every function in Blender could be "massaged"
> a bit (extracting subfunctions, renaming variables, reducing vertical
> scope of variables) to be become a better version of itself. One does
> not have to do a big refactor for that. A simple culture in that,
> whenever you fix a bug in a function, you also commit one simplification
> to the same function, could go a long way.
>
> In summary, to me there are really two kinds of technical depts: large
> scale architecture problems and small problems in every function. Large
> refactors solve the first issue over time, but will not solve the
> second. The second issue can only be solved when every developers
> actively simplifies the code he is working on. I'd love to see this
> enforced somehow in the development process, but I don't know how.
>
> Have a good weekend,
> Jacques
>
> Am 2019-07-04 15:37, schrieb Nathan Letwory:
>
> > Hey all,
> >
> > As you all may know by now I've been asked to help coordinate and manage
> > the Blender development.
> >
> > To get a better understanding of what these days is going on, and to
> > prevent me from just acting through my personal preferences, I'd like to
> > hear from the blender developer community how they see the current dev
> > process.
> >
> > I'm most interested in finding out how devs perceive the process: what
> goes
> > well, and even more so what causes trouble.
> >
> > An open discussion by anyone on this topic is of course welcome, but I'd
> > like (and also a bit expect) input at least from those who are listed on
> > the Modules [1 [1]] page.
> >
> > Cheers!
> >
> > /Nathan 'jesterKing' Letwory
> >
> > [1] https://wiki.blender.org/wiki/Modules
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > https://lists.blender.org/mailman/listinfo/bf-committers
>
>
>
> Links:
> ------
> [1] https://wiki.blender.org/wiki/Modules
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> https://lists.blender.org/mailman/listinfo/bf-committers
>