[Bf-committers] Do we need/want a unicode text engine in Blender?

[Bastien Montagne]

Hi devs,

This is the “sister discussion” of this funboard one: 
http://lists.blender.org/pipermail/bf-funboard/2013-July/005419.html

I decided writing that mail after a discussion with Ton and Campbell on 
IRC, feeling I could not manage to get my point clear enough, and in 
hope to get feedback from more Blender devs.

==Current situation==
We do not support so called "unicode scripts" at all. We simply handle 
unicode chars one after another, for drawing them as well as editing 
them. This works nice for simple languages ("european" ones, "latin" 
unicode script), but not for complex unicode scripts (i.e. most of all 
other languages).

Drawing fails for RTL languages (obviously, we are currently using an 
ugly hack to support arabic & co labels in UI), but also for some LTR 
ones like Indian languages, where some "vowels parts" of letters are 
misplaced (and this cannot be fixed by some kind of pre-processing, like 
for RTL languages).

Editing "fails" for every languages using “compound” letters, in that 
selection, cursor moving, deleting, etc. is done on a unicode char 
basis, and not on a grapheme (“logical” char) basis.

==How to solve==
We basically have four options:
1) Clearly state and communicate we do not fully support complex 
languages (all RTL, Indic, easterns?).
2) Make case-by-case fixes/hacks in current BLF code.
3) Fully support unicode rules in an intern text engine (built based on 
existing code like harfbuzz).
4) Fully support unicode rules using an existing library like Pango or ICU.

Of those, only reasonable choices imho are 1) and 3). 2) would make BLF 
code more and more ugly and complex (we’d have to support modifier chars 
rendered before, or around, their base char :/), while 4) would add huge 
dependency (even including glib with Pango!) we do not want for Blender.

So main question is: Do we want to actually support Unicode in Blender 
in all its aspects, or are we happy with current limited support, 
working well for European languages but somewhat buggy for most of other 
languages?