[Bf-committers] Usage of PyUnicode_FromString() in blender 2.50

[Campbell Barton]

replying to an old thread..
today I came across an object with the name "Mütze", This is latin-1,
and raises an error when given to PyUnicode_FromString().

simply...
print(object.name) # raises an error.

After being sternly lectured in the #python IRC room about how broken
blender's editor, font rendering and python api is I am left wondering
if there is any one good solution.

It was suggested that each blend file could have its own encoding but
I think this is really impractical especially with linking between
blend files.

So I rather just choose one encoding and enforce it across all blend files.

Some options are..
* Only ASCII (enforce input is ascii)
* Only utf-8 (enforce input also)
* Only latin-1 (enforce input)
* Don't use python strings, we can use python byte arrays instead,
this would actually speed up the python api somewhat, on profiling I
found utf conversion took a fair bit of time. But realistically this
would make the API cumbersome to use in other ways.

Notes...

* Scripts can define their own encoding. eg.
# -*- coding: latin-1 -*-
foo = "Mütze"

... this is fine for internal usage but for object names we cant mix
it up since multiple scripts need to access.

* PYTHONIOENCODING can be used to set the default encoding for
blender's python to use.

* Python3 all strings are utf, python2 isnt, incase anyones wondering
why this didnt happen before.


On Tue, Dec 8, 2009 at 5:43 PM, joe <joeedh at gmail.com> wrote:
> BTW, keep in mind that on windows, argv is always in ascii (not UTF-8).
>
> Joe
>
> On Mon, Dec 7, 2009 at 1:44 AM, Yomgui <yomgui1 at gmail.com> wrote:
>> Hi,
>>
>> I've seen in many places that the Python function PyUnicode_FromString
>> (and similar) is used to convert system strings like argv[] and IO
>> pathname.
>>
>> If we look at the Python 3.1 documentation
>> (http://docs.python.org/3.1/c-api/unicode.html?highlight=pyunicode_fromstring#PyUnicode_FromString)
>> this function supposes that the input string is UTF-8 encoded.
>> That's not a strict rule for all OS. This function should be reserved
>> to pass string for runtime usage like if it was given in a script, but
>> not for OS/IO strings.
>>
>> A better (I know... with just more arguments) function to convert
>> system strings is PyUnicode_Decode()
>> and use as encoding parameter: Py_FileSystemDefaultEncoding
>>
>> Thanks.
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
>>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 
- Campbell