[Bf-blender-cvs] [df135b74fc9] master: PyAPI: use PYTHONUTF8/Py_UTF8Mode on all platforms

[Campbell Barton]

Commit: df135b74fc938ec5ff08bc912ece18917165319d
Author: Campbell Barton
Date:   Thu Jan 28 09:27:08 2021 +1100
Branches: master
https://developer.blender.org/rBdf135b74fc938ec5ff08bc912ece18917165319d

PyAPI: use PYTHONUTF8/Py_UTF8Mode on all platforms

System encoding issues have been a paint-point for us with Python 3,
since Blender always uses UTF-8 which might not be the case for the OS.

While the Py_SetStandardStreamEncoding was already set to utf-8,
the file-system could still have an incompatible encoding.

See PEP-540 for details.

===================================================================

M	source/blender/python/intern/bpy_interface.c

===================================================================

diff --git a/source/blender/python/intern/bpy_interface.c b/source/blender/python/intern/bpy_interface.c
index c54c78ae389..830a809408b 100644
--- a/source/blender/python/intern/bpy_interface.c
+++ b/source/blender/python/intern/bpy_interface.c
@@ -341,12 +341,20 @@ void BPY_python_start(bContext *C, int argc, const char **argv)
     }
   }
 
-  /* Without this the `sys.stdout` may be set to 'ascii'
-   * (it is on my system at least), where printing unicode values will raise
-   * an error, this is highly annoying, another stumbling block for developers,
-   * so use a more relaxed error handler and enforce utf-8 since the rest of
-   * Blender is utf-8 too - campbell */
-  Py_SetStandardStreamEncoding("utf-8", "surrogateescape");
+  /* Force `utf-8` on all platforms, since this is what's used for Blender's internal strings,
+   * providing consistent encoding behavior across all Blender installations.
+   *
+   * This also uses the `surrogateescape` error handler ensures any unexpected bytes are escaped
+   * instead of raising an error.
+   *
+   * Without this `sys.getfilesystemencoding()` and `sys.stdout` for example may be set to ASCII
+   * or some other encoding - where printing some `utf-8` values will raise an error.
+   *
+   * This can cause scripts to fail entirely on some systems.
+   *
+   * This assignment is the equivalent of enabling the `PYTHONUTF8` environment variable.
+   * See `PEP-540` for details on exactly what this changes. */
+  Py_UTF8Mode = 1;
 
   /* Suppress error messages when calculating the module search path.
    * While harmless, it's noisy. */