Update Py_UNICODE to Py_UCS4

[nulano]

PyUnicode_AS_UNICODE is now deprecated, replace it with PyUnicode_AsUCS4Copy or PyUnicode_READ_CHAR.

• Fixes Unable to draw an Ext-B+ char on Windows #3344, 32-bit Unicode code points don't work on Windows #3777 (Python 3.x only).
• Add test for Unable to draw an Ext-B+ char on Windows #3344, 32-bit Unicode code points don't work on Windows #3777.

Fixing the issues for Python 2.7 will likely require a lot of effort. Given the coming end of support, I don't think it makes sense to fix it there; therefore the test is set to skip on Windows for Python 2.x.

[nulano]

I'm not sure why the new test fails on Centos-6 and Amazon-1. It seems that on Centos the text is shifted one pixel down...

All other platforms pass.

Centos 6 image:

Xor with template:

[nulano]

It looks like Codecov isn't receiving coverage from any Python 2.7 builds with Raqm installed (such as alpine on Travis: https://travis-ci.org/python-pillow/Pillow/jobs/517268209#L712):
https://codecov.io/gh/python-pillow/Pillow/src/a4fd4c2b/src/_imagingft.c#L370

[hugovk]

Good spot!

The Alpine job is 7325.10 and is shown as received by Codecov:

https://codecov.io/gh/python-pillow/Pillow/commit/a4fd4c2b23d921ea8263738c2e6f87720774d5f2/build

But checking the logs (in the if [ "$LINT" == "" ]; then section), there's some problem with coveralls-merge, which combines coverage data for the Python and C files. I expect we're only getting the Python coverage data in this case. Scrolling back a few lines shows a missing dependency: E: Unable to locate package lcov

https://travis-ci.org/python-pillow/Pillow/jobs/517268209#L4166

[hugovk]

From #3835 (comment):

Our tests didn't pick it up because ... whilst we do test PyPy3 on Travis CI, there is a "implicit declaration of function ‘PyUnicode_AsUCS4Copy’" warning we didn't notice as all the test_imagefont tests are skipped because there's no FreeType (or RAQM) on PyPy3. We should improve these testing gaps.

That's not quite accurate: FreeType is available on Travis CI for PyPy3.

The test_imagefont tests were skipped when this PR was merged:

• https://travis-ci.org/python-pillow/Pillow/jobs/552376944#L4197

But they were run on the previous merge to master:

• https://travis-ci.org/python-pillow/Pillow/jobs/552376428#L4192

Could it be that PY_VERSION_HEX isn't available on PyPy?

[nulano]

Could it be that PY_VERSION_HEX isn't available on PyPy?

It is available.

Because FreeType support is loaded dynamically on Linux, I guess it failed to dynamically link the PyUnicode_AsUCS4Copy function and assumed it is because of missing FreeType support.
https://travis-ci.org/python-pillow/Pillow/jobs/552842796#L3090

[hugovk]

I reported this to PyPy and they've now added PyUnicode_AsUCS4Copy():

We are just following CPython with our emulation of the CPython C API. If CPython deprecates but keeps around functions, then they'll stay in PyPy too.

PyUnicode_AsUCS4Copy() is not implemented in PyPy so far, even though it was present since CPython 3.3. I just added this function in 3ec1002a818c. Closing this as resolved, but please reopen if you hit more missing related functions.

• https://bitbucket.org/pypy/pypy/issues/3035/cpython-old-style-unicode-api-deprecations

Let's check again when the next PyPy release is out. The current version is 7.1.1:

• https://morepypy.blogspot.com/2019/04/pypy-711-bug-fix-release.html

[nulano]

Let's check again when the next PyPy release is out. The current version is 7.1.1:

• https://morepypy.blogspot.com/2019/04/pypy-711-bug-fix-release.html

PyPy3.6-7.2.0 has been released. The fix is ready, but depends on #4140 and #4109 being merged.

Edit: See #4145