Issue 121496

Created attachment 80033 [details]
Patch to add Python3 support

This patch contains modifications from my fork[1] for Python3 except for 
hasModule function.
Summary of patch: 
- Use unicode as string but bytes is not allowed for method argument anymore. 
  For example, uno.getTypeByName(b"long") is not allowed, it should be 
  uno.getTypeByName("long") (for 3.0-) or u"long" (3.3-).
  In my opinion, bytes is no longer string but it is for IO byte sequence.
- getPyUnoClass method does not have any parameters. I met the problem with it 
  in module initialization. When I use Runtime class in the PyInit_pyuno function, 
  pyuno crashes that build as standalone. I simply removed it because of the 
  parameter is not used.
- int type is gone, but bool type should be treated as different type in pyObject2Any function.
- getattrfunc and setattrfunc are deprecated to use in custom type definition. 
  They should be replaced with getattrofunc and setattrofunc.
- cmpfunc is also deprecated, should be replaced with richcompare.
- no __members__ and __methods__ attributes, they should be replaced with __dir__.
- uno.ByteSequence class should allow bytes and bytearray on Python3 but not unicode.

Just in my opinion, uno.ByteSequence should be replaced with bytes and bytearray. 
It was implemented to separate string and byte sequence but we have them as 
separated in Python3.
And uno.Bool should be removed now, it has been deprecated several years.


[1] https://github.com/hanya/pyuno3/commit/801ad6c9e1ee6b059a50e602bfe8b7a453f61a84

Created attachment 80034 [details]
Test files on Linux environment

I wrote test script for standalone build of PyUNO module on Linux. 
This is not working on different environment because not supported by setup.py 
file to build PyUNO as standalone python module.
Attached archive contains instruction to use scripts contained.
This test case worked well on my environment with Python 2.7, 3.1, 3.2 and 3.3 (3.0 was not tested).
It seems more test case required.

One style issue:

I think the macros you defined in
inc/pyuno/pyuno.hxx

would be better placed in
source/module/pyuno_impl.hxx

The reason is that I am trying to keep only compatibility for older python versions in pyuno.hxx so non-standard macros should not be there. Putting them inside the module reduces their scope.

I will see that things still build with Python 2.7.3 before committing. Thanks!

From the SVN-Autofix-Robot (after replacing 121296 with 121496):
"pfg" committed SVN revision 1423272 into trunk:
i121496 - Python3 support for pyuno.

Many thanks to you both, Hanya and Pedro!

(In reply to comment #4)
> From the SVN-Autofix-Robot (after replacing 121296 with 121496):
> "pfg" committed SVN revision 1423272 into trunk:
> i121496 - Python3 support for pyuno.
> 

Oops.. I was wondering why the Robot didn't catch it.

> Many thanks to you both, Hanya and Pedro!

All the credit is Hanya's .. this was the major heavy-lifting effort that was needed.

"pfg" committed SVN revision 1423676 into trunk:
#i121496# - [PyUNO] Python3 support

There is a minor build problem on OSX if one is building for our base platform there. The OSX10.4SDK contains an ancient python 2.3 version with a matching modsupport.h header, where the second argument to pyModule_AddObject is non-const, so the compiler complains about an invalid conversion from const to non-const char in pyuno_type.cxx:297.

I'm not sure what to do here. Change the build so that the modsupport.h from our bundled python is found first? Use a const_cast for now?

Hello;

(In reply to comment #7)
> There is a minor build problem on OSX if one is building for our base
> platform there. The OSX10.4SDK contains an ancient python 2.3 version with a
> matching modsupport.h header, where the second argument to
> pyModule_AddObject is non-const, so the compiler complains about an invalid
> conversion from const to non-const char in pyuno_type.cxx:297.
> 
> I'm not sure what to do here. Change the build so that the modsupport.h from
> our bundled python is found first? Use a const_cast for now?

According to the python 2.7.3 documentation, the second argument for pyModule_AddObject should be constant:

http://docs.python.org/2/c-api/module.html

I guess the const cast would work but you may have to conditionalize it on the python version.

On Mac OSX I think we have a bigger problem: we have been replacing some old python code to use 2.5 features so it may build but some things will not work. I understand modern MacOSX comes with python 2.7.3; no idea how this will interact with AOO built with older SDK.

Ok, the points that we depend on python 2.5 features and that we bundle with python 2.7 indicate that we need to make sure that the header file from our bundled python is found first.

BTW, the const_cast would always work because passing a nonconst as const argument is always allowed.

Hi Herbert;

I am not sure I understand: finding headers from our bundled python but building with the old one wouldn't solve anything.

My suggestion is to add the const_cast for now: we have sufficient glue in the pyuno main header so things will build just fine.

The problem with some of the newer code is only on some python scripts so using the old python will work but with reduced functionality. We may have make sure that MacOSX is using an updated python but perhaps we should do that in configure.in

Having looked into the python section of our configure I saw that system-python was auto-enabled for python from the OSX 10.4 SDK, so this is where the problem came from. This section should be updated to default to the bundled python if the system python is too old.

We should pursue this in another issue though as it is only related indirectly. This issue is about the success story with the new feature "python3 support". Many thanks to Hanya!

"pfg" committed SVN revision 1425754 into trunk:
#i121496# - More Python 2to3 cleanups.

"alg" committed SVN revision 1427802 into trunk:
#121496# non-const char ptr needed, see task

ALG: Cannot build current mac version; error is in line 297 of pyuno/source/module/pyuno_type.cxx where a non-const char* is needed for the targetName variable. I checked that the macro PyBytes_AsString (in python's stringobject.c) indeed returns a non-const char*. To make the build work, I will change pyuno_type.cxx by removing the const from the targetName variable definition.
ALG: Comitted as rev 1427802.
ALG@Pedro: reopened this task, maybe my solution is not good. Please have a look, but the build had to work.

(In reply to comment #14)
...
> ALG@Pedro: reopened this task, maybe my solution is not good. Please have a
> look, but the build had to work.

Your solution is fine. Thank you!

As discussed with hdu@, you are likely to experience some runtime problems with extremely old versions of python; we just can't have everything work well on Python 3 and Python 2.3. But that is a matter for a different BZ issue.

fixed in 4.0.0