Issue 123975

Created attachment 82222 [details]
Experimental patch to update Python 3.3.3 for Linux and Windows environment

Changes contained only for Linux and Windows VS9 build.
pyuno has been migrated to be work with Python 3k. Therefore most changes are in building Python itself.
Changes: 
- without imp module: imp module is deprecated since Python 3.4. Instantiate new module without using it.
- pyuno has some warnings, the patch contains fix for them.
- Official compiler for Python has been changed to VS 2010 for Windows environment. 
  And solution file for VS 2008 is there in PC/VS9.0.

Created attachment 82231 [details]
For delivering all test files for regression test

Use this d.lst files to deliver regression test of Python.
Regression test can be executed as follows: 
> python -m test

There is error.py file in urllib module. When it was copied during 
make_installer.pl, I got the following error: 
>ERROR: ERROR: Found an error in the logfile. Packaging failed.
>in function: analyze_and_save_logfile

> ERROR: Copy: /tmp/ooopackaging/i_40161389076232/wntmsci12.pro/Apache_OpenOffice
>/archive/zip/en-US/00/python-core-3_3_3_zip/python-core-3.3.3/lib/urllib/error.py 
>to /tmp/m6mdIkUwMy/Apache_OpenOffice_4.0.0_Win_x86_install-arc_en-US/OpenOffice 4
>/program/python-core-3.3.3/lib/urllib/error.py

In installer::control::check_logfile function, each log lines are checked against 
the occurrence with /\bError\b/i expression.
The above line matched with the expression and it treated as error and stopped 
the packing process.

Hello,
I don't see this issue in http://people.apache.org/~jsc/developer-snapshots/snapshot/AOO4.1.0_Beta_fixes.html
Did you think that it could be added?

Python updates are always not easy to decide. Can somebody explain the implication for users with scripts written in older versions? 

For 4.1 I would not really add it because it not tested and available on all platforms. But in general I agree that we should test it asap

This is not finished yet. The following tasks are remained: 
- Building code for Windows, Mac OS X, mingw 
- Testing on all platform
- Building problem of Python on Linux environment caused by multithreaded make

Almost all macros and UNO components written in Python will not work without porting from Python 2 to 3 after this update.

I am not sure if we really still support mingw, I don|t know anybody who is building under mingw

Hello,
I hoped with AOO 4.1 beta 1573601 finally see Python 3.3.3 replace an outdated version.
I tested with Python 3 LibO 4.2.2 and also on OsX10.9.2 without problems: 
for example, the French extension "Grammalect 3.8.3"
http://www.dicollecte.org/grammalecte/oxt/Grammalecte-v0.3.8.3.oxt settles properly.

AOO for Win, OSX or Linux, we are forced to use an older version built with python 2.7!
http://www.dicollecte.org/grammalecte/oxt/Grammalecte-v0.3.6.2-py2x.oxt

It is a pity!

(In reply to papayes from comment #8)
> Hello,
> I hoped with AOO 4.1 beta 1573601 finally see Python 3.3.3 replace an
> outdated version.

As explained in comment 6 "Almost all macros and UNO components written in Python will not work without porting from Python 2 to 3 after this update."
Such an incompatible change in client code can only happen on a major release.

Hello, 
To  Ariel Constenla-Haile 
I know how to read, thank you, even if English is not my mother tongue.
Sorry, but I do not think that this beta is a major version, but simply one 4.0.2.
One beta exactly has to allow to test the setting-up of Python 3.3 without waiting for 2015 like that allows to test Java 1.7 with Mac Os!
You were less sensitive to cold when you modified in AOO 4 the management of the menus what involved the modification of the extensions !

Best Regards,

(In reply to papayes from comment #10)
> Sorry, but I do not think that this beta is a major version, but simply one
> 4.0.2.

Well, this was my point, AOO 5 will be the next major version were incompatible changes like switching to Python 3 can to be introduced.

> One beta exactly has to allow to test the setting-up of Python 3.3 without
> waiting for 2015 like that allows to test Java 1.7 with Mac Os!

This is rather different and cannot be compared. Since Revision 1229371 for Bug 118352 AOO can detected and use Java 1.7 and client code didn't need to change a single line of code because Java 1.7 can execute Java 1.6 code; with Python 3 the situation is completely different, as explained in comment 6 .

> You were less sensitive to cold when you modified in AOO 4 the management of
> the menus what involved the modification of the extensions !

This was an incompatible change in a major version switch (which by the way, wasn't asked be my, and could have been reverted in time if it were considered something bad, in context: http://markmail.org/message/jpn4e3no4jgv3n4d http://markmail.org/message/fhsepy2a56yhubi2); an incompatible change like switching from Python 2 to Python 3 will have to wait - IMO - to the next major, AOO 5.

Targeting to AOO Later, meaning that this will be for OpenOffice 5.0. Note that despite all the nice work from hanya the work is unifinished, so this must be completed and tested early in the 5.0 cycle.

On the other hand, the death of Python 2 has been postponed from May 2015 until May 2020: http://legacy.python.org/dev/peps/pep-0373/ . So, while it is still totally recommended to upgrade to Python 3, this is no longer urgent as it used to be.

I met the problem during to build gtest with Python 3.X shipping because gtest-1.7.0 needs Python 2.X to be built. 
Once shipped Python is built, it is used from solver since the path to the solver is there before system paths in PATH.

In the target "fused-gtest:" from misc/build/gtest-1.7.0/Makefile, 
the following file is executed like this:
> "$(srcdir)/scripts/fuse_gtest_files.py" "$(srcdir)/fused-src"

It seems the problems are: 
- syntax for print statement -> should be called as function
- sets module -> gone and they can be used without imporing some modules
used in the fuse_gtest/files.py file.

Created attachment 84920 [details]
Patch to ship Python 3.5.0 for Linux environment

Python 3.3 is already under security fix only phase. 
Python 3.5.0 was released in this month. I tried to migrate required files 
on Linux environment. The attached patch contains changes only for Linux environment to build.

Is there an ETA for this feature?

Remember that Python v.2.7 will receive bugfix support until January 1, 2020. After the last release, this version will receive no support [1].

So, this issue should be targeted for a next AOO build.

[1] https://www.python.org/dev/peps/pep-0373/

For the record:

Modern versions of Python have deprecated the old version of the MSVC compiler used to build AOO on Windows.
https://wiki.python.org/moin/WindowsCompilers

For python 3.5/3.6 we would need to be using MSVC 14.0.

But the problem is not just the Windows compiler: it used to be that we could build AOO with an external 3.x python package but, at least on FreeBSD, it is not possible to build AOO with the external python >= 3.6 anymore. Probably some porting is necessary for newer versions (I don't really have details).

(In reply to Pedro Giffuni from comment #17)
> But the problem is not just the Windows compiler: it used to be that we
> could build AOO with an external 3.x python package but, at least on
> FreeBSD, it is not possible to build AOO with the external python >= 3.6
> anymore. Probably some porting is necessary for newer versions (I don't
> really have details).

Can you please provide more details on that building with Python >= 3.6 on FreeBSD part? I couldn't find any info on it anywhere.

(In reply to damjan from comment #18)
> (In reply to Pedro Giffuni from comment #17)
> > But the problem is not just the Windows compiler: it used to be that we
> > could build AOO with an external 3.x python package but, at least on
> > FreeBSD, it is not possible to build AOO with the external python >= 3.6
> > anymore. Probably some porting is necessary for newer versions (I don't
> > really have details).
> 
> Can you please provide more details on that building with Python >= 3.6 on
> FreeBSD part? I couldn't find any info on it anywhere.

Hi;

Sorry for the delay. There is a FreeBSD bug report:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229408

(In reply to Pedro Giffuni from comment #19)
> (In reply to damjan from comment #18)
...
> > 
> > Can you please provide more details on that building with Python >= 3.6 on
> > FreeBSD part? I couldn't find any info on it anywhere.
> 
> Hi;
> 
> Sorry for the delay. There is a FreeBSD bug report:
> 
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229408

Details / smoking gun in the pyuno module:

....
[ build CXX ] pyuno/source/module/pyuno_util
/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/pyuno/source/loader/pyuno_loader.cxx:117:5: error: no matching function for call to 'Py_SetPythonHome'
    Py_SetPythonHome( o.pData->buffer);
    ^~~~~~~~~~~~~~~~
/usr/local/include/python3.7m/pylifecycle.h:43:18: note: candidate function not viable: no known conversion from 'sal_Char [1]' to 'const wchar_t *' for 1st argument
PyAPI_FUNC(void) Py_SetPythonHome(const wchar_t *);
                 ^
1 error generated.

Created attachment 86813 [details]
Proof-of-concept fix

On python 3 and newer, Py_SetPythonHome() uses a widechar argument.

Current documentation suggests using Py_DecodeLocale() which is only available starting with Python 3.5.

I am currently testing the patch, I plan to submit it via a github pull request.

For the record, hanya's patches are likely to be obsolete, or simply wont apply cleanly, since have updated Python 2.7.

This issue is targeted for 4.2.0
Did you backported for 4.1.8?

Created attachment 86814 [details]
Proof-of-concept fix

(parenthesis oops)

Comment on attachment 86814 [details]
Proof-of-concept fix

Nevermind, this is very broken.

Created attachment 86815 [details]
python3 fixes

This seems to take the build much further but it needs review.

Python 3 support with system-provided Python is now in trunk, implemented in this range of commits:
a5f48dc26160473ef78ab88ce9cc4c1e0bc5f5e1..aec63fa3669010e9bec513e6cd55a25fc335d90e

Internal Python 3, is still TODO.

I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.

I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.

I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.

Since MacOS 12.3, Apple don't provide Python 2.7 anymore[1].
So, unable to run any Python scripts.

Very annoying for Mac Monteray end-users...

Is it possible to embed the inner Python 3 like for Linux systems?

[1] https://developer.apple.com/documentation/macos-release-notes/macos-12_3-release-notes#Python

*** Issue 128570 has been marked as a duplicate of this issue. ***

For me the python packaged in the OpenOffice version for windows (7, 8, 10, 11 in x64 or x32) no longer works. see bug 128570 [1].

Where are we? What has already been done for the port of python 3.x?

[1] https://bz.apache.org/ooo/show_bug.cgi?id=128570

Sorry, I wanted to tell you about bug 128569 [1]
[1] https://bz.apache.org/ooo/show_bug.cgi?id=128569