Issue 123975 - Update to Python 3.X
Summary: Update to Python 3.X
Status: CONFIRMED
Alias: None
Product: General
Classification: Code
Component: code (show other issues)
Version: 4.2.0-dev
Hardware: All All
: P3 Normal with 16 votes (vote)
Target Milestone: AOO Later
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-01-05 16:08 UTC by hanya
Modified: 2020-04-03 14:38 UTC (History)
11 users (show)

See Also:
Issue Type: TASK
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Experimental patch to update Python 3.3.3 for Linux and Windows environment (100.47 KB, patch)
2014-01-05 16:21 UTC, hanya
no flags Details | Diff
For delivering all test files for regression test (18.19 KB, text/plain)
2014-01-07 08:44 UTC, hanya
no flags Details
Patch to ship Python 3.5.0 for Linux environment (38.23 KB, patch)
2015-09-16 16:28 UTC, hanya
no flags Details | Diff
Proof-of-concept fix (777 bytes, patch)
2020-01-13 15:45 UTC, Pedro Giffuni
no flags Details | Diff
Proof-of-concept fix (729 bytes, patch)
2020-01-13 16:28 UTC, Pedro Giffuni
no flags Details | Diff
python3 fixes (2.30 KB, patch)
2020-01-14 02:14 UTC, Pedro Giffuni
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description hanya 2014-01-05 16:08:22 UTC
Please update shipped Python to version 3.X.

Python 2.7.X will be maintained until May 2015 [1]. It should be moved to Python 3k until then.

[1] http://www.python.org/dev/peps/pep-0373/
Comment 1 hanya 2014-01-05 16:21:05 UTC
Created attachment 82222 [details]
Experimental patch to update Python 3.3.3 for Linux and Windows environment

Changes contained only for Linux and Windows VS9 build.
pyuno has been migrated to be work with Python 3k. Therefore most changes are in building Python itself.
Changes: 
- without imp module: imp module is deprecated since Python 3.4. Instantiate new module without using it.
- pyuno has some warnings, the patch contains fix for them.
- Official compiler for Python has been changed to VS 2010 for Windows environment. 
  And solution file for VS 2008 is there in PC/VS9.0.
Comment 2 hanya 2014-01-07 08:44:45 UTC
Created attachment 82231 [details]
For delivering all test files for regression test

Use this d.lst files to deliver regression test of Python.
Regression test can be executed as follows: 
> python -m test
Comment 3 hanya 2014-01-07 08:51:19 UTC
There is error.py file in urllib module. When it was copied during 
make_installer.pl, I got the following error: 
>ERROR: ERROR: Found an error in the logfile. Packaging failed.
>in function: analyze_and_save_logfile

> ERROR: Copy: /tmp/ooopackaging/i_40161389076232/wntmsci12.pro/Apache_OpenOffice
>/archive/zip/en-US/00/python-core-3_3_3_zip/python-core-3.3.3/lib/urllib/error.py 
>to /tmp/m6mdIkUwMy/Apache_OpenOffice_4.0.0_Win_x86_install-arc_en-US/OpenOffice 4
>/program/python-core-3.3.3/lib/urllib/error.py

In installer::control::check_logfile function, each log lines are checked against 
the occurrence with /\bError\b/i expression.
The above line matched with the expression and it treated as error and stopped 
the packing process.
Comment 4 oooforum (fr) 2014-03-07 10:07:57 UTC
Hello,
I don't see this issue in http://people.apache.org/~jsc/developer-snapshots/snapshot/AOO4.1.0_Beta_fixes.html
Did you think that it could be added?
Comment 5 jsc 2014-03-07 11:52:36 UTC
Python updates are always not easy to decide. Can somebody explain the implication for users with scripts written in older versions? 

For 4.1 I would not really add it because it not tested and available on all platforms. But in general I agree that we should test it asap
Comment 6 hanya 2014-03-07 16:12:03 UTC
This is not finished yet. The following tasks are remained: 
- Building code for Windows, Mac OS X, mingw 
- Testing on all platform
- Building problem of Python on Linux environment caused by multithreaded make

Almost all macros and UNO components written in Python will not work without porting from Python 2 to 3 after this update.
Comment 7 jsc 2014-03-07 17:21:46 UTC
I am not sure if we really still support mingw, I don|t know anybody who is building under mingw
Comment 8 papayes 2014-03-16 10:47:28 UTC
Hello,
I hoped with AOO 4.1 beta 1573601 finally see Python 3.3.3 replace an outdated version.
I tested with Python 3 LibO 4.2.2 and also on OsX10.9.2 without problems: 
for example, the French extension "Grammalect 3.8.3"
http://www.dicollecte.org/grammalecte/oxt/Grammalecte-v0.3.8.3.oxt settles properly.

AOO for Win, OSX or Linux, we are forced to use an older version built with python 2.7!
http://www.dicollecte.org/grammalecte/oxt/Grammalecte-v0.3.6.2-py2x.oxt

It is a pity!
Comment 9 Ariel Constenla-Haile 2014-03-16 14:11:49 UTC
(In reply to papayes from comment #8)
> Hello,
> I hoped with AOO 4.1 beta 1573601 finally see Python 3.3.3 replace an
> outdated version.

As explained in comment 6 "Almost all macros and UNO components written in Python will not work without porting from Python 2 to 3 after this update."
Such an incompatible change in client code can only happen on a major release.
Comment 10 papayes 2014-03-17 15:29:29 UTC
Hello, 
To  Ariel Constenla-Haile 
I know how to read, thank you, even if English is not my mother tongue.
Sorry, but I do not think that this beta is a major version, but simply one 4.0.2.
One beta exactly has to allow to test the setting-up of Python 3.3 without waiting for 2015 like that allows to test Java 1.7 with Mac Os!
You were less sensitive to cold when you modified in AOO 4 the management of the menus what involved the modification of the extensions !

Best Regards,
Comment 11 Ariel Constenla-Haile 2014-03-17 18:45:32 UTC
(In reply to papayes from comment #10)
> Sorry, but I do not think that this beta is a major version, but simply one
> 4.0.2.

Well, this was my point, AOO 5 will be the next major version were incompatible changes like switching to Python 3 can to be introduced.

> One beta exactly has to allow to test the setting-up of Python 3.3 without
> waiting for 2015 like that allows to test Java 1.7 with Mac Os!

This is rather different and cannot be compared. Since Revision 1229371 for Bug 118352 AOO can detected and use Java 1.7 and client code didn't need to change a single line of code because Java 1.7 can execute Java 1.6 code; with Python 3 the situation is completely different, as explained in comment 6 .

> You were less sensitive to cold when you modified in AOO 4 the management of
> the menus what involved the modification of the extensions !

This was an incompatible change in a major version switch (which by the way, wasn't asked be my, and could have been reverted in time if it were considered something bad, in context: http://markmail.org/message/jpn4e3no4jgv3n4d http://markmail.org/message/fhsepy2a56yhubi2); an incompatible change like switching from Python 2 to Python 3 will have to wait - IMO - to the next major, AOO 5.
Comment 12 Andrea Pescetti 2014-04-19 16:25:54 UTC
Targeting to AOO Later, meaning that this will be for OpenOffice 5.0. Note that despite all the nice work from hanya the work is unifinished, so this must be completed and tested early in the 5.0 cycle.

On the other hand, the death of Python 2 has been postponed from May 2015 until May 2020: http://legacy.python.org/dev/peps/pep-0373/ . So, while it is still totally recommended to upgrade to Python 3, this is no longer urgent as it used to be.
Comment 13 hanya 2014-10-07 05:29:49 UTC
I met the problem during to build gtest with Python 3.X shipping because gtest-1.7.0 needs Python 2.X to be built. 
Once shipped Python is built, it is used from solver since the path to the solver is there before system paths in PATH.

In the target "fused-gtest:" from misc/build/gtest-1.7.0/Makefile, 
the following file is executed like this:
> "$(srcdir)/scripts/fuse_gtest_files.py" "$(srcdir)/fused-src"

It seems the problems are: 
- syntax for print statement -> should be called as function
- sets module -> gone and they can be used without imporing some modules
used in the fuse_gtest/files.py file.
Comment 14 hanya 2015-09-16 16:28:30 UTC
Created attachment 84920 [details]
Patch to ship Python 3.5.0 for Linux environment

Python 3.3 is already under security fix only phase. 
Python 3.5.0 was released in this month. I tried to migrate required files 
on Linux environment. The attached patch contains changes only for Linux environment to build.
Comment 15 Olivier R 2015-10-18 07:54:11 UTC
Is there an ETA for this feature?
Comment 16 oooforum (fr) 2019-04-14 12:18:08 UTC
Remember that Python v.2.7 will receive bugfix support until January 1, 2020. After the last release, this version will receive no support [1].

So, this issue should be targeted for a next AOO build.

[1] https://www.python.org/dev/peps/pep-0373/
Comment 17 Pedro Giffuni 2019-07-29 16:03:27 UTC
For the record:

Modern versions of Python have deprecated the old version of the MSVC compiler used to build AOO on Windows.
https://wiki.python.org/moin/WindowsCompilers

For python 3.5/3.6 we would need to be using MSVC 14.0.

But the problem is not just the Windows compiler: it used to be that we could build AOO with an external 3.x python package but, at least on FreeBSD, it is not possible to build AOO with the external python >= 3.6 anymore. Probably some porting is necessary for newer versions (I don't really have details).
Comment 18 damjan 2019-07-30 06:22:02 UTC
(In reply to Pedro Giffuni from comment #17)
> But the problem is not just the Windows compiler: it used to be that we
> could build AOO with an external 3.x python package but, at least on
> FreeBSD, it is not possible to build AOO with the external python >= 3.6
> anymore. Probably some porting is necessary for newer versions (I don't
> really have details).

Can you please provide more details on that building with Python >= 3.6 on FreeBSD part? I couldn't find any info on it anywhere.
Comment 19 Pedro Giffuni 2020-01-10 16:21:23 UTC
(In reply to damjan from comment #18)
> (In reply to Pedro Giffuni from comment #17)
> > But the problem is not just the Windows compiler: it used to be that we
> > could build AOO with an external 3.x python package but, at least on
> > FreeBSD, it is not possible to build AOO with the external python >= 3.6
> > anymore. Probably some porting is necessary for newer versions (I don't
> > really have details).
> 
> Can you please provide more details on that building with Python >= 3.6 on
> FreeBSD part? I couldn't find any info on it anywhere.

Hi;

Sorry for the delay. There is a FreeBSD bug report:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229408
Comment 20 Pedro Giffuni 2020-01-11 21:11:41 UTC
(In reply to Pedro Giffuni from comment #19)
> (In reply to damjan from comment #18)
...
> > 
> > Can you please provide more details on that building with Python >= 3.6 on
> > FreeBSD part? I couldn't find any info on it anywhere.
> 
> Hi;
> 
> Sorry for the delay. There is a FreeBSD bug report:
> 
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229408

Details / smoking gun in the pyuno module:

....
[ build CXX ] pyuno/source/module/pyuno_util
/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/pyuno/source/loader/pyuno_loader.cxx:117:5: error: no matching function for call to 'Py_SetPythonHome'
    Py_SetPythonHome( o.pData->buffer);
    ^~~~~~~~~~~~~~~~
/usr/local/include/python3.7m/pylifecycle.h:43:18: note: candidate function not viable: no known conversion from 'sal_Char [1]' to 'const wchar_t *' for 1st argument
PyAPI_FUNC(void) Py_SetPythonHome(const wchar_t *);
                 ^
1 error generated.
Comment 21 Pedro Giffuni 2020-01-13 15:45:22 UTC
Created attachment 86813 [details]
Proof-of-concept fix

On python 3 and newer, Py_SetPythonHome() uses a widechar argument.

Current documentation suggests using Py_DecodeLocale() which is only available starting with Python 3.5.

I am currently testing the patch, I plan to submit it via a github pull request.
Comment 22 Pedro Giffuni 2020-01-13 15:47:47 UTC
For the record, hanya's patches are likely to be obsolete, or simply wont apply cleanly, since have updated Python 2.7.
Comment 23 oooforum (fr) 2020-01-13 16:14:44 UTC
This issue is targeted for 4.2.0
Did you backported for 4.1.8?
Comment 24 Pedro Giffuni 2020-01-13 16:28:15 UTC
Created attachment 86814 [details]
Proof-of-concept fix

(parenthesis oops)
Comment 25 Pedro Giffuni 2020-01-13 22:54:22 UTC
Comment on attachment 86814 [details]
Proof-of-concept fix

Nevermind, this is very broken.
Comment 26 Pedro Giffuni 2020-01-14 02:14:15 UTC
Created attachment 86815 [details]
python3 fixes

This seems to take the build much further but it needs review.
Comment 27 damjan 2020-01-25 10:01:17 UTC
Python 3 support with system-provided Python is now in trunk, implemented in this range of commits:
a5f48dc26160473ef78ab88ce9cc4c1e0bc5f5e1..aec63fa3669010e9bec513e6cd55a25fc335d90e

Internal Python 3, is still TODO.
Comment 28 Eng. Fábio Paulon 2020-04-03 14:29:01 UTC
I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.
Comment 29 Eng. Fábio Paulon 2020-04-03 14:31:48 UTC
I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.
Comment 30 Eng. Fábio Paulon 2020-04-03 14:38:24 UTC
I suggest to include a path feature to configure Python version (such as Java) and an included version (default set) into OpenOffice. The user could set path or leave the default option.
I have some other applications that use a unique Python 3 path: GIMP, LibreCAD, FreeCAD, Scribus, and others, all are using (with some myself modification) the same Python version (the last one) interpreter. I already suggest this same feature for this others applications. I'm waiting the approval and the update.

Thank you.