Apache OpenOffice (AOO) Bugzilla – Issue 74807
OSX mapfile generation does not seem to be parallel-build safe
Last modified: 2013-08-07 15:34:52 UTC
Doing a -P2 / MAXPROCESS=2 build on osx breaks in jvmaccess and cpputools, with missing RTTI symbols from salhelper. Building salhelper again fixes this problem. Further along the build, sax, stoc, ucbhelper etc. suffer from the same problem. Appears that the mapfile filtering/generation process has races...
Example: in cpputools, the following error happens: /usr/bin/ld: Undefined symbols: typeinfo for cppu::OWeakObject
Some documentation of the problem on a MacBook Core Duo with 2GB ram. Attached is a zip containing two directories, one for the failed build, one for the successful rebuild, and a log of a full rebuild without MAXPROCESS.) ccache is hot MAXPROCESS=2 the build was started with the following command: time dmake -P4 >&1 | tee build.log The section building salhelper that I referred to earlier begins on line 12735 of the failed log. For some reason the build log does not show the error that stopped the build, so there is another file - console.log - that has the error and the last full command in the build log. The salhelper build retained the same environment - including MAXPROCESS=2. The console.log shows the commands to make it and the output. The misc directory shows the results - __ZN9salhelper21SimpleReferenceObjectnwEm is missing from the failed, but present in the salhelper_build.
Created attachment 43582 [details] build using -P4, error with MAXPROCESS=2, OK no MAXPROCESS
ause: looks like I need your expertise ;-) On Mac OS X, we have to process the Linux map file to expand symbols using regular expressions. We use the binary objects t get all symbols and then manually match the regexps from Linux map file to them and finally create Mac OS X mapfile and use it. But when doing parallel builds: tr -d "\015" < gcc3_linux_intel.map | awk -f /Users/pavel/BUILD/X11/BuildDir/ ooo_SRC680_m205_src/solenv/bin/addsym.awk > ../unxmacxi.pro/misc/ gcc3_linux_intel_uno_salhelpergcc3.map cat ../unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map | awk -f /Users/pavel/BUILD/X11/ BuildDir/ooo_SRC680_m205_src/solenv/bin/unxmap-to-macosx-explist.awk | grep -v "\*\|?" > ../ unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map.exported-symbols cat ../unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map | awk -f /Users/pavel/BUILD/X11/ BuildDir/ooo_SRC680_m205_src/solenv/bin/unxmap-to-macosx-explist.awk | grep "\*\|?" > ../ unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map.symbols-regexp cat ../unxmacxi.pro/slb/salhelper.lib | sed s\#unxmacxi.pro\#../unxmacxi.pro\#g | xargs -n1 nm -gx | /Users/pavel/BUILD/X11/BuildDir/ooo_SRC680_m205_src/solenv/bin/addsym-macosx.sh ../ unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map.symbols-regexp ../unxmacxi.pro/misc/ symbols-regexp.tmp >> ../unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map.exported- symbols cat: ../unxmacxi.pro/slb/salhelper.lib: No such file or directory cp ../unxmacxi.pro/misc/gcc3_linux_intel_uno_salhelpergcc3.map.exported-symbols ../unxmacxi.pro/ misc/gcc3_linux_intel_uno_salhelpergcc3.map ------------- xml2cmp -func ../unxmacxi.pro/misc/uno_salhelpergcc3_description.cxx ../unxmacxi.pro/misc/ uno_salhelpergcc3.xml ------------------------------ File ../unxmacxi.pro/misc/uno_salhelpergcc3_description.cxx with component_getDescriptionFunc() is created now. Making: ../unxmacxi.pro/slb/salhelper.lib Thus it happened to us that we want to process salhelper.lib which should contain: unxmacxi.pro/slo/condition.o unxmacxi.pro/slo/dynload.o unxmacxi.pro/slo/simplereferenceobject.o But it doesn't exist at that time. We have to add a dependency somewhere, but I don't see where...
Patch by ause attached. Helped here. thb: can you please test it as well?
Created attachment 43657 [details] Add dependencies
this patch was just a quick shot. it introduces unrequired dependencies for other platforms... i tend to revisit these lines anyway.
@pjanik: yay, seems to indeed fix the problems. At least my build is now way past cpputools...
the patch fixes the issue on my MacBook
taking over to commit fix on cws ause077
Really take over :-)
Created attachment 44566 [details] changes in tg_shl.mk
fixed in cws ause077
please verify
Verified in CWS ause077.
This issue was thought to be resolved in m211, but I still have the problem. The build was done with sources from the CVS tunnel, and I verified that patch i74807.diff (attached to this issue) was applied. I built with: MAXPROCESS=4 command line (in instsetoo_native) build --all -P4 results: ld: Undefined symbols: __ZN9salhelper21SimpleReferenceObjectnwEm __ZTIN9salhelper21SimpleReferenceObjectE /usr/bin/libtool: internal link edit command failed dmake: Error code 1, while making '../unxmacxi.pro/lib/libuno_cppuhelpergcc3.dylib.3' ---* tg_merge.mk *--- 2 module(s): jvmaccess cppuhelper need(s) to be rebuilt I then applied the other patch attached to this issue (solenv.diff), and built again. This time without the undefined symbol error. I am attaching the build log from the failure, and I can attach the successful one if needed, but it is big so I won't for now.
I found that I still did not have the m211 sources in solenv. The problem is really as described in http://www.openoffice.org/issues/show_bug.cgi?id=76944 Sorry for the noise, this is fixed.
Verified on master. Closing.