Apache OpenOffice (AOO) Bugzilla – Issue 35742
build conditionals ...
Last modified: 2005-08-10 14:19:06 UTC
So - this is a simple version of Dan's patch (hooking into the existing OOo: / SO: conditional code) that provides for: $(WITH_FOO):modulename dependency rules in prj/build.lst - using the environment to change the build. The code changes are minimal & clear - and this allows eg. $(WITH_BINFILTER):binfilter as a 1st pass, and (in future) should allow much simpler use of system libraries without excessive makefile.mk hackage.
Created attachment 18512 [details] build.pl patch
Created attachment 18513 [details] binfilter sample patch
And, of course, I screwed up the 2nd fragment of the 1st patch, it should be: @@ -2034,7 +2057,10 @@ my @modules_to_build; #my $new_modules = ''; foreach (@mod_array) { - if (/(\w+):(\S+)/o) { + if (/\$\((\w+)\):(\S+)/o) { + push(@modules_to_build, $2) if (get_env_boolean($1,1)); + next; + } elsif (/(\w+):(\S+)/o) { push(@modules_to_build, $2) if (defined $build_modes{$1}); next; }; ie. extra brackets.
reassigned.
adding me to CC:
*** Issue 28202 has been marked as a duplicate of this issue. ***
.
It turns out I can't type: -+ToFile( "WITH_BINFITLER", "@WITH_BINFILTER@", "e" ); ++ToFile( "WITH_BINFILTER", "@WITH_BINFILTER@", "e" ); :-)
Two more patches are needed to make the binfilters really optional; otherwise the instsetoo_native fails. I'll attach them.
Created attachment 18989 [details] Disable binfilters in scp2.
Created attachment 18990 [details] Disable creation of legacy_binfilters.rdb.
Created attachment 19186 [details] clobber cludges in perl installer
change type to patch to get it on various radars.
Created attachment 19320 [details] improved conditional patch
Later improved build-conditional patch much better: * re-factors some cut/paste coding * adds negation to build rules eg. !(SYSTEM_LIBXML):libxml * adds line-breaking to format The line breaking would be _particularly_ helpful for release engineers merging these files & external people patching them; this allows eg.: --- instsetoo_native/prj/build.lst 2 Nov 2004 09:27:38 -0000 1.12 +++ instsetoo_native/prj/build.lst 15 Nov 2004 14:54:53 -0000 @@ -1,4 +1,21 @@ -oon instsetoo_native : helpcontent2 UnoControls basctl configmgr cpputools dbaccess desktop eventattacher extensions extras fileaccess forms io package padmin readlicense_oo remotebridges scaddins scp2 setup_native SO:epm shell sc sch chart2 sd starmath sw ucb wizards officecfg xmlhelp MathMLDTD dictionaries bitstream_vera_fonts filter psprint_config fpicker odk embedserv pyuno crashrep scripting embeddedobj hwpfilter unoxml slideshow binfilter msfontextract xmlsecurity NULL +oon instsetoo_native : \ + helpcontent2 UnoControls basctl \ + configmgr cpputools dbaccess \ + desktop eventattacher extensions \ + extras fileaccess forms io \ + package padmin readlicense_oo \ + remotebridges scaddins scp2 \ + setup_native SO:epm shell sc \ + sch chart2 sd starmath sw ucb \ + wizards officecfg xmlhelp MathMLDTD \ + dictionaries filter \ + psprint_config fpicker odk embedserv \ + pyuno crashrep scripting embeddedobj \ + hwpfilter unoxml slideshow \ + msfontextract xmlsecurity \ + $(WITH_FONTS):bitstream_vera_fonts \ + $(WITH_BINFILTER):binfilter \ + NULL oon instsetoo_native usr1 - all oon_mkout NULL oon instsetoo_native\packimages nmake - all oon_pack NULL oon instsetoo_native\packconfig nmake - all oon_config NULL
although; this needs to replace \\\S with \\\s in the continuation code it seems.
I did an error in binfilter-solenv.diff; there has to be "ne" instead of "!=", of course...
I committed this little lot, with a tweak to allow the default value for negated parameters to cope gracefully (always eventually true) for the Sun build system - with it's fewer environmentally adjusted things. All in CWS buildcond01 now, the patches I applied build fine for me; but some eg. Win32 QA appreciated, perhaps I can enlist Volker, or someone else ? ...
I can build an installset if you like. Just add me to the members of that cws so that I don't forget it. ;)
Oh, someone already did that ;)
to mmeeks: the build list format changes cannot be accepted, because some our inhouse tools would not understand the new format. We plan xml file format for build.lst quite soon, so there's no use to accomodate our tools to changes in outdated format.
I do not see the reason why the env. variables should have two or even more values. I think that it's an overkill, it would be sufficient just to have a varible defined or not, and write something like (in pick_for_build_type procedure, line 2118, r1.127): foreach (@mod_array) { if (/(\w+):(\S+)/o) { next if (!defined $build_modes{$1}); $_ = $2; }; next if (defined $ENV{'DISABLE_BUILD_' . uc($_)}); push(@modules_to_build, $_); } No more changes in build.pl are needed, actually. Of cause, one should accomodate configure.in for this concept.
Martin - some backup appreciated, since you read this & thought it was doable / useful etc. VG: I see the planned XML file format - but a plan doesn't help me with my desire to shrink the 2.0 source-downloads to a manageable size for developers quickly. Furthermore, I see no use whatsoever of any build.xlists in the tree. > I do not see the reason why the env. variables should have two or > even more values. I think that it's an overkill, it would be sufficient > just to have a varible defined or not. Of cause, one should > accomodate configure.in for this concept. Well - the alternative to these few lines of code here, is a huge change to the configure system & perhaps a hundred or so makefile.mk's to reverse / alter the polarity of the conditionals there - or - export other variables that would stuff the environment yet fuller with cruft [ there has to be some DOS limit we'll hit there at some stage ]. So ... - can you enlarge as to what your in-house problems are ? - you should be using build.pl to process the files [ surely ], and [ as you see from the code ], I did work to ensure that the default state in your build system [ with no relevant environment variables defined ] - should be to include the module in the build regardless of the polarity of the conditional etc. - is there some further requirement you have there ?
mmeeks: I do not see how build.lst format changes can help to reduce downloads. There is no build.xlists just because od some delays with our inhouse tools. As to environment variables, as far as can I see, there are no changes in makefiles needed (correct me if it's not so), the only file engaged is configure.in. I admit that there the changes can be sufficient, but they are all homogenuos and don't require complicated logic. No additional environment variables needed. Moreover, I think such approach would make the build process more clear. We have some tools, which we use to automatize our builds. Indeed, our developers use only build tool to perform builds, in RE we have an atomatic build process, which rely on the old build.lst format.
mh->vg: I consider this as a valid patch for a valid concern. providing more managable downloads is a vaild approach to address ease of building. This would attract new contributors.
> mmeeks: I do not see how build.lst format changes can help to reduce > downloads. so - we can split the source base to remove eg. 30+Mb of (compressed) mozilla binary that is not useful for many systems. Of course, we could do that by leaving most of the files in the moz/ directory there - but it is more elegant / simple just to split that top-level directory out => we need these build changes. The binfilter likewise. > As to environment variables, as far as can I see, there are no changes in > makefiles needed (correct me if it's not so), the only file engaged is > configure.in. Currently there is loads of code of the form: .IF "$(SYSTEM_MOZILLA)" != "NO" @echo "Do nothing for mozilla" .ENDIF scattered through many, many makefiles. Furthermore - since there is no build.pl conditional support - this becomes really hard & inelegant for projects with lots of makefiles - the patches are large & painful to manage. we could change configure.in to export (in addition to SYSTEM_MOZILLA) some 'DONTBUILD_MOZILLA' type environment variable that could be used to be set/not-set but it is a hundred of lines of extra (duplicate) configure code, many more confusing & overlapping environment variables & well - just more pain all around. > I admit that there the changes can be sufficient, but they are all > homogenuos and don't require complicated logic. No additional environment > variables needed. I want to use $(SYSTEM_MOZILLA) as it is, in it's current form really. > We have some tools, which we use to automatize our builds. Indeed, our > developers use only build tool to perform builds, in RE we have an atomatic > build process, which rely on the old build.lst format. Can you give more details of these tools - has there really been a cut/paste job done on build.pl ? and if so - why ? surely it doesn't make much sense to do that sort of thing. Martin - what is your suggested action here then ? 'vq' reports a compiling Win32 build - vq can you confirm installing works nicely too - pwrt. binfilter. Clearly - I'd like to get this in as a first step, I have about 6 other conditional-* patches that depend on this ready to go into another cws.
mmeeks: Ok, i see your point. Not the best solution in my opinion, but I can live with it. As to our internanal tools: some of them are written in c(c++) and are quite complicated. We cannot refrain from using them. There are no copy-paste excepts (normally), so build.lst format changes cannot be accepted. But: we accelerate our efforts on XML format.
add me to cc:
regarding the build.lst format change: wouldn't it be sufficient for now to use the existing variable "BUILD_TYPE", set it according the desired "--with[out]-somthing" switches in configure and add the prefix "BUILD_SOMETHIG:something" to the build lists definig this dependency? this could avoid the format change. e.g. "--without-system-moz" => BUILD_TYPE += MOZ and in "xmlsecurity/prj/build.lst" xs xmlsecurity : xmloff unotools offapi unoil svx MOZ:moz libxmlsec NULL did i miss something?
> regarding the build.lst format change: wouldn't it be sufficient for now to > use the existing variable "BUILD_TYPE", set it according the desired > "--with[out]-somthing" switches in configure and add > the prefix "BUILD_SOMETHIG:something" to the build lists definig this > dependency? this could avoid the format change. What you suggest is almost exactly what this patch does; however, currently BUILD_TYPE defines a single build conditional; either 'OO' or 'SO' (or something else), and not a plural set of conditionals. This patch changes that; so you can do [foo]:module - extending the syntax. Ok - so admittedly it uses $(FOO) - to make it more clear this is an environment variable being used as distinct from some 'magic' foo - if you really hate the $() I can trivially remove them from the patch - however the negation via '!' is really critical to save lots of scattered makefile.mk code changes (as discussed ad nauseum). I honestly can't see the: "it is utterly impossible for us to change our internal tooling, so go away" argument making sense since the internal tooling already has to parse / enterpret [foo]:module; [ a new feature since 1.1.x ] and it's simply a matter of making it default to 'always build it' if [foo] is not 'OO'; surely - this should be trivial - and fresh in people's minds since the recent 'SO:foo' vs. 'OO:foo' change ? > e.g. "--without-system-moz" => BUILD_TYPE += MOZ That looks like it would require changing Sun's internal build environment to expand their setting of 'BUILD_TYPE' to :MOZ:FOO:BAA:BAZ _and_ - worse - every time a new conditional was added, it would require a change to your internal build environment. By contrast - the suggested patch does not suffer from this limitation, by using smart defaults when environment variables are not set. I _really_ would like to move on this; however vg's "so build.lst format changes cannot be accepted." Fills me with certainty that collaborating here is not going to be easy. Thus - I would like to propse that in addition to this patch; we create a parallel 'build.lst2' format - we duplicate all the build.lst files, leaving them to bit-rot until Sun sort their act out; so that we can get this committed, shrink / split the source downloads & well - move on. Of course - this will be a huge maintenance headache - particularly since the line-continuation feature is _really_ useful for patching / reading / seeing what changed in long build dependency lists - but ... Martin - should I create an 'every project' cws to copy build.lst -> build.lst2s ? or is such a thing something that rel-eng could do directly on CVS HEAD [ the cws overhead for just that would be huge ] [ of course, we could also copy build.pl if necessary ]. I imagine there are other things that can be improved in the build.lst2 format as well that are perhaps not possible currently.
> Thus - I would like to propse that in addition to this patch; we create a > parallel 'build.lst2' format - we duplicate all the build.lst files, leaving > them to bit-rot until Sun sort their act out; ... Not necessarily. The nasty but low impact solution that comes to mind is that the the build.lst files for SO builds are modified with an easy sed/perl/whatever-script that just removes the "non-SO:" before non-SO:module. Or if you don't want it during the build, create <module>/prj/build_gen.lst by duplicating the current build.lst files, add the FOO: whereever needed, start your "build.lst-generate.pl" script, commit all changed build.lst / build_gen.lst files and teach build.pl to use the build_gen.lst for OOo builds. Not a maintainance nightmare, only a bad dream. ;) Should work as a stop-gap solution until the xml version is ready.
To mmeeks (Sun Nov 28 20:12:36): Just to explain our (Hamburg RE) current situation: The solution Ause suggested would require no change at all for us. 1) In the current implementaion of BUILD_TYPE setting no BUILD_TYPE means 'build everything' and that's just what we do (we have to build SO as well as OOo stuff in order to create SO and OOo install sets - the later ones needed to provide milestone builds). 2) BUILD_TYPE is not restricted to SO and OOo. You already can use any BUILD_TYPE - just try. In contrast your prosed format change (splitting up the first dependency line) would require adaptions of various tools. The magnitude of build list parsers was the most important reason for us to change to xml based build lists. That's why I strongly support Auses suggestion to utilize BUILD_TYPE environment variables. Of course it would have advantages to have dependencies on multiple lines. OTOH your proposal would address just halve of the mess, as on every other line we still have all (inner module) dependencies in one single line. Therefore to get this resolved I would suggest to wait for the xml based build lists. Concerning your proposed 'patch' to hold build.lst2 files in parallel: please be aware that this would really be a permanent maintenance nightmare. We (Hamburg) would use old build.lst files only and not even notice if your new files get inconsistent. Hamburg developers creating new or removing old directories would change build.lst files first and are likely to forget your extra files (in fact they quite often forget even build.lst files, but this gets noticed by us). For me this does not sound like a good idea.
Ok; so BUILD_TYPE may not be restricted to SO or OO but it is restricted to _one_ value for the entire OO.o build; nevertheless - essentially my patch (as I explained) does just change the BUILD_TYPE. However - I can make the patch more ugly so the syntax is: notSYSTEM_MOZILLA:moz instead of: !$(SYSTEM_MOZILLA):moz special case OO and SO & check for environment variables otherwise. Of course - I can also remove the line continuation - which is an _extremely_ useful feature - since inevitably we have to patch these lists anyway ourselves. Is that acceptable ? Finally - I am _extremely_ scared by the potential XML format; particularly since these project lists have to be patched by us - and I've suffered enough pain from patching configuration .xcu/.xcs files. I would love to see a doc. on the proposed XML syntax, pwrt. suggested white-space & absence of any trendy / over-complex XML features such as XSLT etc. it would be nice if the result was still comprehensible. I'm particularly keen that the file is still reliably patchable with reasonable context, by avoiding excessive new-lines & nesting: <node dep="svx"/> <node dep="svt"/> + <node dep="rtl"/> <node dep="sal"/> <node dep="baa"/> Anyhow - I'll re-work the patch to make it use AlphaOnlyThisIsAllONE_CONDITIONAL type variables followed by a ':'
Michael, Sorry, but I still do not understand why you prefer your patch solution to the existing BUILD_TYPE mechanism. BUILD_TYPE is _not_ restricted to one value for a build. Rüdiger
Ah; apologies - so I didn't notice that the build-type can be multiple values; So - I guess; since I now know there are no problems with the Sun internal build tools with adding random/crazy new BUILD_TYPE conditionals of this form - I just need to re-write the patch / re-validate to generate all of this in configure. I will as a convention; use the module name as the conditional where possible; eg. MOZ:moz BINFILTER:binfilter etc. Hope that's ok; I'll re-generate the diff as/when I get time & commit to CWS buildcond01.
Since this was such a cock-up; I'll bin the buildcond01 cws and create a new cws to do this work on: buildcond02 :-)
Since - I guess the buildcond02 cws looks scary - a lot of modules touched; I attach a patch showing the complete change-set to demonstrate how extremely trivial the majority of the changes are;
Created attachment 20323 [details] full cws diff
fixed more elegantly in buildcond02
verified the "build.lst" changes
would you mind to change +.IF "$(WITH_MYSPELL_DICTS)" != "YES" +SCPDEFS+=-DWITHOUT_MYSPELL_DICTS +.ENDIF to +.IF "$(WITH_MYSPELL_DICTS)" == "" +SCPDEFS+=-DWITHOUT_MYSPELL_DICTS +.ENDIF to avoid "Yes", "yes", "yeS" cinfusion?
hjs: ok - I updated the conditional, so it should work in your build system (without that environment variable set) - sorry for that [ obviously not paying attention somewhere ]. I used == 'NO' as is used over the place for the other conditionals. [ I'd prefer not to get into fixing the whole yes/no/true/false thing generically across the code just now ]. I hope that's ok. Does that mean you did the relevant QA on it ? :-)
<ause> michael_: buildcond02 is ok from my side. could you get those issues to verified? <michael_> ause: sure, let me do that ASAP
close
This issue is integrated into a build for OOo2.0, but the 'target milestone' isn't set. To have a better overview about all fixed and integrated tasks in OOo2.0, I set the field 'target milestone' to OOo2.0.