Apache OpenOffice (AOO) Bugzilla – Issue 74914
Reusing old 8.3 names and File Table Keys from a previous MSI installer
Last modified: 2007-07-19 09:21:46 UTC
This problem occurs especially when building multi-language Windows Installer (MSI) installers, but could well occur also with just single-language installers, although not to the same extent. We want to support upgrading from one installed version of OOo to another using Windwows Installer Patching and Upgrading (.msp files). For this to work smoothly and the .msp files to be as small as possible, it is essential that a file has the same key (FTK = File Table Key) in both the old and new MSI database. Currently the Perl scripts in OOo that built the MSI database just make the FTKs unique by incrementing and suffixing a counter when files with the same basename occur in different folders. Especially if the new and old MSI database have different language sets, this leads to the situation that the same FTK in the two databases refer to different files. The patch generating msimsp program doesn't understand this, and is not capable of generating a (binary) diff for the file in question, but includes it as a whole. A similar situation exists for the 8.3 "short" names for files and folders. Currently just a running counter is used to generate those. The problem described above is not restricted just to the multi-language installers that Novell builds. It can occur also with single-language MSI databases, if a file with the same basename as an existing file is introduced in the new version. To fix this problem, we add the possibility to pass an environment variable "PREVIOUS_IDT_DIR" that points to the folder from the old build where the .idt files are. If this environment variable exist, the old File.idt and Director.idt files are read and a couple of mapping tables initialised from them. When the FTK keys and 8.3 names are generated, the Perl code first checks if there are saved mappings from the old database, and uses those. Patch will follow.
@ is: Please have a look.
For MSI patching to work, one other thing that needs to be taken care of is obviously to reuse the same GUIDs for components each time. That can be taken care of by simply copying the components.txt produce last time (if any) in instsetoo_native/wntmsci10.pro/msi/OpenOffice/msi/info_files/blabla on top of the otherwise empty instsetoo_native/inc_openoffice/windows/msi_templates/components.txt. The patch that follows will not bother with that, as it is not clear to me whether different distributors of OOo for Windows want to do that in the same way or not. Also, one needs to use the same productcode each time. Just patching a few lines in msiglobal.pm takes care of that. Thirdly, one should use just one cab file, to avoid the possibility a file that gets patched is in different cab file in the old and new version. If I recall correctly, MSI doesn't like that.
Created attachment 43417 [details] Suggested patch
Hopefully I managed to put into the above patch just those diff chunks that are relevant for this issue. The corresponding diff file in ooo-build is novell-win32-msi-patchability.diff. If upstream feels this issue is something they don't want to bother with at all, please say so, so I don't have to bother with trying to upstream this stuff...
Ah, one thing I forgot: For patching to work, the MsiFileHash tble needs to be populated. Will file that as a separate issue that this depends on.
Great, I will investigate this. Do you create a patch that updates from 2.0 to 2.2 AND from 2.1 to 2.2? Or are this different patches? I think, that the environment variable "PREVIOUS_IDT_DIR" is problematic, because an older packing environment is referenced. Target 2.3 seems to be appropriate.
So far I have just created two kinds of .msp patches: Ones that upgrade from several slightly different 2.0.4 builds to one newest 2.0.4 build, and ones that upgrade from one 2.0.4 build to a 2.1.0 build. It should be quite possible to have a .msp patch that upgrades from either 2.0.4 or 2.1 to 2.2. See below for general requirements though. Depending on how many changes there are, and how many upgrade combinations are covered by the same .msp file, it might well be that the .msp file ends up being as big or larger than an installer for the new version, and in that case it obviously makes little sense. For .msp patching to work without requiring access to the original installation source, the installers one is going to enable patching *from* must have included the MsiFileHash table. So it is in general not possible to produce a standalone .msp that would patch some old installation to a newer, if the old installer didn't contain a populated MsiFileHash table. One thing I need to mention is that unfortunately the name of the installation folder must be the same (like "OpenOffice.org 2.0") in both the "old" and "new" versions for patching to work sensibly.
Integrated into cws native82. An environment variable PREPARE_WINPATCH has to be set, to read the idt files located in the directory PREVIOUS_IDT_DIR.
The process is integrated in cws native82. To use this process, the following settings have to be done: PREPARE_WINPATCH has to be set PREVIOUS_IDT_DIR has to be set and has to contain idt files of older installation set. components.txt has to be copied from older installation set (directory info_files) to inc_openoffice\windows\msi_templates IS -> OF: Please control patches created in cws native82
OF: I can't see any problem in cws native82.
OF: Patches are ok in src213.
I noticed a probem in my patch: In idtglobal.pm in the sub get_next_free_number_with_hash(), the lookup from savedrev83mapping needs to use the extension too in the key, as the whole 8.3 name is used when populating savedrev83mapping. If this is not fixed, the generated File table might have 8.3 names that overlap ones from the previous File table if new files have been added (for instance because of added languages). (As such, clashing 8.3 names is not detected by Windows Installer. It gladly accepts installers where multiple files in the same folder are given the same 8.3 name. If we could just ignore the very small chance that somebody actually installs OO.o on a 8.3-only file system, we could scrap all the code generating 8.3 names and just store a fixed name like "A.B" as the 8.3 name for all non-8.3 names... If I understood correctly, that is what Red Hat's installer for Mugshot on Windows, for instance, does. They don't bother to generate any real unique 8.3 names. But I digress.) The fix is to add a third parameter to get_next_free_number_with_hash(): sub get_next_free_number_with_hash { my ($name, $shortnamesref, $ext) = @_; and a few lines below then use that too in lookups from savedrev83mapping: if ( exists($shortnamesref->{$newname}) || exists($installer::globals::savedrev83mapping{$newname.$ext}) ) { $alreadyexists = 1; } Then we need to pass the extension (already uppercased and truncated) in the calls to get_next_free_number_with_hash(), in five places in the sub make_eight_three_conform_with_hash(). First, three instances of: ($number, $saved) = get_next_free_number_with_hash($name, $shortnamesref, '.'.uc($extension)); then, in the remaining two calls just pass an empty string: ( $number, $saved ) = get_next_free_number_with_hash($name, $shortnamesref, ''); Hopefully this verbal free-form patch description is clear enough.
OF @ IS: Please review.
I will try to include this into cws native89
Shifting from native89 to native93, but still OOo 2.3
Fixed in cws native93
Patch included into cws native93 -> verified.
Fix included in m221, closing issue.