Issue 74914 - Reusing old 8.3 names and File Table Keys from a previous MSI installer
Summary: Reusing old 8.3 names and File Table Keys from a previous MSI installer
Status: CLOSED FIXED
Alias: None
Product: Installation
Classification: Application
Component: code (show other issues)
Version: OOo 2.1
Hardware: All Windows, all
: P3 Trivial (vote)
Target Milestone: OOo 2.3
Assignee: ingo.schmidt-rosbiegal
QA Contact: issues@installation
URL:
Keywords:
Depends on: 74921
Blocks:
  Show dependency tree
 
Reported: 2007-02-27 13:49 UTC by tml
Modified: 2007-07-19 09:21 UTC (History)
1 user (show)

See Also:
Issue Type: PATCH
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Suggested patch (12.01 KB, patch)
2007-02-27 15:37 UTC, tml
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description tml 2007-02-27 13:49:41 UTC
This problem occurs especially when building multi-language Windows Installer
(MSI) installers, but could well occur also with just single-language
installers, although not to the same extent.

We want to support upgrading from one installed version of OOo to another using
Windwows Installer Patching and Upgrading (.msp files). For this to work
smoothly and the .msp files to be as small as possible, it is essential that a
file has the same key (FTK = File Table Key) in both the old and new MSI
database. Currently the Perl scripts in OOo that built the MSI database just
make the FTKs unique by incrementing and suffixing a counter when files with the
same basename occur in different folders.

Especially if the new and old MSI database have different language sets, this
leads to the situation that the same FTK in the two databases refer to different
files. The patch generating msimsp program doesn't understand this, and is not
capable of generating a (binary) diff for the file in question, but includes it
as a whole.

A similar situation exists for the 8.3 "short" names for files and folders.
Currently just a running counter is used to generate those.

The problem described above is not restricted just to the multi-language
installers that Novell builds. It can occur also with single-language MSI
databases, if a file with the same basename as an existing file is introduced in
the new version.

To fix this problem, we add the possibility to pass an environment variable
"PREVIOUS_IDT_DIR" that points to the folder from the old build where the .idt
files are. If this environment variable exist, the old File.idt and Director.idt
files are read and a couple of mapping tables initialised from them. When the
FTK keys and 8.3 names are generated, the Perl code first checks if there are
saved mappings from the old database, and uses those.

Patch will follow.
Comment 1 Olaf Felka 2007-02-27 13:57:59 UTC
@ is: Please have a look.
Comment 2 tml 2007-02-27 15:37:02 UTC
For MSI patching to work, one other thing that needs to be taken care of is
obviously to reuse the same GUIDs for components each time. That can be taken
care of by simply copying the components.txt produce last time (if any) in
instsetoo_native/wntmsci10.pro/msi/OpenOffice/msi/info_files/blabla on top of
the otherwise empty
instsetoo_native/inc_openoffice/windows/msi_templates/components.txt. The patch
that follows will not bother with that, as it is not clear to me whether
different distributors of OOo for Windows want to do that in the same way or not.

Also, one needs to use the same productcode each time. Just patching a few lines
in msiglobal.pm takes care of that.

Thirdly, one should use just one cab file, to avoid the possibility a file that
gets patched is in different cab file in the old and new version. If I recall
correctly, MSI doesn't like that.
Comment 3 tml 2007-02-27 15:37:57 UTC
Created attachment 43417 [details]
Suggested patch
Comment 4 tml 2007-02-27 15:40:46 UTC
Hopefully I managed to put into the above patch just those diff chunks that are
relevant for this issue. The corresponding diff file in ooo-build is
novell-win32-msi-patchability.diff.

If upstream feels this issue is something they don't want to bother with at all,
please say so, so I don't have to bother with trying to upstream this stuff...
Comment 5 tml 2007-02-27 15:42:32 UTC
Ah, one thing I forgot: For patching to work, the MsiFileHash tble needs to be
populated. Will file that as a separate issue that this depends on.
Comment 6 ingo.schmidt-rosbiegal 2007-03-02 13:39:58 UTC
Great, I will investigate this. Do you create a patch that updates from 2.0 to
2.2 AND from 2.1 to 2.2? Or are this different patches?
I think, that the environment variable "PREVIOUS_IDT_DIR" is problematic,
because an older packing environment is referenced.
Target 2.3 seems to be appropriate.
Comment 7 tml 2007-03-02 13:57:06 UTC
So far I have just created two kinds of .msp patches: Ones that upgrade from
several slightly different 2.0.4 builds to one newest 2.0.4 build, and ones that
upgrade from one 2.0.4 build to a 2.1.0 build.

It should be quite possible to have a .msp patch that upgrades from either 2.0.4
or 2.1 to 2.2. See below for general requirements though.

Depending on how many changes there are, and how many upgrade combinations are
covered by the same .msp file, it might well be that the .msp file ends up being
as big or larger than an installer for the new version, and in that case it
obviously makes little sense.

For .msp patching to work without requiring access to the original installation
source, the installers one is going to enable patching *from* must have included
the MsiFileHash table. So it is in general not possible to produce a standalone
.msp that would patch some old installation to a newer, if the old installer
didn't contain a populated MsiFileHash table.

One thing I need to mention is that unfortunately the name of the installation
folder must be the same (like "OpenOffice.org 2.0") in both the "old" and "new"
versions for patching to work sensibly.
Comment 8 ingo.schmidt-rosbiegal 2007-03-22 14:17:59 UTC
Integrated into cws native82. An environment variable PREPARE_WINPATCH has to be
set, to read the idt files located in the directory PREVIOUS_IDT_DIR.
Comment 9 ingo.schmidt-rosbiegal 2007-04-27 10:51:37 UTC
The process is integrated in cws native82. 
To use this process, the following settings have to be done:
PREPARE_WINPATCH has to be set
PREVIOUS_IDT_DIR has to be set and has to contain idt files of older
installation set.
components.txt has to be copied from older installation set (directory
info_files) to inc_openoffice\windows\msi_templates

IS -> OF: Please control patches created in cws native82
Comment 10 Olaf Felka 2007-05-03 14:05:39 UTC
OF: I can't see any problem in cws native82.
Comment 11 Olaf Felka 2007-05-31 13:31:42 UTC
OF: Patches are ok in src213.
Comment 12 tml 2007-06-08 09:51:10 UTC
I noticed a probem in my patch: In idtglobal.pm in the sub
get_next_free_number_with_hash(), the lookup from savedrev83mapping needs to use
the extension too in the key, as the whole 8.3 name is used when populating
savedrev83mapping.

If this is not fixed, the generated File table might have 8.3 names that overlap
ones from the previous File table if new files have been added (for instance
because of added languages).

(As such, clashing 8.3 names is not detected by Windows Installer. It gladly
accepts installers where multiple files in the same folder are given the same
8.3 name. If we could just ignore the very small chance that somebody actually
installs OO.o on a 8.3-only file system, we could scrap all the code generating
8.3 names and just store a fixed name like "A.B" as the 8.3 name for all non-8.3
names... If I understood correctly, that is what Red Hat's installer for Mugshot
on Windows, for instance, does. They don't bother to generate any real unique
8.3 names. But I digress.)

The fix is to add a third parameter to get_next_free_number_with_hash():

sub get_next_free_number_with_hash
{
	my ($name, $shortnamesref, $ext) = @_;

and a few lines below then use that too in lookups from savedrev83mapping:

	if ( exists($shortnamesref->{$newname}) ||
	     exists($installer::globals::savedrev83mapping{$newname.$ext}) )
	{
		$alreadyexists = 1;
	}

Then we need to pass the extension (already uppercased and truncated) in the
calls to get_next_free_number_with_hash(), in five places in the sub
make_eight_three_conform_with_hash(). First, three instances of:

($number, $saved) = get_next_free_number_with_hash($name, $shortnamesref,
'.'.uc($extension));

then, in the remaining two calls just pass an empty string:

( $number, $saved ) = get_next_free_number_with_hash($name, $shortnamesref, '');

Hopefully this verbal free-form patch description is clear enough.

Comment 13 Olaf Felka 2007-06-08 10:24:57 UTC
OF @ IS: Please review.
Comment 14 ingo.schmidt-rosbiegal 2007-06-08 10:31:48 UTC
I will try to include this into cws native89
Comment 15 ingo.schmidt-rosbiegal 2007-06-27 16:59:45 UTC
Shifting from native89 to native93, but still OOo 2.3
Comment 16 ingo.schmidt-rosbiegal 2007-07-03 11:00:27 UTC
Fixed in cws native93
Comment 17 ingo.schmidt-rosbiegal 2007-07-03 14:53:58 UTC
Patch included into cws native93 -> verified.
Comment 18 ingo.schmidt-rosbiegal 2007-07-19 09:21:46 UTC
Fix included in m221, closing issue.