Bug 113341

I guess the root problem is:
 
       Text API - The module Editor MIME Lookup was requested in version >= 1.5 but only 1.3.31 was found.
 

The problem seems to be that we moved mimelookup and options.api to platform cluster, while in 5.5.x these modules 
used to be in ide7 cluster. As a result the platform 6.0 modules require new versions of these modules, but the 
modules system does not see them - they are hidden by config/Modules/*xml files provided by 5.5's ide7 cluster.

I am not sure how to fix this. The fact that cluster can hide definitions in lower clusters like platform7 is 
essencial for many applications including BlueJ. So that cannot be changed. Maybe this is wontfix, if we provide some 
guidelines for QA to run 5.5 clusters on top of platform7?

The workaround to this problem is to delete mimelookup module from the ide7 cluster:

rm ide7/config/Modules/org-netbeans-modules-editor-mimelookup.xml 
ide7/modules/org-netbeans-modules-editor-mimelookup.jar 
ide7/update_tracking/org-netbeans-modules-editor-mimelookup.xml 

So if I understand the issue correctly:

platform55 - text(1.0)
platform60 - mimelookup(1.1), text(1.1) > mimelookup(1.1)
ide55      - mimelookup(1.0), whatever > text(1.0)
ide60      - whatever > text(1.0)

where when loading ide55 + platform60, mimelookup(1.0) is loaded (just because ide cluster is preferred to platform
cluster?), so text(1.1) fails. Did I get this right?

The only fix I can think of would be for the module system to always preferentially load the newest version of modules
it finds duplicates of, rather than the first version it encounters. I am not sure what problems such a behavior change
could introduce.

Unimportant correction:
ide60      - whatever > text(1.1)
otherwise this is the case.

As far as the fix goes, yes, I was hoping the module system will always prefer the newer version. However this is 
easier to say that to do. Problems:
#1 InstalledFileLocator returns just the first JAR (from ideX) not the second (from platformX), it would need to be 
extended to return enumeration
#2 List of modules is read from SystemFileSystem and here the ideX's Modules/bla-bla-mimelookup.xml overrides 
Modules/bla-bla-mimelookup.xml from platformX. As a result the system may not even know that there is the platform's 
version.

I guess the issue is so complex, that I can reassign to owner of core/modules.

*** Issue 113351 has been marked as a duplicate of this issue. ***

Downgrading to P3. I am not aware of any user-visible bug associated with this issue, i.e. it is an architectural issue;
no one that I know of is actually trying to use a new platform cluster with an old ide cluster, and if they are, there
is a known workaround. Given the risks of making a significant change like this, I do not propose to touch anything in 6.0.

Agreed that fixing InstalledFileLocator to return the file from the correct cluster is necessary. I had not thought of
this aspect of the problem. (It is not used by the module class loader, but if a module explicitly calls IFL for
associated files that could matter.)

Regarding Modules/*.xml, this is indeed a bit subtle:

1. The ide cluster can have a _hidden mask which prevents the module from appearing at all.

2. The ide cluster could have a copy of the XML file which makes a regular module be disabled.

3. Both clusters could specify the module as enabled, but the (overridden) platform version could show the newer spec
(or release) version.

I think the answer is to continue to read Modules/*.xml from the system filesystem, with normal mask & override
semantics, which would preserve #1 and #2. Regarding #3, I am actually not sure that the version info in Modules/*.xml
is used for anything (it duplicates information in the JAR); it would be odd but perhaps OK for the module system to
encounter the following situation:

platform/modules/x.jar: OIDE-M-S-V: 1.1
platform/config/Modules/x.xml: <param name="specversion">1.1</>
ide/modules/x.jar: OIDE-M-S-V: 1.0
ide/config/Modules/x.xml: <param name="specversion">1.0</>
SFS:config/Modules/x.xml: <param name="specversion">1.0</>

If the module system were changed to load platform/modules/x.jar preferentially (because it sees the newer 1.1 version
in it), then SFS:config/Modules/x.xml would show the wrong version. I think this would cause there to be written out:

userdir/config/Modules/x.xml: <param name="specversion">1.1</>

resulting in:

SFS:config/Modules/x.xml: <param name="specversion">1.1</>

which is not ideal but may be OK.

*** Issue 121252 has been marked as a duplicate of this issue. ***

I am unaware of any problem outside of our artificial compatibility tests.

Later.

NetBeans.org Migration: changing resolution from LATER to WONTFIX

Looks fixable.

Created attachment 92834 [details]
Start of patch

Created attachment 92876 [details]
Near-final patch (apichanges still missing)

Please review this patch (also for bug #36701). Patch header describes changes in detail, but briefly:

- adding IFL.locateAll
- looking up newest version of a module in any cluster
- deleting old compatibility code for ModuleInstall "upgrades" and serialization

I will plan to commit in January.

core-main #1fcfef0f3d39

Integrated into 'main-golden', will be available in build *201001070200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ee38a52b0f40
User: Jesse Glick <jglick@netbeans.org>
Log: Test failure from #113341 patch.

Created attachment 102097 [details]
System property to go back to "First module jar wins" strategy

Today I try to update our rcp based applications and build environment to platform 6.9.1. In our development process we use the ability of the suite project to overlap platform modules. With this change it did not work any more. The "Pick the newest one available" strategy won't fit in our continuous integration environment in all cases.

Explanation:
Our build environment is similar to the netbeans maven project (binary based with module repositories). The modules are build using hudson and continuous integration.
Our developers modify and build existing modules locally and use the suite project as a "patch" suite against installations of our rcp based applications to test there changes. Do get this work our suite project has a small extension, which installs a list of modules (patch nbms) into the suite cluster.

A system property to switch back to "First module jar wins" strategy would be fine.

Integrated into 'main-golden', will be available in build *201009220000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/77fd5b59bb90
User: Jesse Glick <jglick@netbeans.org>
Log: Bug #113341 comment #19: Conditionally disable "newest JAR wins" semantics from specialized development environments.

fvogler, best to be on CC for issues you comment on, so in this case you would see comment #20.