Bug 206126

Created attachment 113947 [details]
Order of categories second case

See d3256c05e288 and 5a0d1f358ec9. I do not understand why you are prompted to add servlets to SE projects but this seems to be intentional. Reproducible in 7.0.1 and probably long before.

The J2SE Library can be a library of Web project, this i the reason why it has more items.

One problem with that is that when it causes the Web category (and some others) to appear, they come before the Java category, which looks unnatural; but I suppose this is the desired order for an actual web project.

Unfortunately the infrastructure decides on the folder order based on a global static ordering of template folders, and then merely shows or hides a given folder depending on whether RecommendedTemplates lists entries in it or not. Using the order of items in RecommendedTemplates (it is an array, not a set, and the LookupMerger respects order) might be better, but it is not so simple since the items in RT are categories which do not map 1-1 to folders. It would need to be a bit more subtle: go through RT categories one by one, displaying all folders (in layer order) containing at least one template in that category, skipping folders already displayed for an earlier category. Such a behavioral change would need to be reviewed and documented and it is likely that many RT impls and template folder positions would need to adjusted accordingly.

Leaving aside the order, it is a UI question whether it is desirable for a Web template folder to appear at all for a library used only in an SE app, and if so whether it makes sense to hide this folder once a main class is added to the project.

Yeah, some of the Web file templates are applicable for J2SE class library project. But ordering of New File templates for J2SE projects should stay the same and Web/Struts/Spring templates should be displayed lower in the list in case of J2SE Class Library project type.

I'm not sure about the best way to fix this. "Web" templates do not care about their placement in J2SE Class library project but they do want to be shown first in case of Web/EE project types. Perhaps the ordering mechanism of templates should allow to be project type specific? Or perhaps "Web" template should be able to express that they want to be shown at the top for Web/J2EE project types as these are primary project types for these templates and for everything else the order is less important? Or even simpler: perhaps project type should have the power to split the list of shown New File Templates into two: primary and pluggable ones. Primary are "hardcoded" in project type and if a 3rd party module want to get there they must negotiation with the project type author. Otherwise they fit into pluggable category which follows current ordering contract.

Re. "main class presence" - I do not think that would work well. Alternative would be a presence of Servlet API on project classpath but again I do not think it would work well either.

(In reply to comment #5)
> ordering of New File templates for J2SE projects should stay the
> same and Web/Struts/Spring templates should be displayed lower in the list in
> case of J2SE Class Library project type.

Then there is a problem, because this behavior is not supported by the current infrastructure.

> Perhaps the ordering mechanism of
> templates should allow to be project type specific?

That is essentially what I proposed in comment #4.

> Re. "main class presence" - I do not think that would work well.

Perhaps you misunderstood - this is the _current_ behavior. If there is a main class, you get just basic Java folders shown; if there is not, you get web-related folders (currently at the top).

(In reply to comment #4)
> Leaving aside the order, it is a UI question whether it is desirable for a Web
> template folder to appear at all for a library used only in an SE app, and if
> so whether it makes sense to hide this folder once a main class is added to the
> project.

Yes, I did misunderstood this. It might be the easiest to just show Web templates always in J2SE project regardless whether it has main class or not. If Web templates are not shown at the top of the list there is no problem, ay? There seems to be bunch of other New File categories for J2SE project which make sense only in certain scenario, eg. Web Services, Persistence, so Web is not different from these.

(In reply to comment #7)
> If Web templates are not shown at the top of the list there is no problem, ay?

The problem remains, because the New File infrastructure uses a fixed global order for template folders. If we do want to continue to show the Web folder in j2seproject's - whether in some such projects or all does not matter - then there needs to be an API change.

The web project templates should not be shown in J2SE App project.
It's better to extend the current infrastructure to be able to define per project ordering, for example the RecommendedTemplates can be extended to RecommendedTemplates2 where the order will be given).

Then we need an API change of sorts.

(In reply to comment #9)
> the RecommendedTemplates can be extended to
> RecommendedTemplates2 where the order will be given

RecommendedTemplates already implicitly specifies an order, since it returns String[] rather than Set<String>; just currently this order is ignored, whereas it could be used.

I have tried to make it work for RecommendedTemplates's order to be specific, but it does not work well for certain cases. For example: maven.persistence specifies the 'persistence' category for all o-n-m-maven, while maven.apisupport specifies 'nbm-specific' (and others) for o-n-m-maven/nbm. Since Maven's lookup places o-n-m-maven services before o-n-m-maven/$packaging services, the Persistence folder gets shown before NetBeans Module Development, and there is no way to fix this.

An alternate approach is to deprecate RecommendedTemplates and introduce a new SPI that returns (say) a Map<? extends Number,String> of ordered categories. That would be similar to what was requested in bug #130851 for PrivilegedTemplates. Such an SPI would be less natural if you are used to using @ProjectServiceProvider ordering, but may be easier to work with.

Created attachment 116493 [details]
Patch based on String[] RT.gRT() being considered ordered

Created attachment 116494 [details]
Generated template summary (cluster.config=enterprise)

Created attachment 116495 [details]
Reworked patch using RecommendedTemplates2

Please review. The order of template folders resulting from the patch is certainly not the desired final list, but it at least shows how some example cases can be handled relatively easily. To keep the diff as small as possible, RT impls providing a long list of categories were simply changed to RT2 impls locating the first at 100, the second at 200, etc.; better impls would explicitly specify positions for particular categories, and insert additional categories at particular spots according to dynamic conditions.

To fully polish templates, a user interface expect would need to go through each project type (e.g. Ant-based Java SE projects) and variant (e.g. application project with main class), check the list of folders in the full IDE distribution, and request changes.

Please note that the order of templates within a folder is _not_ changed by this patch - they are listed in layer order. But template folders are displayed primarily by category order, falling back to layer order only in cases where more than one folder supply at least one template in a category which is the first to use any of those folders.

Also consider a completely different approach to the problem: leave the API alone; remove the servlet category from library j2seproject's (and other similar cases); but introduce a GUI (such as a checkbox) to the New File dialog allowing the user to disable all category filtering on templates - a frequently requested change.

Though this may sound at first reading to be simple, it would be a major behavioral change that could be incompatible - some custom wizard iterators are surely written under the assumption that their Project target satisfies some conditions.

For example, NetBeans Module Development > Action needs an NbModuleProvider in the project lookup so it can add module dependencies. If written properly, it would check for a null value and display an appropriate error message in the first wizard panel ("Not a NetBeans module project!"), disabling the Next and Finish buttons. But since this scenario could never before have occurred, the author might have just assumed that the result is not null, meaning a confused user trying to run this wizard on a j2seproject would get a ClassCastException unless the wizard were fixed.

Therefore such a change would need to be documented as incompatible, and every custom New File wizard would need to be tested against some kind of dummy project providing no services to make sure it behaves gracefully.

Withdrawing for consideration in 7.2 since either the current patch, or the alternative given in comment #16, would need UI input.

Created attachment 120926 [details]
Same as previous patch but with one Javadoc change in Project.java

what about a fairly low level simple API that does something along these lines:

1. new interface FileCategorySorter  that would be implemented by project types and reside in project's lookup. Non mandatory, if not present, layer files ordering applies.
2. public List<DataObject> sort(List<DataObject>); method defined in the interface. Implementations would take the category dataobjects, modify it based on some arbitrary rules and return the optimal ordering.
3. the sorting code would be used in TemplateChooserPanelGUI.TemplateChildren.createKeys() only for root categories. 
4. Theoretically we can allow filtering as well (remove a DataObject from the list). Not sure if that's of any use, filtering is to be done by the RecommendedTemplates API.. personally I would prohibit it.

the algorithms for sorting are undefined, can be just forcing "java" to be first, or some sort of attribute based marking can be used.. out of scope for the API proposal.

could be a projectui module friend API or public API in project.uiapi module.

Comments?

Created attachment 128833 [details]
suggested api change for review

patch including the api change (not maven/web implementation). introduced an spi package and interface in projectui alone. All implementers will need to become friends. That's ok from my perspective, the API is not really mainstream and commonly to be used.

please review

Just one note:

MJ01: In the current version it's not as obvious that the API client shouldn't use the method for filtering. I know there is an assert to check this but it would be better to either specify this in JavaDoc or use a different approach (e.g. method returning "Comparator<DataObject>" instead of current "List<DataObject> sort(List<DataObject>" ..functionality would stays the same and it would be easier to implement it correctly).

(In reply to comment #22)
> Just one note:
> 
> MJ01: In the current version it's not as obvious that the API client shouldn't
> use the method for filtering. I know there is an assert to check this but it
> would be better to either specify this in JavaDoc 

I will mark this in the javadoc.

> or use a different approach
> (e.g. method returning "Comparator<DataObject>" instead of current
> "List<DataObject> sort(List<DataObject>" ..functionality would stays the same
> and it would be easier to implement it correctly).

with comparator it would be non obvious how to achieve the currently only known usecase. Keep everything as is, but move Java to the top.

TZ01: Minor comment. Rather than:
assert sorted.size() == dobjs.size();
the content should be checked:
assert sorted.size() == dobjs.size() && new HashSet<DO>(dobjs).equals(new HashSet<DO>(sorted))

thanks for the review

http://hg.netbeans.org/core-main/rev/11b95c938b12

with comments incorporated. reassigning now to j2ee/maven to implement the new spi.

Integrated into 'main-golden', will be available in build *201212140001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/11b95c938b12
User: Milos Kleint <mkleint@netbeans.org>
Log: #206126 spi to allow low level sorting of template categories

Reassigning to J2SE project. I'm sorry, I don't understand what JavaEE Maven project should do within the SPI implementation. I didn't read the whole discussion from the beginning but the issue seems to be more about J2SE Ant project and the order of it's categories. Or am I missing something?

Fixed jet-main d424f6d151b6

Integrated into 'main-silver', will be available in build *201307232300* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/d424f6d151b6
User: Tomas Zezula <tzezula@netbeans.org>
Log: #206126:Web-related templates shown at top for Java SE library projects