Surely it would be nice.
Not as easy as it looks like. The editor uses IDE for parsing java sources.
Making parser database form classes only is possible, but looses
one nice feature - the formal names of metod parameters, making it
from sources is much better, especially when the sources are easily available.

Only changed version to Development.

Target milestone -> 3.3

Changing the state of the issue to ASSIGNED, because it was covered in 
Requirements document at http://editor.netbeans.org/doc/Requirements.html

Target milestone -> 3.3.1.

Set target milestone to TBD

Set target milestone to TBD

I implemented an Ant task to wrap this tool for issue #26453. Note
that it does not retain information about method parameter names, nor
does it know about deprecations. For these reasons, it is pretty much
a toy as of now. Example usage from nb_all/foo/build.xml:

<target name="parser-db">
    <taskdef name="makeparserdb"
classname="org.netbeans.nbbuild.MakeParserDB"
classpath="../nbbuild/nbantext.jar"/>
    <mkdir dir="netbeans/system/ParserDB"/>
    <makeparserdb db="netbeans/system/ParserDB/Foo" sources="src"
nbroot=".."/>
</target>


I also have a request for the JCStorage.main method, which should be
pretty simple to implement: rather than taking a directory and
scanning for **/*.java itself, it should permit you to pass in a list
of Java source files. (If necessary, a base dir and then a list of
class names.) Ideally would let you have more than one classpath root,
too.

This would allow the Ant task to be written more naturally and
flexibly, e.g.:

<makeparserdb db="netbeans/system/ParserDB/Foo" nbroot="../..">
    <sources dir="libsrc"/>
    <sources dir="src">
        <exclude name="org/netbeans/modules/foo/experimental/"/>
    </sources>
</makeparserdb>

To just provide an explicit list of classes, it looks like it would
suffice to just change JCStorage.main to have an option to not call
JCUtilities.getClassNameList, but just accept an explicit list. I'm
not sure how much harder it would be to permit multiple classpath
roots in one DB.

I would suggest raising the priority here. See nbdev "Exposing module
APIs". Many API modules (Looks, Classpath, etc.) keep on appearing,
and they should all ideally have development-time counterparts (Docs
for Looks, Docs for Classpath) incl. Javadoc, code completion,
templates, etc. But maintaining code completion manually for a small
API is a fair amount of effort, and likely most of the databases will
always be a bit behind the code, making them less useful than just
mounting sources.

This can't be so hard: if you parse sources using SourceCookie, it
should be possible to start the IDE in non-GUI mode with a limited
classpath, e.g.:

openide.jar
core.jar
editor*.jar
java*.jar
classfile.jar
clazz.jar

Note that any modules found on the classpath will be enabled
automatically, so there is no need to use the official IDE launcher.
Unit tests do this kind of thing all the time; it is pretty fast to
start up with no GUI and only a handful of modules - five seconds at
the most, reasonable for inclusion in a build script.

This set of modules is enough to create & mount LocalFileSystem's;
create JavaDataObject's; ask them for SourceCookie; etc.

Ok Jesse, I will try to implement this for the current behaviour of
parsing. Anyway, it should be changed somehow in the future, because
of the projects.

Is there any documentation how to run NetBeans in non gui mode?
I tried the following:
 System.getProperties().put (
     "org.openide.TopManager", // NOI18N
     "org.netbeans.core.NonGui" // NOI18N
 );

 System.getProperties().put (
    "netbeans.home", // NOI18N
    "C:\\Mato\\testHome" // NOI18N
 );

TopManager tm = TopManager.getDefault();

It started to run IDE in nongui, but I got exception:

javax.xml.parsers.FactoryConfigurationError: Provider for
javax.xml.parsers.SAXParserFactory cannot be found
        at javax.xml.parsers.SAXParserFactory.newInstance(Unknown Source)
        at org.openide.xml.XMLUtil.createXMLReader(XMLUtil.java:147)
        at org.openide.xml.XMLUtil.createXMLReader(XMLUtil.java:120)
        at org.netbeans.core.modules.ModuleList$1.run(ModuleList.java:264)
        at
org.openide.filesystems.EventControl.runAtomicAction(EventControl.java:89)
        at
org.openide.filesystems.FileSystem.runAtomicAction(FileSystem.java:416)
        at
org.netbeans.core.modules.ModuleList.readInitial(ModuleList.java:257)
        at
org.netbeans.core.modules.ModuleSystem.readList(ModuleSystem.java:232)
        at org.netbeans.core.NonGui.run(NonGui.java:545)
        at
org.openide.TopManager.initializeTopManager(TopManager.java:122)
        at org.openide.TopManager.getDefault(TopManager.java:83)
        at RunIde.main(RunIde.java:47)

Then how can I specify classpath for modules, is it intended to get it
as a list from users of JCStorage.main?

Which unit test use nongui? I have tried to look at junit sources but
I found no usefull info there.
Thanks, 
Mato

Starting in non-GUI mode should be easy. You don't have to set
TopManager sys prop; defaults to org.netbeans.core.Plain (which is
more appropriate than NonGuiMain for this anyway). Just make sure
core.jar, openide.jar, and whatever module JARs you want are on the
classpath - it should load those modules automatically. You may need
to define "system.dir" to be a temporary directory (similar to
$userdir/system/) for some things to work. Then just run some
main(String[]) class from the classpath and use
TopManager.getDefault() to initialize everything - it is supposed to work.

For examples, see most any test in
core/test/unit/src/org/netbeans/core/ or subdirs. It is a little
complicated to see what is happening, because XTest sets up the classpath.

Jesse,
I have implemented the initial version of the feature. Please, try, if
it will run as you expected. Also, please replace Class.forName in
org.netbeans.nbbuild.MakeParserDB to
org.netbeans.modules.editor.java.StandAloneJCDBCreator.
Thanks.

http://editor.netbeans.org/source/browse/editor/src/org/netbeans/modules/editor/java/JCUpdater.java.diff?r1=1.47&r2=1.48

http://editor.netbeans.org/source/browse/editor/src/org/netbeans/modules/editor/java/StandaloneJCDBCreator.java?rev=1.1&content-type=text/x-cvsweb-markup

Thanks! I made some minor changes to the impl in the editor to clean
it up a bit, and updated the Ant task to work with it. It seems fine -
correctly picks up method param names, @deprecated, etc.

committed   * Up-To-Date  1.2        
editor/src/org/netbeans/modules/editor/java/StandaloneJCDBCreator.java
committed   * Up-To-Date  1.2        
nbbuild/antsrc/org/netbeans/nbbuild/MakeParserDB.java

Resolved for 3.4.x or earlier, no new info since then -> closing.

Please provide documentation or an example on how the new ant task is
used.

Please, look at apisupport/apidocs/build.xml 
<target name="parser-db"> how to use this ant task. 

Thanks, but that is not enough =).  I don't know what sources and
nbroot are and there's no javadoc even if I look at the code for the
ant extension itself.  Suppose I have:
<myproject>/<mysubproject>/build.xml
<myproject>/<mysubproject>/src/com/mypackage/api
<myproject>/<mysubproject>/src/com/mypackage/spi

I only want to generate for
<myproject>/<mysubproject>/src/com/mypackage/api.  What are the values
for sources and nbroot?  How can I inspect the old (manually created)
files and ensure that the content is the same?

Re. nbroot - must be root of a built NB source tree. (Packaging all
the needed classes with the task would make it consume several megs of
disk space. Using a NB distribution tree would prevent it from being
used during NB's own build process.)

Re. sources - must be source root to gen docs for (.../src). No
support for choosing packages currently. See my comments of 2002-08-15.

Re. inspecting old files: good luck. You need to try it. The format is
binary and hard to read.

Is nbroot optional?  Which part of the tree needs to be there and
built?  We have a copy of nbantext.jar (built), but no access to the
whole distribution.

As far as choosing packages, I think that's important in order to
complete this.  I don't think providing completion for non-api
packages is a good idea.  If your source code is structured in a way
that the whole dir is what you want, then you're lucky =).

As far as inspecting old format - is there something in bean browser
that would help?  So far, I just tried using the code completion
enough to find that api was there and not spi, but I'm not sure about
public and other options.  In any case, javadoc should be added for
this task.

Re. nbroot - it is mandatory. You need the whole tree built, at least
several modules. See
nbbuild/antsrc/org/netbeans/nbbuild/MakeParserDB.java for the complete
list. If the java module were substantially refactored to not depend
on so many things at once, it would be feasible to reduce the required
JAR list to the point where you could package up all needed classes
with the task into one JAR, but currently that seems unrealistic.

Re. choosing packages - ideal IMHO would be for the tool class to
accept a list of .java files within each source root, which would
permit you to write e.g.

<makeparserdb nbroot="...">
  <database prefix=".../system/ParserDB/foo">
    <fileset dir="/my/sources1/main">
      <include name="**/*.java"/>
    </fileset>
    <fileset dir="/my/sources1/aux">
      <include name="pkg1/*.java"/>
    </fileset>
  </database>
  <database prefix=".../system/ParserDB/bar">
    <fileset dir="/my/sources2/main">
      <include name="pkg1/**/*.java"/>
      <include name="pkg2/**/*.java"/>
    </fileset>
  </database>
</makeparserdb>

Re. inspecting old format - there is no better tool than (1) hex
editor, (2) trying completion and seeing what it offers. It might be
nice to have a separate tool (with an Ant wrapper?) that takes a *.jcb
file and dumps a sorted list of classes and members included in the DB
(as Java signatures).

Patches welcome, of course. :-) Neither the tool class in the editor
nor the Ant wrapper task are particularly complicated code.

For nbroot, it would be nice if as a first step, pointing at an
installed nb would be enough.  For build purposes, we already have
that.  However, the list in MakeParserDB is not of that form, i.e.
java/api/netbeans/modules/autoload/java-api.jar and not
<root>/modules/autoload/java-api.jar.

For packages, I agree that filesets would be the most natural type of
specification.

Unfortunately, I don't have time to submit patches now =).  But the
bottom line is, I think this bug is not yet resolved/fixed =).

It could be an *option* to point at a NB installation. This is
unusable during the build of NB (contrary to what you say) because the
merged installation is created as the last step.

Right - I know that for a build of nb it makes more sense the way it
is now.  I did not mean to imply otherwise.  But for other users of
the task, an option for pointing at an installation would be a short
term solution (until the jar list could be reduced and the relevant
classes packaged).

We do need filesets more and more - lots of new APIs (e.g. Looks,
Convertor) have both public and private packages, and it is not nice
for the DBs generated for apisupport/apidocs to include the private
packages. Could mislead novice API users.

Changing subcomponent to "code completion"

I guess the functionality is gone as part of the MDR scanning. Will
remove the Ant task.