This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 182490 - Generate SIO Proxies command fails
Summary: Generate SIO Proxies command fails
Status: RESOLVED FIXED
Alias: None
Product: javacard
Classification: Unclassified
Component: Java Card (show other bugs)
Version: 6.x
Hardware: PC Windows Vista
: P3 normal (vote)
Assignee: _ tboudreau
URL:
Keywords:
: 182489 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-03-22 13:47 UTC by msmirnov
Modified: 2010-05-27 09:36 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description msmirnov 2010-03-22 13:47:53 UTC
If "Generate SIO Proxies" command is invoked on Classic Applet Project prior to
executing Build command then the command fails with the following message:
"
    [Error] Missing Java Card Runtime Descriptor "META-INF/MANIFEST.MF"
Packager/create FAILED [1 error(s) and 0 warning(s)]

"

though file META-INF/MANIFEST.MF exits in the base project directory (not in
build directory).

If the same command is invoked for Classic Library Project without invoking
Build command before then packager fails with other message:

"
Packager [v3.0.2]
    Copyright (c) 2009 Sun Microsystems, Inc.
    All rights reserved.
    Use is subject to license terms.

[ INFO: ] Validating Command Line
[ INFO: ] Setting packaging information
[ INFO: ] Package is being created
Usage:  converter  <options>  package_name  package_aid 
major_version.minor_version 
OR
converter -config <filename>
                    use file for all options and parameters to converter
Where options include: 
        -classdir <the root directory of the class hierarchy>
                      set the root directory where the Converter  
                      will look for classes
        -i            support the 32-bit integer type 
        -exportpath  <list of directories> 
                      list the root directories where the Converter  
                      will look for export files 
        -exportmap    use the token mapping from the pre-defined export  
                      file of the package being converted. The converter 
                      will look for the export file in the exportpath
        -applet <AID class_name>
                      set the applet AID and the class that defines the
                      install method for the applet 
        -d <the root directory for output> 
        -out  [CAP] [EXP] [JCA]
                      tell the Converter to output the CAP file, 
                      and/or the JCA file, and/or the export file
        -V, -version  print the Converter version string 
        -v, -verbose  enable verbose output
        -help         print out this message 
        -nowarn       instruct the Converter to not report warning messages
        -mask         indicate this package is for mask, so restrictions on  
                      native methods are relaxed
        -debug        enable generation of debugging information
        -nobanner     suppress all standard output messages
        -noverify     turn off verification. Verification is default
        -sign         sign the output CAP file
        -keystore <keystore>
                      keystore to use in signing 
        -storepass <storepass>
                      keystore password
        -alias <alias>
                      keystore alias to use in signing
        -passkey <passkey>
                      alias password
        -useproxyclass 
                      user supplied proxy classes (proxy classes not generated
by converter)
        -usecapcomponents
                      user supplied cap components
        -keepproxysource <directory>
                      keep generated proxy source files in specified directory

!!!!SIO-Proxy Generation!!!!
Source for Proxies is generated. To use these sources, Use My Proxies option in
Packaging settings of the project must be enabled.

BUILD SUCCESSFUL (total time: 1 second)

"

I.e. in the case of Classic Applets packager fails because neither descriptors
nor compiled classes are copied into build dir, and in the case of Classic
Library project descriptor is copied, but compiled classes aren't.
Comment 1 msmirnov 2010-03-22 13:49:05 UTC
*** Bug 182489 has been marked as a duplicate of this bug. ***
Comment 2 msmirnov 2010-03-24 15:53:35 UTC
I suggest to add dependencies to "generate-sio-proxies" target like the following:

"<target name="generate-sio-proxies" depends="-init,compile,create-descriptors">
...
"

instead of 

"<target name="generate-sio-proxies" depends="-init">
...
"
Comment 3 _ tboudreau 2010-03-24 18:37:00 UTC
Anki added a feature to generate SIO proxies as source files so the user can edit them.  The property maps to the project property "use.my.proxies"

So probably the generate proxies target should not run if they have been generated and may be modified, e.g. 
<target name="generate-sio-proxies" ... unless="use.my.proxies">
Comment 4 msmirnov 2010-03-25 09:07:58 UTC
Agreed. I think we need to warn user about existing proxy sources anyway and probably ask him to delete these sources if he does want to regenerate them? Regeneration in case of use.my.proxies=false could also be unwanted. Maybe we need to rename menu item "Generate SIO Proxies" to "(Re)Generate SIO Proxies" so its consequences be more clear?

Another question: what if a user edited some Proxies, then added more Shareable interfaces and wants to generate proxy sources for new interfaces only?
Comment 5 _ tboudreau 2010-03-25 09:30:32 UTC
That's exactly why I wanted to avoid implementing this for 6.9 :-)  Generated sources - especially ones generated outside the IDE need careful handling; it would be better for the IDE to have complete control over the code generation process.  To do this right, you would:
 - For each file, on first generation action:
   - Store a hash of the generated file
   - Store a hash of the existing file (or more likely, an MD5 hash of its public signatures only)
   - Or, more ideally:
       - For generated  hashes on a per-method basis
   - Store the hashes in a file in nbproject/ (NB has some built-in support for this)
   - In generated code, use guarded (blue) blocks for the method signature lines, so they are non-editable.
 - On regeneration:
   - Compute the new generated file w/o saving and compare its hash to the stored one (either per-file or per-method)
   - If per-method, (re) generate only those methods which have changed
   - For conflicts where there is user-edited code, open the diff window and allow the user a chance to merge their changes into the newly generated files

All of this is fairly non-trivial to do.

The simple way is to not allow regeneration, only generation - user must manually delete the proxy source dir (perhaps unchecking Use My Proxies can offer to delete them).

What we don't want is to either end up with invalid proxy code or uncompilable projects.

The root problem (not that we can change it), is that the entire idea of generating source code for this is the wrong approach.  It should be a reasonably simple problem to solve, to analyze the closure of all public methods in a project and use bytecode patching to add synchronization where needed.  In fact, by analyzing the disjuction of the closure of all methods, a code generator could probably do a better job than many programmers - by using more locks carefully - if two code paths cannot ever invoke each other, a shared lock is not needed and would create a thread-liveness issue.  All of this could be done by bytecode post-processing with no sources involved, and the correctness of the result would be provable.
Comment 6 msmirnov 2010-03-25 12:41:26 UTC
What about more simple solution? It is definitely better than what we have now:

1. Add property proxy.dest.dir for classic applets and libraries. This property indicates the location of generated proxy sources outside of src.dir folder.

2. "Generate SIO Proxies" target generates proxy sources in ${proxy.dest.dir} and copies these sources to ${src.dir} without overwriting existing files.

3. The following message is displayed upon successful proxies generation:

"!!!!SIO-Proxy Generation!!!!

    Source for Proxies are generated. To use these sources,
    "Use Proxy Sources" option in Packaging settings of the project must be enabled.
    Note: existing proxy sources at '${src.dir}' weren't replaced. Only new ones were added.
    To replace any of existing proxy sources with generated ones you need to delete corresponding source files from '${src.dir}'. 
    You can see *all* generated sources at '${proxy.dest.dir}'"

Thus if we have proxy sources, ${proxy.dest.dir} will contain all sources generated by packager and ${src.dir} will contain all user-edited sources and new generated ones. 

You're right that for full automatic solution we need to compare sources on a per-method basis. But with the solution above user at least can compare his own sources in ${src.dir} with generated ones in ${proxy.dest.dir}.
Comment 7 _ tboudreau 2010-03-25 22:03:46 UTC
> 1. Add property proxy.dest.dir for classic applets and libraries. This property
> indicates the location of generated proxy sources outside of src.dir folder.

OK.  Suggest we default it to something like "$PROJECT/proxies".  Clicking Use My Proxies should create the dir if needed (and perhaps perform generation in that case?) and add it to project.xml as a source root, so they appear in the UI.

For consistency (and compatibility with some general purpose Java project classes we're calling, and which will generate some of the project.xml), the property name should be "src.proxies.dir".

If not set, we can just do something like
<target name="configure-proxy-dest" unless="use.my.proxies">
   <property name="src.proxies.dir" value="build/proxies"/>
</target>
and generate proxies on-the-fly into the build dir (which will be deleted on clean).

If we want to (and can) generate new proxies for any non-existent classes, into src.proxies.dir, wherever it is, that is fine.

There is still the confusing possibility that a proxy source will still exist for a class which has been deleted.  That could happen for either destination.  Not sure what we do about this.

> Source for Proxies are generated. To use these sources,
> "Use Proxy Sources" option in Packaging settings of the project must be enabled.

I don't quite follow this.  Probably the user should see nothing about proxies (i.e. it's just a transparent implementation detail) unless they really check the Use My Proxies checkbox.

>  Note: existing proxy sources at '${src.dir}' weren't replaced. Only new ones were added.

src.dir should be src.proxies.dir

Or do you really want to generate proxies directly mixed in with the user's source files?  I think if we ever want to improve the solution, they should be separate.


BTW, what is the naming convention for proxy sources?

What happens if the user has 
$PROJECT/src/pkg/A.java
$PROJECT/src/pkg/AProxy.java
and a generated
$PROJECT/proxies/pkg/AProxy.java
?

I guess we can choose which order to copy class files in to let things under src/ win?

What happens if the user refactors the name of either A or A's proxy?

I suppose mixing the proxies with the source files would solve that problem.  I think, since most users will never need to edit them, and we don't really want users to edit them, it would be safer to keep them somewhere else.  Especially if we want to do any sort of hashing scheme later

Can we ever guarantee the user will not have their own proxy sources mixed in with their sources?

Is the packager intelligent enough to not generate proxies for things it can find proxies for?

(all of these issues are reasons why source-generation is almost never the right solution to a problem like this)
Comment 8 _ tboudreau 2010-03-25 22:24:19 UTC
Okay, I think I misunderstood a bit:  So, packager *does not* generate proxy sources by default - we only do that if the checkbox is checked?

I do think we should generate them into a separate source root (we can just have src.proxies.dir=proxies set by default in the project.properties from template, perhaps).  That way, if the user *unchecks* the checkbox, we can simply remove that root from the source path.  The files are not deleted, but they will not be used anymore and not visible in the UI.  

Otherwise, if they were under src/, we would need to programmatically figure out which sources are proxies and delete those (ideally only if not modified).  This way, we can generate proxies, but we do not have to delete anything (which the user may have modified) - we can just leave them on disk - they won't be used anymore - the user can delete them if they want.

If the user puts their own proxy classes under src/ and also has checked use.my.proxies, I guess we can just say "don't do that - which proxy class will be used is undefined" for now (or will packager try to generate proxies for the proxies?).

So, generally, go for it, but do lets use a separate source root for the proxy sources.
Comment 9 msmirnov 2010-05-27 09:36:13 UTC
Fixed in http://hg.netbeans.org/main/rev/0675e444698d