This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 171610 - Set execute permissions in ZIP distribution according to nbm.executable.files
Summary: Set execute permissions in ZIP distribution according to nbm.executable.files
Status: RESOLVED FIXED
Alias: None
Product: www
Classification: Unclassified
Component: Builds & Repositories (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords:
: 179665 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-09-07 11:28 UTC by soldatov
Modified: 2010-02-03 21:55 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description soldatov 2009-09-07 11:28:41 UTC
Scenario:
- Download
http://bertram.netbeans.org/hudson/job/cnd-main/lastSuccessfulBuild/artifact/nbbuild/NetBeans-dev-cnd-main-1573-on-090907-full.zip

- open gnome-terminal
- switch in 'root' mode
- unzip file into /usr/local directory
- close terminal
- start NetBeans (in user mode)
- create Welcome project
- load project in debugger
==>
SEVERE: Cannot run program "/usr/local/netbeans-6.8/cnd2/modules/ext/lib/Linux-amd64/tools_exec": java.io.IOException:
error=13, Permission denied
java.io.IOException: java.io.IOException: error=13, Permission denied
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
        at java.lang.ProcessImpl.start(ProcessImpl.java:65)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
Caused: java.io.IOException: Cannot run program "/usr/local/netbeans-6.8/cnd2/modules/ext/lib/Linux-amd64/tools_exec":
java.io.IOException: error=13, Permission denied
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
        at java.lang.Runtime.exec(Runtime.java:593)
        at com.sun.tools.swdev.toolscommon.base.UnixProcessFactory.spawn(UnixProcessFactory.java:181)
[catch] at com.sun.tools.debugger.dbxgui.utils.ExecutorUnix.startEngine(ExecutorUnix.java:368)
        at com.sun.tools.debugger.dbxgui.debugger.dbx.CommonDbx$Factory.start2(CommonDbx.java:783)
        at com.sun.tools.debugger.dbxgui.debugger.dbx.CommonDbx$Factory.start(CommonDbx.java:770)
        at com.sun.tools.debugger.dbxgui.debugger.dbx.DbxDebuggerImpl.start2(DbxDebuggerImpl.java:496)
        at com.sun.tools.debugger.dbxgui.debugger.dbx.DbxDebuggerImpl.start(DbxDebuggerImpl.java:435)
        at com.sun.tools.debugger.dbxgui.debugger.actions.DbxStartActionProvider.doAction(DbxStartActionProvider.java:100)
        at com.sun.tools.debugger.dbxgui.debugger.actions.DbxStartActionProvider.postAction(DbxStartActionProvider.java:91)
        at org.netbeans.api.debugger.ActionsManager.postAction(ActionsManager.java:229)
        at org.netbeans.api.debugger.DebuggerManager.startDebugging(DebuggerManager.java:386)
        at com.sun.tools.debugger.dbxgui.debugger.DebuggerManager.debugNoAsk(DebuggerManager.java:948)
        at com.sun.tools.debugger.dbxgui.debugger.DebuggerManager.startDebugger(DebuggerManager.java:919)
        at com.sun.tools.debugger.dbxgui.debugger.DebuggerManager.debug(DebuggerManager.java:1158)
        at com.sun.tools.debugger.dbxgui.debugger.actions.DebugProjectAction.performAction(DebugProjectAction.java:232)
        at com.sun.tools.debugger.dbxgui.debugger.actions.DebugProjectAction.performAction(DebugProjectAction.java:246)
        at
com.sun.tools.debugger.dbxgui.debugger.actions.DebugProjectAndStepAction.performAction(DebugProjectAndStepAction.java:25)
        at com.sun.tools.debugger.dbxgui.DbxActionHandler$1.run(DbxActionHandler.java:64)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
        at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:117)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
/usr/local/netbeans/cnd2/modules/ext/lib
Comment 1 Vladimir Voskresensky 2009-09-07 17:35:15 UTC
executable flags are missed on build machine. 
when browse unzipped bits (doesn't matter who unzipped "root" or "user") => 
vv159170@volvo:/var/tmp/netbeans/cnd2/modules/ext/lib$ la -R .
.:
total 32
drwxr-xr-x 8 vv159170 uucp 4096 2009-09-07 17:42 .
drwxr-xr-x 3 vv159170 uucp 4096 2009-09-07 17:42 ..
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 Linux-amd64
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 Linux-i386
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-07 17:42 SunOS-amd64
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 SunOS-sparc
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 SunOS-sparcv9
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-07 17:42 SunOS-x86

./Linux-amd64:
total 292
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 262991 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  16806 2009-09-03 20:33 tools_exec

./Linux-i386:
total 216
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 192229 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  13606 2009-09-03 20:33 tools_exec

./SunOS-amd64:
total 320
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-07 17:42 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 292912 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  18632 2009-09-03 20:33 tools_exec

./SunOS-sparc:
total 228
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 201740 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  14284 2009-09-03 20:33 tools_exec

./SunOS-sparcv9:
total 320
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 293240 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  17328 2009-09-03 20:33 tools_exec

./SunOS-x86:
total 232
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-07 17:42 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-07 17:42 ..
-rw-r--r-- 1 vv159170 uucp 205604 2009-09-03 20:33 libbase.so
-rw-r--r-- 1 vv159170 uucp  12928 2009-09-03 20:33 tools_exec

---------------------
It exists when I build manually from command line:
---------------------
vv159170@volvo:~/devarea/trunk/nbbuild/netbeans/cnd2/modules/ext/lib$ la -R .
.:
total 32
drwxr-xr-x 8 vv159170 uucp 4096 2009-09-03 20:33 .
drwxr-xr-x 3 vv159170 uucp 4096 2009-09-07 20:07 ..
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 Linux-amd64
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 Linux-i386
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-07 20:07 SunOS-amd64
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 SunOS-sparc
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-03 20:33 SunOS-sparcv9
drwxr-xr-x 2 vv159170 uucp 4096 2009-09-07 20:07 SunOS-x86

./Linux-amd64:
total 292
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 262991 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  16806 2009-09-03 20:33 tools_exec

./Linux-i386:
total 216
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 192229 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  13606 2009-09-03 20:33 tools_exec

./SunOS-amd64:
total 320
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-07 20:07 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 292912 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  18632 2009-09-03 20:33 tools_exec

./SunOS-sparc:
total 228
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 201740 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  14284 2009-09-03 20:33 tools_exec

./SunOS-sparcv9:
total 320
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-03 20:33 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 293240 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  17328 2009-09-03 20:33 tools_exec

./SunOS-x86:
total 232
drwxr-xr-x 2 vv159170 uucp   4096 2009-09-07 20:07 .
drwxr-xr-x 8 vv159170 uucp   4096 2009-09-03 20:33 ..
-rwxr-xr-x 1 vv159170 uucp 205604 2009-09-03 20:33 libbase.so
-rwxr-xr-x 1 vv159170 uucp  12928 2009-09-03 20:33 tools_exec


We have some more binary files in cnd cluster. I.e. cnd2/bin. And "x" exists for all top level files.
Someone experienced with nb builds investigate, please.
Comment 2 Vladimir Voskresensky 2009-09-15 00:37:06 UTC
Please, evaluate. This issue has very serious impact on CND.
Please, make sure that all binaries inside cnd3/bin, cnd3/modules/ext/lib & it's subfolders have executable flag.
The problematic distributions are 
http://bits.netbeans.org/download/trunk/nightly/latest/zip/netbeans-trunk-nightly-*-cpp.zip 
http://bertram.netbeans.org/hudson/job/cnd-main/lastSuccessfulBuild/artifact/nbbuild/NetBeans-dev-cnd-main-*-full.zip
Comment 3 Jesse Glick 2009-09-15 03:16:20 UTC
Currently nbbuild/build.xml#zip-cluster-config manually enumerates executable files, and misses e.g. **/*.so. That could
be fixed up the simple way, and this ought to be done immediately to downgrade from P1 to P3, but probably it should be
reading nbm.executable.files somehow?
Comment 4 Vladimir Voskresensky 2009-09-15 09:12:58 UTC
please, do not forget the original problematic executable misses needed flag: tools_exec (no extensions)
Comment 5 dlipin 2009-09-16 11:49:51 UTC
What should be done (in brief):
1) The list of modules should be identified that belongs to the current cluster config.
    <resolve name="cluster.config.modules" value="clusters.config.${cluster.config}.list"/>
    <resolvelist name="cluster.config.modules" list="${all.cluster.modules}"/>

2) The list of project.properties for that modules is:
   <pathconvert property="nb.all.project.properties" pathsep=",">
       <dirset dir="../" includes="${all.modules}"/>
       <mapper type="regexp" from=".*/([^/\\]*)" to="\1/nbproject/project.properties"/>
   </pathconvert>

3) The list of all modules (full paths) with nbm.executable.files defined:
    <pathconvert property="executable.modules.project.properties" pathsep=",">
    <fileset dir=".." includes="${nb.all.project.properties}">
        <containsregexp expression="^nbm.executable.files=.*$"/>
    </fileset>
        <mapper type="regexp" from=".*/([^/\\]*)/nbproject/project.properties" to="\1"/>
    </pathconvert>
(e.g. for cnd cluster config that would be cnd.debugger.dbx,cnd.debugger.gdb,cnd,dlight.nativeexecution,dlight.tools)

4) The next step is something that I can`t process with. 
For this list of modules we need to 
a) read the nbm.executable.files property (which is defined relatively to the cluster) from <module>/nbproject/
project.properties 
b) identify the cluster name (cluster.name) for the module. 
c) define the file set for each module
<fileset dir="${netbeans.dest.dir}/${cluster.name}" includes="${nbm.executable.files}">
d) sum up all the file sets into one

Jesse, Robert, any idea how to do that? I could image that existing tasks (like setclusterpatternset, setcluster, 
resolvelist, resolve, etc) can also be re-used for this need but I have no knowledge of them.


BTW, the same issue can also happen with zipclusters/zip-one-cluster tasks.
Comment 6 dlipin 2009-09-17 09:22:07 UTC
workarounded
http://hg.netbeans.org/main/rev/3bc0a0f8410a
Comment 7 dlipin 2009-09-17 10:30:09 UTC
... followed by http://hg.netbeans.org/main/rev/251466c10fd1 after discussion with Vladimir.
Comment 8 Jesse Glick 2009-09-17 15:15:19 UTC
Richard do you want to try this? Not quite in harness but related anyway (and probably harness' ZIP targets should be
enhanced in a similar way).
Comment 9 rmichalsky 2009-09-17 15:57:33 UTC
Could do, no problem. I'm not sure if I'll get to it next week, I suppose issue #169040 will take some more days with
necessary testing. But as the urgent case is worked around now, it shouldn't hopefully be a problem.
Comment 10 Jesse Glick 2010-01-20 18:18:55 UTC
I can try to get to this in 6.9.

Marking P2 due to duplicate bug #179665.
Comment 11 Jesse Glick 2010-01-20 18:19:01 UTC
*** Bug 179665 has been marked as a duplicate of this bug. ***
Comment 12 Jesse Glick 2010-02-01 16:14:26 UTC
Should be implemented now in core-main #5307494caa17, using a custom Ant task to collect the right patternset. I have also tried to use nbm.executable.files in some modules in java & ruby cluster configs which previously were not using it, including most importantly o.n.bootstrap (there is no longer special treatment for platform/lib/nbexec).

The main build script no longer sets a+x on anything not requested by modules other than bin/netbeans, so if you were relying on some *.so files etc. being executable please ensure that you are defining nbm.executable.files correctly. I think cnd/dlight modules were already doing this for some files but there may be others missing.

I made no attempt to generalize this fix to the 'zip' target of external module suites. While NB module source projects define the property in project.properties, and NBMs also define it in Info/executables.list, a suite is built against a cluster path and the clusters do not contain this information (except in the form of actual executable bits if running on Unix). To fix properly would require a module build to write some new .executables file to the cluster (or add some information to update_tracking/*.xml), and I did not want to go this far until there is some demand for it.
Comment 13 Quality Engineering 2010-02-03 21:55:22 UTC
Integrated into 'main-golden', will be available in build *201002040200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/444b26907a75
User: Jesse Glick <jglick@netbeans.org>
Log: #171610: set a+x in cluster ZIPs according to nbm.executable.files.