This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
After upgrade I cannot list files in /download space using wildcards. $ ssh upload@cvs.netbeans.org "ls -ld 4_1/daily/200503131900/netbeans*-link.txt" 4_1/daily/200503131900/netbeans*-link.txt: No such file or directory $ ssh upload@cvs.netbeans.org "ls 4_1/daily/200503131900/*" 4_1/daily/200503131900/*: No such file or directory $ ssh upload@cvs.netbeans.org "ls 4_1/daily/200503131900/" 5309f345ee7c0568cd6de3d03b8e7eac netbeans-4.1-daily-200503131900-link.txt $
I have found further consequences of this issue. This issue actually blocks build publishing system from working. I cannot workaround this issue w/o rewriting core part of build publishing script. After discussion with Jack I promote this issue to P1.
Looking into this issue , will be updating the issue with the status as soon as possible . Jobin.
I had filed a issue to operations the issue no is 35501 , will be updating this issue with the status . Jobin.
The operations have worked on the issue , and was able to upload the file "ls -ld 4_1/daily/200503131900/netbeans*-link.txt", please verify if this is still a issue . Jobin.
Created attachment 20834 [details] SSH verbose session log
The issue is still there.
It looks like the custom "shell" does escaping of wildcards. I mean it changes "*" to "\*" before passing the command to regular shell.
How did the build publishing system work on the staged version of the software then?
Unfortunatelly the ssh/scp to /download was not working properly before upgrade. I haven't had even role-request access to sun-netbeans project on extranet and my ssh/scp issue probably got lost amongst other issues Jack was solving. We didn't get the staging box's ssh/scp access to state where it could be worth to test it with production publishing tools. BTW: can we get back the version of "custom shell" which was there before upgrade? IMHO the new version of "custom shell" comes with more unrequested restrictions and w/o any useful additional features (I don't count allowing 'mkdir' command to additional features as we used to use recursive uploads instead).
It is bad that we did not test this thoroughly on the staging box. In any case, we still need it fixed ASAP. This is blocking publishing of any builds.
I have an issue open internally to get some investigation here but I could use an explanation as to how not being able to use wildcards in ssh commands prevents the build publishing system from working.
I don't know how this could help you fix the issue, but here we go. The build publishing system uses 'ls' command to list available files. To list those files the script uses set of wildcards. The result lists are then registered in downloads database (not @netbeans.org). Updating the script to avoid using wildcards requires 2-3 days of shell scripting work. This is good reason to call this issue major regression. Please do not assign the issue to me before you want me to test the fix. From my point of view you have got enough information for fixing and obviously to keep it assigned to 'support' until the fix is in place. Was there any good reason to update the custom shell assigned to the 'upload' account?
I forgot to mention that even the publishing script update is 2-3 days of shell scripting working it will then require update with every new deliverable file. In this case wildcards are used to save copy-pasting of the almost-the-same code for each and every possible deliverable. In year total it could become 15-20 days of shell scripting work if we go w/o wildcards. Does this sound like pretty good business justification?
We have disabled globbing in the upload shell for security reasons and that is not going to change, which is why I asked about the process you are using. Why could the script (and command lines for that matter) not be easily amended to list the files, recursively if necessary, and egrep for the files needed?
Brian, there is simply no security risk in doing 'ls *'. This is P1 REGRESSION and please fix it ASAP by allowing 'ls' command to be used with wildcards. Actualy I don't need to do 'rm *', what is probably the mentioned security reason. Just in case there is even minimal security risk related with current version of 'ls' command installed on the box, I would like to see i.e. SERT response or document on how-to issue a security attack with 'ls' command in connection with wildcards (where wildcards is a must for successful security breach). I see no reason to accomodate work-labour workaround into publishing scripts, because there's no reason to keep this regression in place as it is now.
Collab, any update ? Regardless of how "*" is used or not, this is changed behaviour in a critical SC component. We need this fixed ASAP. Please respond.
I have updated the internal issue on the reason why this is a critical issue and the importance why this issue has to be fixed as soon as possible , awaiting the engineers response , will be updating as soon a response is received .
The workaround to your scripting would be to use: ssh upload@cvs.netbeans.org "ls -ld 4_1/daily/200503131900/" | \ egrep -e 'netbeans.*-link.txt' instead of: ssh upload@cvs.netbeans.org "ls -ld 4_1/daily/200503131900/netbeans*-link.txt" A find-and-replace in the scripts should suffice to make this change. It would be nice to have a _technical_ reason why our workaround would be effort intensive to implement or why it would not work at all. Please call the support hotline at 650.228.2561 if you would like to discuss in person. Wildcard use has been disabled following the fundamental security principle of least privilege. (Searching www.cert.org for the words "least privilege" provides hundreds of hits, if you are looking for information on this principle. ) The concept is such that capabilities are provided only if they are absolutely necessary. We do not need to provide exploit information to justify our decision to disable globbing. We dispute this is a regression given the wildcard functionality was an undocumented feature in the product. Also, this situation is only critical now because this was not tested while the stage system was available.
I've spoken to Rudolf, he's home and offline and not able to chekc the details here. However he is very adamant that wildcards are the only real soloution here. The workaround will require some days work now, and updates at least once a month. This is not a satisfactory option from his POV. "Least privilege" could also imply removal of the entire ssh shell we have, so I don't really see that as a valid justification here. Of course it is good practice to always allow least privilege, until that affects the required functionality. Yes, it is critical because it wasn't tested properly. Yes, it is not a regression since Collab didn't know we were using this functionality. Moving on, we need it back.
Rudolf is adamant wildcard utilization is the only solution and we have provided a solution that makes use of the wildcard within the egrep command. It would be of great help if we could be edified why, specifically, the grep solution would not work. To date we have been only been told, "it is necessary", but there has been no justification. Given wildcards are not going to be available, the scripts need to be modified and if portions of the scripts could be included here, maybe we could begin to understand the adjustments that need to be made such that they can be adapted with as little effort as possible. Given Rudolf's explanation of how the system works at present ("The build publishing system uses 'ls' command to list available files. To list those files the script uses set of wildcards."), the grep solution will work. We have mobilized some of the best people at CollabNet to help with this and want to find a solution as urgently as you do. Please provide us with the details of your scripts such that we can assist in adapting them.
I was pushed to fix it on my side as there's much better way how to utilize my work time than handling this ridiculous restriction. Also time constraint was taken into account. It is probable my fix could be ready much earlier than yours.
We recently moved out from Collabnet's infrastructure