The tags... <property environment="osenv"/> <exec executable="ls" /> do not work properly under OS/400. The problem is that the output from the QSH commands is EBCDIC. The output must be converted so it is appropriate for Java. I have made the appropriate changes to the 1.5 release to accomplish this and request that these changes be reviewed and considered for placement into the product. The files affected and the differences are attached. I believe that there is also a bug in the org.apache.tools.ant.taskdefs.Execute.toString (ByteArrayOutputStream bos) method. It attempts to convert the byte array encoding for z/os, but does not actually return the encoded string. This is a moot point because the encoding conversion belongs in a different place anyways (along with the os/400 encoding in the StreamPumper class.) Source Differences: diff -r ant/src/main/org/apache/tools/ant/taskdefs/Execute.java ant400/src/main/ org/apache/tools/ant/taskdefs/Execute.java 237a239,241 > } else if (Os.isFamily("os/400")) { > String[] cmd = {"env"}; > return cmd; 262,267d265 < if (Os.isFamily("z/os")) { < try { < bos.toString("Cp1047"); < } catch (java.io.UnsupportedEncodingException e) { < } < } diff -r ant/src/main/org/apache/tools/ant/taskdefs/StreamPumper.java ant400/src/ main/org/apache/tools/ant/taskdefs/StreamPumper.java 59a60,61 > import java.io.ByteArrayOutputStream; > import org.apache.tools.ant.taskdefs.condition.Os; 88a91,114 > public static byte [] recode(byte [] buf, int length) { > byte [] rbuf = null; > if (Os.isFamily("z/os")) { > try { > ByteArrayOutputStream bos = new ByteArrayOutputStream(); > bos.write(buf, 0, length); > rbuf = (bos.toString("Cp1047")).getBytes(); > } catch (java.io.UnsupportedEncodingException e) { > } > } > else if (Os.isFamily("os/400")) { > try { > ByteArrayOutputStream bos = new ByteArrayOutputStream(); > bos.write(buf, 0, length); > rbuf = (bos.toString("Cp500")).getBytes(); > } catch (java.io.UnsupportedEncodingException e) { > } > } > else > { > rbuf = buf; > } > return rbuf; > } 106c132,133 < os.write(buf, 0, length); --- > byte [] rbuf = recode(buf, length); > os.write(rbuf, 0, length); diff -r ant/src/main/org/apache/tools/ant/taskdefs/condition/Os.java ant400/src/ main/org/apache/tools/ant/taskdefs/condition/Os.java 102a103 > * <li>os/400</li> 216a218,219 > } else if (family.equals("os/400")) { > isFamily = osName.indexOf("os/400") > -1;
Brian, I have committed a modified version of your patch to CVS HEAD, could you please verify using a homebrewed Ant or a nightly build of 2002-09-28 or later that it works on OS/400? I've kept the Execute#toString method as it also gets used from other places (to set the property in when using <exec>'s outputproperty) and it is in Execute to keep all system specific code in a single place. Thanks for pointing out the missing return BTW. I'm setting the target for this patch to appear to 1.6, but we may talk about getting it into 1.5.1 if things work as expected.
I have reviewed the source changes for this enhancement and have the following comments: - The changes submitted for StreamPumper.java were not present in CVS. - The code removed from Execute#toString was intentional and necessary. The recoding is being done in the StreamPumper class for both the <property environment> and <exec> tags. Performing the recoding in the Execute#toString method will have the affect of performing the recoding twice. Please complete the changes to StreamPumper.java and remove the recoding for z/os and os/400 in Execute#toString. Once this is complete I will pull the code and test on os/400. Thanks, Brian Farrar
Not adding it to StreamPumper but leaving it in Execute has been intentional as well, sorry. StreamPumper can be used (and I think it will be used) in other contexts where it would be wrong to convert characters.
Then there is a problem. Execute#toString only fixes it for the <property environement> tag, so this isn't a good place to put it. The appropriate place to do the recoding for os/400 (and any other os needing recoding) would be where the stream is coming in the system exec command. I was thinking that StreamPumper was where the stream from the system command was being handled. If this is so then it would be the correct place, if not then where would that be?
For some reasons the support for z/OS only needed to fix the sitation for property (but then again, the actual code patch doesn't seem to have been tested on z/OS). StreamPumper would probably be the right place, but we may need to make translation conditional. Let's see what Ant does with output of <exec> 1) pipe it to a file (<exec output="....")/>) 2) pipe it to Ant's internal logging system (<exec> with no output-attribute) 3) capture the result in a property (<exec outputproperty>) 4) postprocess the output (the environment stuff, javadoc to swallow some lines, some other tasks parse the output as well). You say you need to recode the output in all cases for OS/400, correct?
Adding in to StreamPumper took care of all the problems for me (I'm surprised z/os "only" needed it for <property environment>, could be z/os didn't test the <exec> tag.) It is my belief that _any_ stream coming back from the "command shell" will be in the native encoding (EBCDIC Cp500 in my case) and therefore will need to be recoded for Java. You have raised a very good point that the recoding should be conditional (and the encoding type should be selectable.) My environment is using Cp500, which is pretty safe but is just one of several (hundred) possible. If I were to rewrite the StreamPumper, rather than look at the machine type I would look at a property to determine if recoding should occur and to what encoding type. This would allow for recoding on any future platform with a property setting.
I never noticed this outstanding bug before, but reading it makes me wonder two things: (1) Whether the new <redirector> stuff in CVS could handle this, and (2) if that's the reason Stefan suggested transcoding as a feature of the <redirector>.