Bug 13047 - Support for <property environment> and <exec> on OS/400
Summary: Support for <property environment> and <exec> on OS/400
Alias: None
Product: Ant
Classification: Unclassified
Component: Core tasks (show other bugs)
Version: 1.5
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 1.6
Assignee: Ant Notifications List
Depends on:
Reported: 2002-09-26 21:04 UTC by Brian Farrar
Modified: 2008-02-22 12:18 UTC (History)
0 users


Note You need to log in before you can comment on or make changes to this bug.
Description Brian Farrar 2002-09-26 21:04:47 UTC
The tags...
<property environment="osenv"/>
<exec executable="ls" />
do not work properly under OS/400.

The problem is that the output from the QSH commands is EBCDIC.  The output 
must be converted so it is appropriate for Java.

I have made the appropriate changes to the 1.5 release to accomplish this and 
request that these changes be reviewed and considered for placement into the 

The files affected and the differences are attached.  I believe that there is 
also a bug in the org.apache.tools.ant.taskdefs.Execute.toString
(ByteArrayOutputStream bos) method.  It attempts to convert the byte array 
encoding for z/os, but does not actually return the encoded string.  This is a 
moot point because the encoding conversion belongs in a different place anyways 
(along with the os/400 encoding in the StreamPumper class.)

Source Differences:

diff -r ant/src/main/org/apache/tools/ant/taskdefs/Execute.java ant400/src/main/
>         } else if (Os.isFamily("os/400")) {
>             String[] cmd = {"env"};
>             return cmd;
<         if (Os.isFamily("z/os")) {
<             try {
<                 bos.toString("Cp1047");
<             } catch (java.io.UnsupportedEncodingException e) {
<             }
<         }

diff -r ant/src/main/org/apache/tools/ant/taskdefs/StreamPumper.java ant400/src/
> import java.io.ByteArrayOutputStream;
> import org.apache.tools.ant.taskdefs.condition.Os;
>     public static byte [] recode(byte [] buf, int length) {
>       byte [] rbuf = null;
>         if (Os.isFamily("z/os")) {
>             try {
>                 ByteArrayOutputStream bos = new ByteArrayOutputStream();
>                 bos.write(buf, 0, length);
>                 rbuf = (bos.toString("Cp1047")).getBytes();
>             } catch (java.io.UnsupportedEncodingException e) {
>             }
>         }
>         else if (Os.isFamily("os/400")) {
>             try {
>                 ByteArrayOutputStream bos = new ByteArrayOutputStream();
>                 bos.write(buf, 0, length);
>                 rbuf =  (bos.toString("Cp500")).getBytes();
>             } catch (java.io.UnsupportedEncodingException e) {
>             }
>         }
>       else
>       {
>            rbuf = buf;
>       }
>       return rbuf;
>     }
<                 os.write(buf, 0, length);
>               byte [] rbuf = recode(buf, length);
>                 os.write(rbuf, 0, length);

diff -r ant/src/main/org/apache/tools/ant/taskdefs/condition/Os.java ant400/src/
>      *               <li>os/400</li>
>                 } else if (family.equals("os/400")) {
>                     isFamily = osName.indexOf("os/400") > -1;
Comment 1 Stefan Bodewig 2002-09-27 08:19:54 UTC

I have committed a modified version of your patch to CVS HEAD, could you please
verify using a homebrewed Ant or a nightly build of 2002-09-28 or later that it
works on OS/400?

I've kept the Execute#toString method as it also gets used from other places
(to set the property in when using <exec>'s outputproperty) and it is in Execute
to keep all system specific code in a single place.

Thanks for pointing out the missing return BTW.

I'm setting the target for this patch to appear to 1.6, but we may talk about
getting it into 1.5.1 if things work as expected.
Comment 2 Brian Farrar 2002-09-27 12:55:45 UTC
I have reviewed the source changes for this enhancement and have the following 

- The changes submitted for StreamPumper.java were not present in CVS.

- The code removed from Execute#toString was intentional and necessary.  The 
recoding is being done in the StreamPumper class for both the <property 
environment> and <exec> tags.  Performing the recoding in the Execute#toString 
method will have the affect of performing the recoding twice.

Please complete the changes to StreamPumper.java and remove the recoding for 
z/os and os/400 in Execute#toString.  Once this is complete I will pull the 
code and test on os/400.

Brian Farrar
Comment 3 Stefan Bodewig 2002-09-27 13:04:24 UTC
Not adding it to StreamPumper but leaving it in Execute has been intentional
as well, sorry.  StreamPumper can be used (and I think it will be used) in other
contexts where it would be wrong to convert characters.
Comment 4 Brian Farrar 2002-09-27 13:14:26 UTC
Then there is a problem.  

Execute#toString only fixes it for the <property environement> tag, so this 
isn't a good place to put it.  The appropriate place to do the recoding for 
os/400 (and any other os needing recoding) would be where the stream is coming 
in the system exec command.  I was thinking that StreamPumper was where the 
stream from the system command was being handled.  If this is so then it would 
be the correct place, if not then where would that be?
Comment 5 Stefan Bodewig 2002-10-04 12:15:53 UTC
For some reasons the support for z/OS only needed to fix the sitation for
property (but then again, the actual code patch doesn't seem to have been tested
on z/OS).

StreamPumper would probably be the right place, but we may need to make
translation conditional.  Let's see what Ant does with output of <exec>

1) pipe it to a file (<exec output="....")/>)

2) pipe it to Ant's internal logging system (<exec> with no output-attribute)

3) capture the result in a property (<exec outputproperty>)

4) postprocess the output (the environment stuff, javadoc to swallow some lines,
some other tasks parse the output as well).

You say you need to recode the output in all cases for OS/400, correct?
Comment 6 Brian Farrar 2002-10-04 12:46:22 UTC
Adding in to StreamPumper took care of all the problems for me (I'm surprised 
z/os "only" needed it for <property environment>, could be z/os didn't test the 
<exec> tag.)  It is my belief that _any_ stream coming back from the "command 
shell" will be in the native encoding (EBCDIC Cp500 in my case) and therefore 
will need to be recoded for Java.  

You have raised a very good point that the recoding should be conditional (and 
the encoding type should be selectable.)  My environment is using Cp500, which 
is pretty safe but is just one of several (hundred) possible.  

If I were to rewrite the StreamPumper, rather than look at the machine type I 
would look at a property to determine if recoding should occur and to what 
encoding type.  This would allow for recoding on any future platform with a 
property setting.
Comment 7 Matt Benson 2004-03-31 22:24:37 UTC
I never noticed this outstanding bug before, but reading it makes me wonder two 
things:  (1)  Whether the new <redirector> stuff in CVS could handle this, and 
(2) if that's the reason Stefan suggested transcoding as a feature of the