Bug 28320 - copy task preserving permissions
Summary: copy task preserving permissions
Status: NEW
Alias: None
Product: Ant
Classification: Unclassified
Component: Core tasks (show other bugs)
Version: 1.6.1
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---
Assignee: Ant Notifications List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-04-09 20:40 UTC by Steven Schveighoffer
Modified: 2011-05-31 20:13 UTC (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Steven Schveighoffer 2004-04-09 20:40:48 UTC
The copy task currently does not preserve the permissions of the source files 
and uses the permissions dictated by umask.  It would be nice if there was a 
flag to preserve the permissions as much as possible, in the manner that cp 
does it.
Comment 1 Dominique Devienne 2004-04-09 20:54:25 UTC
Unfortunately, there's no way to do this with Java... You have to resort to 
native command you'd <exec> to preserve permissions. --DD
Comment 2 Alexey Solofnenko 2004-04-09 20:59:27 UTC
I had a similar problem. Somebody recommended to use Perl to copy permissions and it worked fine. This is a script that I am using to copy executable permission:

my ($src, $dst)=@ARGV;
#print "cpmod '$src' '$dst'\n";

my $srclen=length $src;

find (
  {
    no_chdir=>1,
    wanted=>sub {
      return if -d;
      my $destfile= $dst.substr $_, $srclen;
      if (-x and -f $destfile and not -x $destfile) {
        my $mode=(stat $destfile)[2];
        my $newmode=$mode | 0111; # +x
        if ($mode != $newmode) {
          #print "chmod +x [".sprintf("%lo", $mode)." -> ".sprintf("%lo", $newmode)."] '$destfile'\n";
          chmod $newmode, $destfile
        }
      }
    }
  },
  $src
);
Comment 3 Alexey Solofnenko 2004-04-09 21:00:22 UTC
Sorry, lost two lines at the beginning:

use strict;
use File::Find;
Comment 4 Steven Schveighoffer 2004-04-09 21:03:10 UTC
I think it is possible, since you have the chmod task.  If you claim it is not 
possible through Java to change the permissions, then you are doing it some 
other way for this native task, why can't the copy task also use this method?
Comment 5 Dominique Devienne 2004-04-09 21:14:58 UTC
<chmod> exec's chmod on *nix, and does nothing on Windows.

<copy> is implemented fully in Java, OTOH. You'd need to somehow post-process 
the copied files to restore the same permission from the source files on to 
the destination files. Patches welcomed ;-) --DD
Comment 6 Steven Schveighoffer 2004-04-12 21:01:48 UTC
How against JNI code is ant?

The reason I ask is I did some investigating on *reading* file permissions, 
and the only options are:

1. use ls (which doesn't help much for spitting out ONLY file permissions)
2. use JNI and call the system calls directly from java.
3. write a program similar to chmod which reads file permissions, and use that 
program to read the permissions before copying and chmod'ing them.

I appreciate the response with the perl script, but that isn't an option for 
me as I don't know or use perl.  The problem I'm trying to solve is that I'm 
trying to use ant to build executables (from C++ code), which it is able to do 
just fine (I'm using ant-contrib).  However, when copying the executable to 
the output directory, the file loses its executable permission.

The benefits of using JNI would be greater than just being able to read file 
permissions, you would also be able to set file permissions without having to 
execute another program (i.e. change the chmod task).  You could also easily 
port a JNI version to non-unix filesystems (NTFS comes to mind).

The only drawback, of course, is that to build file permission support, you 
need to have a c-compiler to build the native part of the class.  In my case, 
I have no problem with this.  However, other people may have issues with using 
a C-compiled library with ant.  In reality, it is no different than using 
chmod, since you are in effect using a program built in c to do the dirty work 
that Java does not support.  The build script for ant could also be modified 
to optionally use the old method for chmod in case people are very averse to 
building the JNI interface.

Anyways, I made my case, if the ant developers think this is a BAD idea, then 
I will just not contribute the work that I will inevitably do, as I still need 
to solve the problem, and ant is just way too good to not use for this minor 
issue :)
Comment 7 Dominique Devienne 2004-04-12 21:12:01 UTC
It's unlikely Ant will accept C-code Steven, and a dependence on a JNI native 
lib... But the guys Ant would be happy (I think ;-) to add some hook (like a 
protected setPermission(File src, File dst) method in Copy.java) which would 
allow you to extend <copy> in your own AntLib, which could depend on JNI code.

If you make your code OpenSource, others would be able to contribute ports of 
the C code to the many platforms Ant is used on. This project could also act as 
a small CppTasks demo project, to compile the C-code ;-) --DD

Comment 8 Alexey Solofnenko 2004-04-12 21:41:40 UTC
If you know what files you need to be executable (like **/*.sh or bin/) you can always do it manually with <chmod>. A perl or some native script would be simple too and [almost] all Unixes have perl already installed.
Comment 9 Steve Loughran 2004-04-12 21:47:24 UTC
A native code impl would probably find its home in the ant-contrib project on
sforge, presumably with a task <cp> that does permission setting. 

I dont know about DD's idea of putting a special permission preserving back door
in everywhere, as <copy> is only one of many places people would ask; zip, tar &
lots of others could add it. But then the result would be that build files that
worked to build a redistributable on unix would work wrongly on windows, or
netware, or VMS.

NTFS file permissions are so different from Unix land that I never want to go
near them. And that is me speaking from experiene :)

Coming back to the specific build file problem, is a chmod after copy so hard? 

(NB, one fix that would be strongly acceptable, a patch to java1.5 :)

Comment 10 Steven Schveighoffer 2004-04-12 22:16:50 UTC
I understand the unwillingness to accept native code as a part of Ant, 
especially since there really isn't an ant-proper way to build shared 
libraries yet.  I also agree that a patch to java 1.5 (very unlikely) would be 
the best answer.

I must disagree on your interpretation in that building with file-permission 
correct copy would work "wrongly" on other platforms.  It currently 
works "wrongly" on ALL platforms, as the operating system/environment decides 
the permissions of the new file, allowing differences in operating system 
environments to affect the build process.  In the case of the current release 
of Ant, it is by your standards broken since the chmod task is not implemented 
on all platforms, making any build which uses chmod build "wrongly" on non-
unix platforms.  This new feature would open the door to an actual correct 
solution, which is consitant across all platforms.

I'm more inclined not to use chmod as I need to build my projects on both 
Linux and Windows (hence the attractive nature of Ant).  In addition, a 
permission-preserving copy seems like a much more natural solution than having 
to use chmod after each copy.

For now, I'll see if ant-contrib will accept the library, and some task I 
create, as part of its build, since it really is the only ant-like project 
that could build such a library anyways :)
Comment 11 Steve Loughran 2004-04-13 11:23:48 UTC
We have mis-explained <chmod>. 

<chmod> is a noop on Windows. So you put it in your code to set the exe
permissions. On unix, after the task is invoked, the target files are executable
because the permissions were set. On windows, after the task is invoked, the
target files are executable because there was no need to set the permissions.

To repeat: use chmod after your copy, it has the desired outcome on all platforms.

Also, when archiving stuff up, use <tarfileset> to set the permissions on a tar
file. It will do this regardless of the platform you are running on. 

Comment 12 Steven Schveighoffer 2004-04-19 20:27:00 UTC
I have completed my addition to ant which allows file permission preserving 
while copying.

The way I did this was to extend the FileUtils class, overriding the copyFile 
method.  Then I extended Copy to a new task which used the new FileUtils class 
to do the copying.  So far, the library I wrote will (eventually) work on both 
Posix and NTFS filesystems, but I haven't yet tested it on NTFS.

I ran into one snag, which I hope can be fixed.  Copy keeps the FileUtils 
object that it uses as a private member, so I had to modify ant's Copy.java 
file to change the permission of the fileUtils member to protected.  Any way 
this update could be made permanent?

Another thought on this is that the copyFile method in FileUtils is used in 
more than just the Copy task.  If there was a way to instruct all of Ant's 
core tasks to use a *different* object other than Ant's FileUtils (i.e. my 
extended version), then I could keep the same task names, and all the copy 
operations will preserve permissions.
Comment 13 Steve Loughran 2004-04-20 09:07:43 UTC
1. yes, we could unprotect the access.

2. But you point to a better way. The FileUtils() stuff was meant to be
exensible, which is why the methods are not static. Instead everyone has to go 
new FileUtils.doSomething();

So, we could provide a means to name the factory object used to create the
fileutils instance, and from then on everyone who asks for one, gets the new
one. That would be very slick. It would give you what you want with minimal
changes to the rest of ant. 
Comment 14 Steven Schveighoffer 2004-04-20 14:19:06 UTC
That sounds like a great solution.

I will wait to contribute my stuff to the ant-contrib project until this has 
been implemented, as my current fix requires a patch to ant.  If you need 
someone to do it, I'll try my best, but I may need help on the design.  I have 
sort of learned Java only when necessary :)
Comment 15 Steven Schveighoffer 2004-06-16 14:55:40 UTC
Digging this back up again.

Just to let you know, I have completed a library which does file permissions 
on both filesystems with Posix-based permissions and NTFS.

I have contributed this to the jst (Java System Toolkit) library, which is 
maintained on sourceforge (jst.sourceforge.net).  Interestigly enough, there 
was NO build system in place on that project, which I fixed by creating an ant 
build file, which uses cpptasks from ant-contrib to build the native libraries.

I now await the fix for the FileUtils to be overridable.  Any status on this?  
Is there something I can help with?

I noticed some other annoying things on ant that could be fixed by permission 
getting/setting.  untar doesn't set the file permissions up properly (which 
could be fixed by the new jst library).  Also, tar itself relies on the ant 
arguments to set permissions, very annoying.
Comment 16 Stefan Bodewig 2005-03-17 17:48:42 UTC
Unfortunately we've changed FileUtils to statics by now and the original
factory approach went away.

That would leave us with adding hooks to tasks that you then needed to extend and
override.
Comment 17 J.M. (Martijn) Kruithof 2005-03-17 19:20:33 UTC
(In reply to comment #16)
> Unfortunately we've changed FileUtils to statics by now and the original
> factory approach went away.
> 
> That would leave us with adding hooks to tasks that you then needed to extend and
> override.
> 

The shared static FileUtils is always fetched via the getFileUtils() method on
the FileUtils class. If wanted, this shared instance could be a different one,
per platform. the getFileUtils / FileUtils can be adapted to get the primary
instance via a factory method. (The old newFileUtils() wasn't a configurable
factory either.) Just different FileUtils per Task cannot be made via
getFileUtils(), another for the entire build, depending on OS would be feasible
though.
Comment 18 J.M. (Martijn) Kruithof 2005-03-17 19:30:23 UTC
The following methods are static and are good newFileUtils(deprecated,
superseded by getFileUtils) / getFileUtils

The following methods do not really do much harm:

close(*)
delete(File)
readFully() <-- what is this one doing in FileUtils anyway?


The following methods are "bad" to be static, because platform specific
implementations should have the possibility to use different implementations:
isAbsolutePath/isContextRelativePath
Comment 19 Matthew Wilson 2007-05-30 00:43:46 UTC
Hi guys,

Even if we can't preserve full unix permissions, It would still be useful to be 
able to preserve just the read-only status of files copied by the <copy> task.

Java can do this much, and it's something my project found very handy (I did a 
quick change to ant - FileUtils and Copy are all that's changed).  We use an 
ant script (exec'ed by an eclipse external task) to synchronise each desktop 
development area with a source baseline, and by preserving the read-only status 
on the copied files we can hamper accidental changes to files which haven't 
been explicitly write-abled.  The developers like it.

Is this worth adding to the standard build?   It's a simple change, and I'm 
sure there are people who'd find it useful.  

Ta.

Comment 20 Igor Urisman 2011-05-27 21:40:39 UTC
The following note is completely bogus:
Unix Note: File permissions are not retained when files are copied; they end up with the default UMASK permissions instead. This is caused by the lack of any means to query or set file permissions in the current Java runtimes. If you need a permission-preserving copy function, use <exec executable="cp" ... > instead. 

Ant is the application here, not the build script.  Java never claimed to be an OS. But there's nothing that keeps Ant from knowing what OS it's running on and if it's Unix, use a system call to do the OS appropriate copy.  Indefensibly bogus.
Comment 21 Jesse Glick 2011-05-31 20:13:06 UTC
Note: Java 6 supports setting general r/w/x permissions on a file. Java 7 supports access to a broad range of platform-specific permissions, and also has Files.copy(..., ..., COPY_ATTRIBUTES).