Bug 34633 - ReplaceRegExp periodically errors out with "Couldn't rename temporary file"
ReplaceRegExp periodically errors out with "Couldn't rename temporary file"
 Status: REOPENED None Ant Unclassified Optional Tasks (show other bugs) 1.7.0 All All P3 normal with 8 votes (vote) 1.8.0 Ant Notifications List

 Reported: 2005-04-26 20:20 UTC by nlitt 2013-02-14 16:16 UTC (History) 2 users (show) eolsen swordnik

Attachments
Improve Error Message for ReplaceRegExp Rename Failure (652 bytes, patch)
2008-04-02 11:46 UTC, Eric Olsen
Details | Diff

 Note You need to log in before you can comment on or make changes to this bug.
 nlitt 2005-04-26 20:20:23 UTC Recently I encountered an specific laptop on which the replaceregexp task intermittently fails with errors like: - Couldn't rename temporary file C:\\rep1253479585.tmp - Couldn't rename temporary file C:\\replace690408058.txt I see that this error message is generated by line 431 of the ReplaceRegExp task. The task creates a temp file, manipulates it, then overwrites the original file with the temp file. The "Couldn't rename temp file" error signifies a failure to mv the tempfile to the original. The user is running as root, so it's not a permissions issue. And replaceregexp works most of the time, but certain calls always seem to crash with this error. So a single target might call replaceregexp successfully four or five times, then error out. Googling the error, I found one other person who has encountered it and was also stumped: http://64.233.161.104/search?q=cache:SyAlUcJaA7cJ:www.mail-archive.com/user% 40ant.apache.org/msg01075.html+Ant+%22Couldn%27t+rename+temporary+file%22&hl=en Searching through ASF Bugzilla, I could not find any bug that appears to be related. Heinz van Pee 2005-06-13 10:24:38 UTC I have the same problem with Apache Ant version 1.6.1 compiled on February 12 2004. Ant is executed via Eclipse Im Running Windows XP with service pack 2. Heinz van Pee 2005-06-14 09:59:52 UTC I also encountered this effect when running ant 1.6.5 from console. Heinz van Pee 2005-06-14 10:01:22 UTC Forgot to mention the java version, using j2sdk1.4.2_04 Heinz van Pee 2005-06-14 11:38:49 UTC i have updated to j2sdk1.4.2_08 and disabled indexing for fast search, maybe update to 1.4.2_08 is enough, all run since then proceeded without errors. Heinz van Pee 2005-06-14 11:44:49 UTC (In reply to comment #4) i was too fast, its still there occassionally. Heinz van Pee 2005-06-14 12:20:49 UTC It seems that if this behaviour occured once it occures on every run thereafter.  Heinz van Pee 2005-06-14 13:19:15 UTC It seems to help if you delete all temporary files. Mark Crocker 2006-05-04 21:49:39 UTC There's a possibility that this is a file naming bug related to long vs. short file names in Windows. The error I got was: An error occurred processing file: 'C:\dev\LongFilenameExample\build\text\Example.java.html': C:\dev\LongFilenameExample\build.xml:138: Couldn't rename temporary file C:\DOCUME~1\mcrocker\LOCALS~1\Temp\replace1599386392.txt Notice that the source file and build file names use the full file names, but the temporary file uses the tilde vFAT naming style. This error occurs on Windows XP Pro 2002 SP2, with ANT 1.6.5 under Java 1.4.2_04 and 1.5.0_02 from the command line or inside of Eclipse 3.1. csaona 2007-07-25 03:36:41 UTC I'm also experiencing this problem with Ant 1.7.0 and Java 1.5.0_11 under Windows XP SP2 Peter Reilly 2007-08-21 11:55:31 UTC Marking as windows xp, but it applies to all windows versions. Karol Bucek 2007-09-20 02:00:01 UTC I think I have encountered it under Linux. I am using Fedora 7, ant (1.6.5) is running under the hood while using maven (2.0.7) with appfuse project. My stack trace is: [INFO] ------------------------------------------------------------------------ [ERROR] FATAL ERROR [INFO] ------------------------------------------------------------------------ [INFO] Couldn't rename temporary file /tmp/replace688779408.txt [INFO] ------------------------------------------------------------------------ [INFO] Trace Couldn't rename temporary file /tmp/replace688779408.txt at org.apache.tools.ant.taskdefs.optional.ReplaceRegExp.doReplace(ReplaceRegExp.java:431) at org.apache.tools.ant.taskdefs.optional.ReplaceRegExp.execute(ReplaceRegExp.java:491) at org.appfuse.mojo.installer.InstallSourceMojo.removeWarpathPlugin(InstallSourceMojo.java:609) at org.appfuse.mojo.installer.InstallSourceMojo.execute(InstallSourceMojo.java:207) at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:443) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:539) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:493) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:463) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:311) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:278) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:143) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:334) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:125) at org.apache.maven.cli.MavenCli.main(MavenCli.java:280) It seems I am not alone with this issue: http://www.mail-archive.com/users@appfuse.dev.java.net/msg06914.html http://www.nabble.com/appfuse-and-SOAP-t4035996s2369.html  Martyn Hiemstra 2007-12-13 03:29:57 UTC I can now confirm that this a serious Java bug. I discovered this by doing the following: I downloaded the ant source and I discovered that the original file is copied to a temp file and then that temp file is renamed to the original file. So in short, the original file is copied, edited and then overriden. So I decided to do some experimenting. I moved my source code from 'My Documents' directory which is located under the Document and Settings directory and moved it into c:\Project. Now the replaceregexp task works. Apparently Java doesnt know to handle files like this: C:\Documents and Settings\User\My documents\Projects\MyApp\projects\dao\src\conf\hibernate.properties I tried the following: if (!new File("C:\\Documents and Settings\\User\\My documents\\Projects\\MyApp\\projects\\dao\\src\\conf\\hibernate.properties").delete()) System.out.println( "Could not delete" ); else System.out.println( "Could delete" ); As output I get the message 'Could delete' and the file is deleted. This makes me beleive that there is something totally messed up in Java. Deleteing a file in one instance doesnt work but manually creating a File object pointing to that file and then deleting it will work. This isnt a Ant issue but a Java issue. Peter Reilly 2007-12-13 03:48:43 UTC I would suggest that it may be a windows issue. Things like this can happen with virus checkers. If a virus checker is configured to check a file when a file is created/read, and the file is short lived - for example a temp file created with there replaceregexp, there exists a race condition where the file cannot be deleted (while the virus checker is checking the file). [This is a windows "feature" apparently added with WindowsNT - previously one could delete files while other "processes" were reading them - corrupting the file system] Sometimes a virus checker is configured to only do "heavy" checking on certain directories - (My documents for example) so you may be seeing this for some directories and not others. Martyn Hiemstra 2007-12-14 08:02:36 UTC > I would suggest that it may be a windows issue. > Things like this can happen with virus checkers. To test this, I disabled my virus scanner and retested it. Still the same error. So it isnt a virus scanner that is causing the problem. It was a good suggestion though. David Jackman 2007-12-14 08:30:05 UTC (In reply to comment #14) > > I would suggest that it may be a windows issue. > > Things like this can happen with virus checkers. > > To test this, I disabled my virus scanner and retested it. Still the same error. > So it isnt a virus scanner that is causing the problem. It was a good suggestion > though. This sounds like a problem I was experiencing with files in a different situation. Virus scanning software was also the expected culprit, but wasn't. Instead, I tracked it down to the TSVNCache.exe process of TortoiseSVN. I was able to work around the problem by going to the Icon Overlays page of the TortoiseSVN settings and under "Status Cache" choosing "Shell" instead of "Default". Eric Olsen 2008-04-02 08:11:34 UTC (In reply to comment #8) This is not an issue with Windows long vs. short names. I ran into the same error you did in the Documents and Settings Temp area with Ant 1.7.0, Windows Server 2003, and Java 1.4.2_13 (don't ask). So, I switched the user's environment TEMP to point to C:\temp and got: BUILD FAILED K:\O_build_stable\RSA_Build\Build\build.xml:268: The following error occurred while executing this line: K:\O_build_stable\RSA_Build\Build\subprojects\buildsummary.xml:350: Couldn't rename temporary file C:\Temp\replace8787748.txt  Eric Olsen 2008-04-02 11:46:33 UTC Created attachment 21771 [details] Improve Error Message for ReplaceRegExp Rename Failure Eric Olsen 2008-04-02 11:46:56 UTC The problem (at least the one that I encountered) is definitely not a Java bug or even a 'real' bug in Ant. Instead it is one of misdirection due to a bad error message. It turns out that everyone seems to be focused on the temporary file (e.g., C:\\replace690408058.txt) when the problem is with the original file due to the way the Ant FileUtils rename works in order to work across file systems: 1 - delete original file if there 2 - copy updated file (temp file) to original file name and location 3 - delete updated file (temp file) The diff patch that I have attached to org/apache/tools/ant/taskdefs/optional/ReplaceRegExp.java will help sort out the bad error message confusion. In one of my tests the original output was: BUILD FAILED C:\AntTestDir\build.xml:32: Couldn't rename temporary file C:\DOCUME~1\eolsen\LOCALS~1\Temp\replace2143556994.txt The new (correct) output is: BUILD FAILED C:\Documents and Settings\eolsen\My Documents\AntTestDir\build.xml:32: Failed to delete W:\test2.txt while trying to rename C:\DOCUME~1\eolsen\LOCALS~1\Temp\replace1613181649.txt The real issue as you can now see from the improved error message was a failure to delete the original file (which was on a filesystem to which I only had read permissions). To misquote Matt Inger: "Read code, not error messages. Error messages can lie." Eric Olsen 2008-04-02 11:50:02 UTC Updating versions and systems since the patch applies to all platforms. Eric Olsen 2008-04-02 11:54:25 UTC One more note -- the real problem was that ReplaceRegExp.java gobbled up the good error message from org.apache.tools.ant.util.FileUtils.rename() instead of passing it on up the chain to the build failure where we could see it. Stefan Bodewig 2008-10-09 08:59:41 UTC the fix for bug 45960 may have improved the situation. For other cases (like when Ant simply cannot write to the file), svn revision 703192 will now include the original error information (as suggested by Eric). Daniel Holmes 2011-04-13 11:28:26 UTC I am still seeing this issue in Ant 1.8.1. Would there be a way to possibly handle the exception with a retry of the failed command (in this case looks like the original file delete). This seems to happen often enough in my windows build execution since I started using the ReplaceRegExp task. Once every 50 or so uses, which based on my CI build, fails every fourth or fifth build execution. See ant -v full stact trace below for the underlying cause in the ant FileUtils BUILD FAILED c:\hudson\workspace\main_windows\build.xml:298: The following error occurred while executing this line: c:\hudson\workspace\main_windows\build.xml:140: The following error occurred while executing this line: c:\hudson\workspace\main_windows\modules\build-module-common.xml:399: The following error occurred while executing this line: c:\hudson\workspace\main_windows\modules\build-module-common.xml:532: Couldn't rename temporary file C:\WINDOWS\TEMP\replace7543532359339722099.txt at org.apache.tools.ant.taskdefs.optional.ReplaceRegExp.doReplace(ReplaceRegExp.java:480) at org.apache.tools.ant.taskdefs.optional.ReplaceRegExp.execute(ReplaceRegExp.java:536) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) ... (Extra stack trace clipped) ... Caused by: java.io.IOException: Failed to delete c:\hudson\workspace\main_windows\modules\dfm-cli\target\reports\javac\javac_stdout.txt while trying to rename C:\WINDOWS\Temp\replace7543532359339722099.txt at org.apache.tools.ant.util.FileUtils.rename(FileUtils.java:1239) at org.apache.tools.ant.taskdefs.optional.ReplaceRegExp.doReplace(ReplaceRegExp.java:474) ... 72 more gw1500se 2012-02-22 15:32:49 UTC I know this is an old thread but I have it today. It is also not a Windows specific problem as I am running Mandriva 2010.2, ant 1.7.1 and I don't know what version of replaceregexp I have. This error seems to be like a rash on the internet with no one coming up with a solution or even a workaround. Some have said it goes way on its own but mine hasn't yet and I cannot find any way to get rid of it even temporarily. My project is at a dead standstill at this point. Will rebooting help any? Stefan Bodewig 2012-02-22 16:43:49 UTC (In reply to comment #23) > I know this is an old thread but I have it today. It is also not a Windows > specific problem as I am running Mandriva 2010.2, ant 1.7.1 and I don't know > what version of replaceregexp I have. Since replaceregexp is a built-in task it should be the same version as Ant itself. Before tyring to dig any deeper, you should give a more recent version of Ant a try (1.8.2 is current with 1.8.3 being close). Even if bug 45960 says it is about Windows, it also applies to other platforms in that a failed delete operation is retried once (on Windows there is an additional GC between the attempts to delete the file). gw1500se 2012-02-22 18:40:18 UTC 1.8.2 is not yet a stable release for Mandriva. I am reluctant to try the cooker release if there is a workaround. I can survive for a while if there is some way to manually clear this problem even if temporarily. As for the version, I had lots of trouble getting replaceregexp since it kept telling me it was an ant-contrib optional function. I had to install it separately which is why I thought it might be a different version. Whatever I did, when I installed the Mandriva ant rpm, replaceregex was not included. Stefan Bodewig 2012-02-23 05:04:42 UTC (In reply to comment #25) > 1.8.2 is not yet a stable release for Mandriva. I am reluctant to try the > cooker release if there is a workaround. I can survive for a while if there is > some way to manually clear this problem even if temporarily. > > As for the version, I had lots of trouble getting replaceregexp since it kept > telling me it was an ant-contrib optional function. I had to install it > separately which is why I thought it might be a different version. Whatever I > did, when I installed the Mandriva ant rpm, replaceregex was not included. Well, we don't have any influence on the RPMs (or any other distribution you don't download from one of the ASF's mirrors), I don't have any idea whether the Mandriva distribution has made any modifications. In Ant 1.7.x the replaceregexp task has been in a jar called ant-nodeps.jar, with Ant 1.8.x we've merged those into ant.jar. You should be able to download a binary release from Ant's download page, untar the tarball anywhere, set ANT_HOME to point to that location and start using it without messing with your OS installation. I'm not aware of any workaround if the problem is that Java is holding on to a lock to the file it has just written even after it has been closed. gw1500se 2012-02-23 15:28:57 UTC I took your advice and installed 1.8. No joy, same error. However, I wonder if I needed to do some kind of clean up. Stefan Bodewig 2012-02-23 15:52:46 UTC At least 1.8.x should tell you which file it failed to delete. gw1500se 2012-02-23 18:08:25 UTC I've always had that, from replaceregexp: Couldn't rename temporary file /tmp/replace5358820582224705130.txt However, I've just discovered what I think is an important piece of information and, as an aside, may be related to what I thought might be a different bug. I have been running this build via a perl script. What I have found is everything works as expected when the script is run from the command line. When it fails with this error, it is running in background which means no console (tty). The possible related bug is that I use the echo property in a couple of places. When I try to capture the ant output, the echos do not show up when run in background. They do from command line. I was able to work around that problem by using -logfile and reading that into my script but I shouldn't have had to do that. The bottom line is that ant behaves differently when run in background and/or with no console. It does not seem to matter if I use 'nohup'. Unless it is trying to write directly to a tty rather than STDOUT and STDERR, this behavior should not happen. I suppose it is possible that there is some environment variable missing that it needs but there is no indication in the error message if or what that might be. ittai 2012-03-30 11:40:52 UTC I'm also seeing this on a CentOS installation. I can check on sunday what version exactly the ant is and I think I can update it without a problem. This is a real problem for me. Any workarounds? (BTW, the file isn't held by java in my case) raghavendra.1729 2012-04-02 12:27:53 UTC Hi I'm too facing the same issue, I tried with ANT version ant_1.6.5 on RHEL 5.4. Any quick workaround can help me lot. Thanks Raghav David S 2012-12-10 22:20:33 UTC I just faced the same problem, but got it fixed very easily. Turns out that apparently, it is related to the path/filename of the file to which the regexp replacement is being done. So, instead of trying to do the regexp directly to thefile (located in a path like C:\build\nightly\files\product_20121210_1600\file.txt), I just tried the following: 1)copy the file to C:\Temp 2)attempt the regexp to that file (C:\Temp\file.txt) 3)copy the file from C:\Temp to C:\build\nightly\files\product_20121210_1600\file.txt and that's it. Worked like a charm. I was using ANT 1.7.1, then moved to 1.8.4 to see if it solved the problem (which didn't). However, I'm assuming it should work with 1.7.1 as well. David S 2012-12-10 23:52:12 UTC In fact, by double checking, my previous comment is wrong. When I tried to copy back the file to the original path, I got an access denied error. And after verifying, indeed the original file was always having read-only permissions. That was my problem (not the file path/ file name). Check your file permissions, you might be facing the same thing I was (rookie's problem!) Pitecantrop 2013-02-13 14:11:06 UTC Another workaround is redefining temporary dir location. Сan't find some rule, but overriding the temporary dir to the drive where you run ant should help. ant -Djava.io.tmpdir=D:\temp ...