Bug 38353 - Unable to post large(>2GB) files using Slide API
Summary: Unable to post large(>2GB) files using Slide API
Status: NEW
Alias: None
Product: Slide
Classification: Unclassified
Component: Stores (show other bugs)
Version: 2.0
Hardware: All Windows 2000
: P2 critical (vote)
Target Milestone: ---
Assignee: Slide Developer List
Depends on:
Reported: 2006-01-23 07:42 UTC by Anupama
Modified: 2006-04-07 01:42 UTC (History)
0 users


Note You need to log in before you can comment on or make changes to this bug.
Description Anupama 2006-01-23 07:42:44 UTC
We are not able to store large files > 2GB when we use File Content Storage
(TxFileContentStore Storage).
Any file less then 2GB we are able to store into file system successfully.

When storing >2GB, we encounter the below pasted exception:

org.apache.slide.common.ServiceAccessException: Service TxFileStore at E:/Textup
loadtest/slide/store/content  working on E:/Textuploadtest/slide/work/content ac
cess error : Thread-19-1137482059804-60: Failed to undo change / create of '/113
7481500664.54_1.0' (ERR_SYSTEM)org.apache.slide.common.ServiceAccessException: S
ervice TxFileStore at E:/Textuploadtest/slide/store/content  working on E:/Textu
ploadtest/slide/work/content access error : Thread-19-1137482059804-60: Failed t
o undo change / create of '/1137481500664.54_1.0' (ERR_SYSTEM)
        at org.apache.slide.store.txfile.AbstractTxFileStoreService.throwInterna
        at org.apache.slide.store.txfile.TxFileContentStore.storeRevisionContent
        at org.apache.slide.store.txfile.TxFileContentStore.createRevisionConten
        at org.apache.slide.store.AbstractStore.createRevisionContent(Unknown So
        at org.apache.slide.store.ExtendedStore.createRevisionContent(Unknown So
        at org.apache.slide.store.BindingStore.createRevisionContent(BindingStor
        at org.apache.slide.store.BindingStore.createRevisionContent(BindingStor
        at org.apache.slide.content.ContentImpl.create(ContentImpl.java:538)
        at com.hyperion.eie.common.cms.WebdavPutMethod.putNewResource(Unknown So
        at com.hyperion.eie.common.cms.WebdavPutMethod.run(Unknown Source)
        at com.hyperion.eie.common.cms.WebdavPutMethod.executeMethod(Unknown Sou
        at com.sample.fileContent.Repository.uploadFile(Repository.java)

When debugging the slide code for the same we came to know the exception are 
thrown in the below code for the condition (if (contentLength != -1 && 
contentBytes != contentLength)), basically the contentLenght & contentBytes are 
not matching which should be same to store the file.

In TxFileContentStore.java there are two parameters:
long contentBytes = FileHelper.copy(is, os);
long contentLength = revisionDescriptor.getContentLength();
And a condition:
if (contentLength != -1 && contentBytes != contentLength) 
   rm.deleteResource(getActiveTxId(), revisionUri);
   throwInternalError("Content length does not match expected");

When the above condition(contentLength and contentBytes) fails, the flow will 
go to delete the resource(rm.deleteResource) which is not yet stored in 
repository and the exception what we pasted above is thrown from 
rm.deleteResource() which is not actually the real cause to store large files. 
So the actuall cause what we found is value of the contentLength and 
contentBytes are not matching.

Please let us know if any one has already faced/fixed this issue in Slide2.0.
We are also trying to fix this, but it would help us if any one has already 
solution for this.
Comment 1 Anupama 2006-01-23 07:51:36 UTC
When we commented the below code in In TxFileContentStore.java of method 

if (contentLength != -1 && contentBytes != contentLength) 
   rm.deleteResource(getActiveTxId(), revisionUri);
   throwInternalError("Content length does not match expected");

We were able to store on to the file system above 6 GB.
Does commenting this part of the code affect the normal functioning of the 
Comment 2 SVen 2006-01-23 08:00:18 UTC
I envountered this problem almost one and a half year ago. One reason was that
the content-length was read as int (which overflows at 2147483647 byte, almost 2GB).
But after patching all Slide sources to use long the problem remained, so I
guess this is a bug in Tomcat (which I was using).
Comment 3 Anupama 2006-01-25 13:38:29 UTC
The Problem we are facing is because InputStraem.available() returns a maximum 
value of 2147483647.And for 2.5GB it crosses this.
Do we have any alternative for calculating the number of bytes for the 
Comment 4 dema rogatkin 2006-04-07 08:42:45 UTC
        at $Proxy0.getContentLength(Unknown Source)
        at org.apache.slide.webdav.method.PutMethod.executeRequest
        at org.apache.slide.webdav.method.AbstractWebdavMethod.run
        at org.apache.slide.webdav.WebdavServlet.service
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at rogatkin.web.WebAppServlet$SimpleFilterChain.doFilter
        at org.apache.slide.webdav.filter.LogFilter.doFilter
        at rogatkin.web.WebAppServlet$SimpleFilterChain.doFilter
        at rogatkin.web.WebAppServlet$WebAppContextFilter.doFilter
        at rogatkin.web.WebAppServlet$SimpleFilterChain.doFilter
        at rogatkin.web.WebAppServlet.service(WebAppServlet.java:402)
        at Acme.Serve.Serve$ServeConnection.runServlet(Serve.java:2011)
        at Acme.Serve.Serve$ServeConnection.parseRequest(Serve.java:1989)
        at Acme.Serve.Serve$ServeConnection.run(Serve.java:1827)
        at java.lang.Thread.run(Thread.java:620)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke
        at sun.reflect.DelegatingMethodAccessorImpl.invoke
        at java.lang.reflect.Method.invoke(Method.java:589)
        at rogatkin.web.WebAppServlet$WebAppContextFilter$1.invoke
        ... 15 more
Caused by: java.lang.NumberFormatException: For input string: "3447844432"
        at java.lang.NumberFormatException.forInputString
        at java.lang.Integer.parseInt(Integer.java:459)
        at java.lang.Integer.parseInt(Integer.java:497)
        at Acme.Serve.Serve$ServeConnection.getIntHeader(Serve.java:2499)
        at Acme.Serve.Serve$ServeConnection.getContentLength(Serve.java:2178)
        ... 20 more