Bug 57595 - Can't build due to the current build configuration
Summary: Can't build due to the current build configuration
Status: RESOLVED LATER
Alias: None
Product: POI
Classification: Unclassified
Component: POI Overall (show other bugs)
Version: 3.11-FINAL
Hardware: PC All
: P5 normal (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-18 20:23 UTC by Vasili
Modified: 2015-02-26 17:58 UTC (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vasili 2015-02-18 20:23:53 UTC
Hi,
I'm trying to build the sources.
BTW you have to create lib manually, otherwise you get an error immediately. This is minor, but probably you want to know this to add mkdir into build.xml

Then I'm getting Connection timed out due to maven.
It seems our organization does not allow direct connections (not from a browser): when I'm opening the same repository jar in a browser - I'm getting it.
So, I have to set a proxy. 
But the build system uses ant and I do not see a place - where could I set the proxy.

The ideal way would be placing all 3rd party libs into ./lib since we already know the exact versions of libraries: why download them every time? It is not a robust way to build - when everything depends on the Internet connections. I would prefer to have everything local not depending on the Internet. But this is ideally.
For now I'd like to build by any mean.

Thanks!
Comment 1 Vasili 2015-02-18 20:38:02 UTC
So, my question is: how to set the proxy for the maven requests in the current build system?

Thanks!
Comment 2 Nick Burch 2015-02-19 00:08:40 UTC
Can you try with a recent checkout from svn? There have been a few build fixes since 3.11 final was released

Also, if you download both the source and binary packages, then you should have all the jars pre-downloads for a rebuild, can you try that?
Comment 3 Vasili 2015-02-19 15:08:06 UTC
I just tried svn co and although I have set up a proxy - it does not work in our environment (either proxy has a problem or it's a problem with svn: I tried different svn clients including cygwin svn and TortoiseSVN). So, remote checkout does not work for me.
It would be more convenient if you could have conventional tarball or zip containing at least some snapshots downloadable. Since they seems to be not existent - I use src release distributions. 

So, the second option remaining. As you suggested, I've copied lib from bin distribution but it still does not contain all jars required, and I'm getting error again (hamcrest-core-1.3.jar is missing in lib and maven tries to download):

-----8<------
testexist:

downloadfile:
      [get] Getting: http://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.
3/hamcrest-core-1.3.jar
      [get] To: C:\poi-src-3.11-20141221\poi-3.11_fixed\lib\hamcrest-core-1.3.ja
r
      [get] Error getting http://repo1.maven.org/maven2/org/hamcrest/hamcrest-co
re/1.3/hamcrest-core-1.3.jar to C:\poi-src-3.11-20141221\poi-3.11_fixed\lib\hamc
rest-core-1.3.jar

BUILD FAILED
C:\poi-src-3.11-20141221\poi-3.11_fixed\build.xml:450: The following error occur
red while executing this line:
C:\poi-src-3.11-20141221\poi-3.11_fixed\build.xml:387: java.net.ConnectException
: Connection timed out: connect
        at java.net.DualStackPlainSocketImpl.connect0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketI
mpl.java:79)
-----8<------

Does POI have any binary distributions containing EVERYTHING necessary for it's building, so there will be no need to download automatically?
Just a tarball or zip.
Yes, I can download that particular jar directly (hamcrest) but as I remember, in the past I already tried to download all the dependencies manually, a dozen of files from different places, but got some missing OpenOffice file or something else (don't remember) which gave me a conflict or error.

What will you suggest?
The only restriction we have is the proxy, so maven, svn, wget or other automatic downloads do not work. What we need to build a source and all the dependencies should be here, locally. That's it. in the past (before maven it was so simple! Even with the plain ant, not speaking about pre-ant times, i.e. make, how the Opensource had been distributed 90s). Sorry for complaining here, I'm just tired of inconveniences introduced by the technologies themselves for the sake of technology, especially when it is not absolutely necessary and making software building procedures less portable. In my belief, the OpenSource software, to be successful, should be buildable under any field conditions, without internet connection, even in field conditions. So, everybody will be able to build it - not only the experts.

Thank you for your support!
Comment 4 Vasili 2015-02-19 15:20:05 UTC
Sorry for the typos above and for my grumbling again. I'm complaining not about the POI build system - my complaints are about today's standards in software development which became (in my opinion) much lower nowadays, and Opensource is not an exclusion. I really want it to be winning, but for that the build from sources must be really easy, such as "make" (with >95% success rate) or "./configure; make make install".
My experience with modern software and today's standards gives me opinion that only 5% are buildable completely without problems, and I blame frameworks and so many dependencies and maven for that.

Sorry again.
Comment 5 Vasili 2015-02-19 16:07:15 UTC
An update:

I was able to get a snapshot from http git.

It does not contain jars too, so the build failed with the same problem as above.

My suggestion is placing 3rd party jars right into the src distributions under ./lib.

Any particular version usually depends on a particular fixed version of a 3rd party library, so there is no reason to download it, it is known, and it is only more robust for the product - to be independent on internet and on somebody else, not speaking about potential versioning problems due to human mistakes on the 3rd party side. As more dependencies, as more complex systems, and the probability of failure in complex systems grows exponentially. So, having everything locally provided, even after 20 years the project will be compilable, having everything what it needs inside itself. I'm speaking about the sources distributions.
Comment 6 Nick Burch 2015-02-19 16:51:45 UTC
I'm not sure we want to be putting jars into SVN, as most people will just download them on demand, and it'd bulk out the SVN set and might annoy the Apache Infra folks

Maybe one answer is to provide a "build" or "deps" package as well, which contains the build-time dependencis?

You should be able to checkout from SVN despite a proxy, see the SVN FAQ entry for how to do it: http://subversion.apache.org/faq.html#proxy 

Apache Ant also supports HTTP proxies, as described in their docs at https://ant.apache.org/manual/proxy.html - what happens if you try setting your proxy details to Ant before you try running a clean build?
Comment 7 Vasili 2015-02-19 17:14:05 UTC
Yes, providing "deps" package will obviously help.

Thanks for the proxy tips! I will try them
Comment 8 Vasili 2015-02-19 17:17:31 UTC
I'm wondering, why anybody should be annoyed by binaries in svn?
I was always keeping a lot of binaries in cvs, including the project files, and never had any problems. It acts as a reference build implementation, and developers who want to overwrite the default - do it on their machines.
svn should work well with binaries.
The benefit - independence and robustness of the build.
Comment 9 Nick Burch 2015-02-19 17:21:46 UTC
We download files from Maven Central, which will provide a consistent build as those files are known and won't change. If you were building a similar project using Maven rather than Ant, it'd be downloading jars like these for you as well

If you could test setting the proxy, that'd be great
Comment 10 Vasili 2015-02-19 18:56:39 UTC
I've set the proxy as global env var, and it is passed to JVM as -D parameter (I can see it) but it seems we have a problem with the proxy in our organization and resolving it (may be I have a wrong IP/port or it's the proxy issue) may take time. So, I will likely be testing this later, when it will be demonstrated that the proxy works.

That is why I hate Maven. It depends on the remote connections and remote connection is one of the weakest links. In 20 years domains will be likely changed, links will be broken, policies will change, organizations will change owners with changing the priorities. Only if we have everything locally, the programs depend on, up to OS, I can somehow guarantee the reliability of my product to my clients. So, I never use maven, I use ant only for java projects.

I'll be back with the tests later.

Thanks for your support!
Comment 11 Vasili 2015-02-19 20:11:13 UTC
Yes, that was the proxy problem, I just received new IP and proxy works.

Now I'm getting POI-specific dependencies errors (can't find bcprov):


    [unzip] Expanding: C:\poi-src-3.11-20141221\poi-3.11\lib\jacoco-0.7.1.201405
082137.zip into C:\poi-src-3.11-20141221\poi-3.11

testexist:

downloadfile:
      [get] Getting: http://repo1.maven.org/maven2/org/apache/rat/apache-rat/0.1
1/apache-rat-0.11.jar
      [get] To: C:\poi-src-3.11-20141221\poi-3.11\lib\apache-rat-0.11.jar

testexist:

downloadfile:
      [get] Getting: http://repo1.maven.org/maven2/org/bouncycastle/bcprov-ext-j
dk15on/1.51/bcprov-ext-jdk15on-1.51.jar
      [get] To: C:\poi-src-3.11-20141221\poi-3.11\compile-lib\bcprov-ext-jdk15on
-1.51.jar
      [get] Error getting http://repo1.maven.org/maven2/org/bouncycastle/bcprov-
ext-jdk15on/1.51/bcprov-ext-jdk15on-1.51.jar to C:\poi-src-3.11-20141221\poi-3.1
1\compile-lib\bcprov-ext-jdk15on-1.51.jar

BUILD FAILED
C:\poi-src-3.11-20141221\poi-3.11\build.xml:479: The following error occurred wh
ile executing this line:
C:\poi-src-3.11-20141221\poi-3.11\build.xml:387: java.io.FileNotFoundException:
C:\poi-src-3.11-20141221\poi-3.11\compile-lib\bcprov-ext-jdk15on-1.51.jar (The s
ystem cannot find the path specified)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
        at org.apache.tools.ant.taskdefs.Get$GetThread.downloadFile(Get.java:739
)
        at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:586)
        at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)


P.S. BTW why using bcprov when today built-in into JDK javax.crypto package gives everything (in imho better shape and performance) out-of-the-box including AES256?
Comment 12 Vasili 2015-02-19 20:22:25 UTC
And now using the jit snapshot (the same problem with bouncycastle):


downloadfile:
      [get] Getting: http://repo1.maven.org/maven2/org/apache/rat/apache-rat/0.1
1/apache-rat-0.11.jar
      [get] To: C:\poi-src\git\poi\lib\apache-rat-0.11.jar

testexist:

downloadfile:
      [get] Getting: http://repo1.maven.org/maven2/org/bouncycastle/bcprov-ext-j
dk15on/1.51/bcprov-ext-jdk15on-1.51.jar
      [get] To: C:\poi-src\git\poi\compile-lib\bcprov-ext-jdk15on-1.51.jar
      [get] Error getting http://repo1.maven.org/maven2/org/bouncycastle/bcprov-
ext-jdk15on/1.51/bcprov-ext-jdk15on-1.51.jar to C:\poi-src\git\poi\compile-lib\b
cprov-ext-jdk15on-1.51.jar

BUILD FAILED
C:\poi-src\git\poi\build.xml:495: The following error occurred while executing t
his line:
C:\poi-src\git\poi\build.xml:398: java.io.FileNotFoundException: C:\poi-src\git\
poi\compile-lib\bcprov-ext-jdk15on-1.51.jar (The system cannot find the path spe
cified)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
        at org.apache.tools.ant.taskdefs.Get$GetThread.downloadFile(Get.java:739
)
        at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:586)
        at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
Comment 13 Vasili 2015-02-19 20:24:04 UTC
This is what I meant by not robustness:

The URL hit returns:

404 Not Found

What will be in 20 years?
Comment 14 Vasili 2015-02-19 22:04:47 UTC
Hi Nick,

The last post here.
Since you asked me to test svn build, let me post the results here (and sorry for 'spamming'):

I was not able to checkout from svn, this is again our proxy issue:

---------------8<------------------
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/POILogger.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/IOUtils.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/LittleEndianInput.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/LittleEndianByteArrayOu
tputStream.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/LittleEndianOutput.java

A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/IntegerField.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/ShortField.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/StringUtil.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/NullLogger.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/CommonsLogger.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/LittleEndian.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/HexRead.java
A    poi/tags/REL_3_5_BETA6/src/java/org/apache/poi/util/POILogFactory.java
svn: E215004: Proxy authentication failed
---------------8<------------------

As the result, I'm not able to test svn build.


Regards and thank you for the support
Vasili
Comment 15 Nick Burch 2015-02-21 12:06:36 UTC
If you can't even manage a svn checkout through your proxy, then you have much bigger problems than we can help with. You'll need to go have strong words with your IT team about providing a working, bit-clean proxy.

Once you have a working and reliable (possibly proxied) connection to the internet, you should then be fine with the checkout and build. We provide the binary package for people who want to get everything they need in one go to run, we provide the source package to let people get the source to look at, but we require a working internet connection at least once to rebuild.

(A deps package might be possible, but I'm not sure how much it'd help, especially as your proxy looks so broken I doubt it'd reliably let you download it. Maven central is so widely mirrored that it should be around for pretty much forever)
Comment 16 Vasili 2015-02-23 15:01:13 UTC
Yes, I finally succeeded with svn checkout. Proxy is a headache and I do not control it, this is the reality. We have to live in such reality, including with the unreliable Internet.

Yes, it would be so much better and easier - to have a tarball or zip with all the deps, how it was usually done in 90s since it is always possible to guarantee download and verification of one binary file: almost any paranoid organization have means to download or ftp a single binary file.
BTW security may and will require verifying of the complete build's authenticity  by means of a signature, while automatic download from 3rd parties through maven (or getting from svn) leaves that hole opened.
I mean, providing of cryptographic hash/signature may be even a requirement in some organizations. 
So, I would suggest to provide a self-sufficient package for build - at least as an option.

Thanks!