Issue 121837 - CWS hsqldb19: Update hsqldb to version 2.x
Summary: CWS hsqldb19: Update hsqldb to version 2.x
Status: CONFIRMED
Alias: None
Product: Base
Classification: Application
Component: code (show other issues)
Version: 4.0.0-dev
Hardware: All All
: P3 Normal with 5 votes (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
: 126309 (view as issue list)
Depends on:
Blocks:
 
Reported: 2013-02-28 15:45 UTC by Pedro Giffuni
Modified: 2015-05-17 11:08 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Patches from the hsqldb cws (copyrighted Oracle) (30.32 KB, patch)
2013-02-28 15:45 UTC, Pedro Giffuni
no flags Details | Diff
Permit newer hsqldb versions in configure (1.17 KB, patch)
2013-02-28 15:53 UTC, Pedro Giffuni
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description Pedro Giffuni 2013-02-28 15:45:41 UTC
Created attachment 80371 [details]
Patches from the hsqldb cws (copyrighted Oracle)

I had a try at merging the hsqldb19 cws into AOO.
http://hg.services.openoffice.org/cws/hsqldb19/

Unfortunately my attempt was unsuccessful and the build breaks in
the connectivity module.

The original repository is somewhat of a mess but the changes
there are valuable enough to consider doing some work on this.
Comment 1 Pedro Giffuni 2013-02-28 15:53:13 UTC
Created attachment 80372 [details]
Permit newer hsqldb versions in configure

configure.in specifically prohibits newer versions of hsqldb from being used.
This patch restores the behaviour of having newer versions.

The hsqldb19 CWS would make it possible to use 2.2.x versions.

hsqldb 2.3 (not yet released) will include backwards compatibility with 1.8.0.9, so even if the CWS never sees the light, this patch in configure.in should be useful.
Comment 2 Kay 2013-03-29 18:57:40 UTC
Is there some reason we shouldn't at least apply the configure (configure.in) patch?
Comment 3 Pedro Giffuni 2013-03-30 02:53:30 UTC
(In reply to comment #2)
> Is there some reason we shouldn't at least apply the configure
> (configure.in) patch?

I would suggest waiting until hsqldb 2.3 is released.

This said, please note that both patches are untested.
(Someone used to Mercurial may fix the CWS stuff, and
perhaps its even easy, though.)
Comment 4 Kay 2013-03-30 16:46:59 UTC
OK, I basically did something similar in my recent testing with 2.2.9, though I just commented out lines and my final configure script. Nothing as elegant as what you've done to configure.in.

Right now, we REALLY need to get on with testing of the most recent version of hsql to see if it helps with end user java 7+ problems, so this is why I suggest at least committing the configure.in stuff to save others work for doing this .

So, I did commit this to my local configure.in though I have not rebuilt...yet.

I'll wait to see a few days to see what others say.

At least in the US, users have basically been told to disable java in their browsers due to security issues:

http://www.nbcnews.com/technology/technolog/homeland-security-still-says-no-java-1B8000547

Yes, this JUST applies to disabling it in your browser, but, it's tough to say how people will interpret this. They may think they should completely de-install ALL of java. Now Oracle is up to java 1.7, ver 17. Folks certainly, in any case, will NOT be installing older versions.
Comment 5 Pedro Giffuni 2013-03-30 18:30:41 UTC
(In reply to comment #4)
> OK, I basically did something similar in my recent testing with 2.2.9,
> though I just commented out lines and my final configure script. Nothing as
> elegant as what you've done to configure.in.
> 

Without the CWS, this change would basically break all existing databases.
I dont really care but some people in the community is known for vetoing much smaller changes :-P.

> Right now, we REALLY need to get on with testing of the most recent version
> of hsql to see if it helps with end user java 7+ problems, so this is why I
> suggest at least committing the configure.in stuff to save others work for
> doing this .
> 
I think thats beibg handled in another BZ issue.
Comment 6 Ariel Constenla-Haile 2013-03-30 18:42:20 UTC
Updating HSQLDB seems the kind of thing that has to be done in a separated branch, as it is not simply dumping the new zip file, make it compile, and then run without crashing; it has to work without regressions.

Besides, patches from a 2 years old CWS, from a developer that used to work full time for OpenOffice, but no longer works, may require the same degree of expertise to understand (I assume that applying patches is not simply running patch and building to see it does not break: the one applying the patches must understand the code).

In short, IMHO, until we find someone that knows the Base code, patches shouldn't be applied, at least not in trunk.
Comment 7 Pedro Giffuni 2013-03-30 19:05:17 UTC
(In reply to comment #6)
> Updating HSQLDB seems the kind of thing that has to be done in a separated
> branch, as it is not simply dumping the new zip file, make it compile, and
> then run without crashing; it has to work without regressions.
> 
> Besides, patches from a 2 years old CWS, from a developer that used to work
> full time for OpenOffice, but no longer works, may require the same degree
> of expertise to understand (I assume that applying patches is not simply
> running patch and building to see it does not break: the one applying the
> patches must understand the code).
> 

I dont think anyone is considering that. AFAICT no one is even working on the code.

> In short, IMHO, until we find someone that knows the Base code, patches
> shouldn't be applied, at least not in trunk.

Kay's proposal is to permit newer hsqldb versions in configure. The check is there for a good reason though so I advise against doing it until either the CWS merge is ready or hsqldb 2.3 is released.
Comment 8 Fred Toussi 2013-03-31 12:56:31 UTC
Hi All, Iam adding some short comments here on the CWS and current HSQLDB development.

I will support AOO developers and QA testers with integration of new HSQLDB versions. I have looked at the AOO code and know to some extent what it is doing. It needs at least one, but preferably two or three developers to work on the AOO side. Pedro has made a good start and he should be encouraged to continue with the help of others.

I was working on the HSQLDB side while Ocke Janssen was developing the CWS and OOo 3.4 in parallel. We were in regular contact. In April 2011, the final version of CWS was ready to be released with OOo 3.4, but (I think due to a push to get the 3.4 milestone release out) it was postponed to 3.5.

We released HSQLDB 2.1 specifically to have an official release to be integrated in OOo. Patches to HSQLDB were included in CWS and applied to 2.1.0 before 2.2.0 came out with those changes integrated. Therefore the CWS sync point would be HSQLDB 2.2.0 and later (but excluding 2.2.9 which had some reported issues). You can simply place the HSQLDB 2.x jar in the jars directory and it works. No need to compile the jar. See this thread for some ongoing comments from testers.

https://sourceforge.net/projects/hsqldb/forums/forum/73674/topic/5485246

Exactly because Base has not changed much from the OOo code, and the fact that CWS and the inherited OOo code were developed in synch, will make it easy to move to CWS. There are improvements in CWS which are possible only with HSQLDB 2.x.

As Ariel commented, this change should happen in a branch and tested by QA volunteers before integration.

The main technical difference between pre and post-CWS is the use of a different Java interface org.hsqldb.persist.RandomAccessInterface instead of org.hsqldb.lib.Storage. We developed this change to allow Base to make more intelligent choices about when to persist the changed data. Other differeces are due to improved SQL support in 2.x.

Version 2.3.0 is gradually getting ready for release. We have released snapshot jars of this version, which are available here:

http://www.hsqldb.org/repos/org/hsqldb/hsqldb/SNAPSHOT/

Since last year, I have tested the 2.3.0 code with LO and AOO code and improved the code. The last test was with the snapshot hsqldb-20130206.203335-33.jar. These tests are primarily to ensure interoperability with "mixed-mode" external file databases. I have not tested with CWS myself, but will start to do so in the future.

Because the new HSQLDB has better logging options and the fact that persistence can be controlled better in OO with CWS with the new Java interface, it should be possible to avoid the ocassional crashes of the embedded database that users have complained about. One possible option is to warn the user when the maximum size of the database exceeds a limit. At the same time, this integration providss a easy way to use large mixed-mode external file databases with backup capabilities.  

After CWS integration, you can open the old databases in readonly mode (no alteration) or upgrade them to the new version. Some people have said they want to be able to modify the data in an old database without upgrading it. I intend to support this capability in a newer version after the release of 2.3.0.

I would suggest to use the forum.openoffice.org to exchange information about the branch development, with separate threads used by developers and QA people to discuss issues. As the forum posts are editable, it is a better place for this task than Bugzilla.
Comment 9 Kay 2013-03-31 21:42:03 UTC
First, re Fred's last comment, YAY! Better logging would be a big PLUS! HUGE as a matter of fact.

Now, my original suggestion. Although applying Pedro's original patch will not solve all problems relating to current implementation, it will at least do no harm to current use of HSQL 1.8.x and will save experimenters having to muck with the section of configure that applies to this.
Comment 10 Fred Toussi 2013-03-31 22:22:31 UTC
(In reply to comment #9)
> applying Pedro's original patch ... will save experimenters having
> to muck with the section of configure that applies to this.

There is a hack that can be uncommented in HSQLDB 2.3.0 code to pretend it is version 1.8.0 and allow experimenting. I could release a "hacked" version with the next snapshot. But because there are subtle dependencies in the current AOO code on HSQLDB 1.8.0's persistence, it doesn't work properly with 2.x which persists somewhat differently. There is a chance that 2.3.0 can be made better to emulate 1.8.0 persistence, but the better option is CWS. I must emphasize that this was developed over two years and was in good working order and ready for release for the last several months.
Comment 11 Kay 2013-04-01 16:48:08 UTC
@Fred's comment of 31-03-2013

Regarding CWSes,

The last reasonable discussion we had on this was:

http://markmail.org/message/n7lqij6rymm3k2fc

So, we can not think about using a CWS approach.

Ok, based on your comment.

"There is a chance that 2.3.0 can be made better to emulate 1.8.0 persistence, but the better option is CWS. I must emphasize that this was developed over two years and was in good working order and ready for release for the last several months."

I commend the amount of development that's already been put to this. And, it's good to hear it seems "ready for release for the last several months". If that's the case, why aren't we just using it?

...with the understanding that we will need to make changes on the AOO side.

Any help gratefully appreciated of course.

At any rate, I will grab your latest snapshot and go from there.

Thanks for all your dedication.
Comment 12 Fred Toussi 2013-04-01 17:10:37 UTC
> ...with the understanding that we will need to make changes on the AOO side.

Let me reword what I said to avoid misunderstanding:

There is a chance that 2.3.0 can be made better to emulate 1.8.0 persistence for use with the current AOO code base, but this is not a great option.

"The better option is 2.3.0 toegther with CWS. I must emphasize the CWS was developed over two years and was in good working order and ready for release for the last several months before OOo development stopped."

As for HSQLDB 2.3.0, yes, it is ready for release. If any regression from 2.2.0 comes up in integrating with CWS, it would be very easy to fix. On the other hand, if there are issues integrating with current AOO code, the fix would be soemwhat more difficult.

As for CWS development, the post you pointed at is nearly 2 years old, when code was being imported. There must be a way, such as a repository branch, to allow developers to work together on a major new feature.
Comment 13 Pedro Giffuni 2013-04-01 18:13:45 UTC
(In reply to comment #11)
> @Fred's comment of 31-03-2013
> 
> Regarding CWSes,
> 
> The last reasonable discussion we had on this was:
> 
> http://markmail.org/message/n7lqij6rymm3k2fc
> 
> So, we can not think about using a CWS approach.
> 

Let me make this clear:

I created this BZ issue exclusively for people that want to get *some* start at rescuing the hsqldb19 CWS: yes ideally someone should start a new branch but for the time being I am just thought I would save the patch before it risked dying in my HD.

Other topics should be taken to dev list.
Comment 14 oooforum (fr) 2013-06-11 09:46:25 UTC
Try with last AOO build rev. 1489073
HSQL is still in 1.8.0.10
Comment 15 Kay 2013-06-11 23:42:32 UTC
This is true. The old CWS patches were attempted but the build was not successful. We will readdress this situation after AOO 4.0. So, nothing has changed with hsqldb since 3.4.1.
Comment 16 oooforum (fr) 2015-05-17 11:08:56 UTC
*** Issue 126309 has been marked as a duplicate of this issue. ***