Bug 42496 - concurrence of editors causes failing of document creation.
Summary: concurrence of editors causes failing of document creation.
Status: NEW
Alias: None
Product: Lenya
Classification: Unclassified
Component: Site Management (show other bugs)
Version: Trunk
Hardware: Other other
: P2 regression
Target Milestone: 2.0.1
Assignee: Lenya Developers
URL:
Keywords:
: 43136 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-05-23 08:14 UTC by Thorsten Scherler
Modified: 2007-10-18 12:34 UTC (History)
1 user (show)



Attachments
lenyaJMeterTest (41.26 KB, application/octet-stream)
2007-05-23 08:14 UTC, Thorsten Scherler
Details
Log for the jmeter test run (205.04 KB, text/x-log)
2007-05-23 08:15 UTC, Thorsten Scherler
Details
jmeter user (13 bytes, text/csv)
2007-05-23 08:16 UTC, Thorsten Scherler
Details
jmeter user admin 1 to 5 (passwd: levi) (1011 bytes, application/x-gzip)
2007-05-23 08:57 UTC, Thorsten Scherler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thorsten Scherler 2007-05-23 08:14:15 UTC
Javier Puerto has done some JMeter tests against the default pub.

Bottom line is everything is fine if only one user is creating content, as soon
there is another user creating documents lenya is failing. Meaning we have a
problem with concurrence.
Comment 1 Thorsten Scherler 2007-05-23 08:14:59 UTC
Created attachment 20247 [details]
lenyaJMeterTest
Comment 2 Thorsten Scherler 2007-05-23 08:15:38 UTC
Created attachment 20248 [details]
Log for the jmeter test run
Comment 3 Thorsten Scherler 2007-05-23 08:16:08 UTC
Created attachment 20249 [details]
jmeter user
Comment 4 Andreas Hartmann 2007-05-23 08:22:26 UTC
Can we add the test files to modules/development/tests/jmeter/?
Comment 5 Andreas Hartmann 2007-05-23 08:36:47 UTC
The log contains an error about a particular entity in an i18n catalogue:

Caused by: org.xml.sax.SAXParseException: The entity "iexcl" was referenced, but
not declared.
Comment 6 Andreas Hartmann 2007-05-23 08:39:34 UTC
Would it be possible to provide an English version of the test case? TIA!
Comment 7 Andreas Hartmann 2007-05-23 08:53:32 UTC
Am I right that one has to create the users admin1 and admin2?
Comment 8 Thorsten Scherler 2007-05-23 08:55:42 UTC
Yes you need to create both user. Will attach the access files.
Comment 9 Thorsten Scherler 2007-05-23 08:57:49 UTC
Created attachment 20250 [details]
jmeter user admin 1 to 5 (passwd: levi)
Comment 10 Andreas Hartmann 2007-05-24 02:48:21 UTC
It looks like the sitetree is modified without locking its repository node ...
Comment 11 Andreas Hartmann 2007-05-24 02:55:36 UTC
Well, I don't see much we can do about this besides making the usecases in
question pessimistic.
Comment 12 Andreas Hartmann 2007-05-24 05:52:05 UTC
Setting the create usecase to pessimistic avoids the exception. Of course the
documents aren't created anymore.
Comment 13 Thorsten Scherler 2007-05-24 05:59:13 UTC
(In reply to comment #12)
> Setting the create usecase to pessimistic avoids the exception. Of course the
> documents aren't created anymore.

jeje.

Javier and I talked about it and a possible fix is to introduce a cue. 
admin 1 and 2 are trying to create content at the time. Instead directly
allowing it, each creation goes into a cue and lenya will process it one after
the other.

BTW Daisy is the only CMS that did not made any problems with concurrence. I
still have not found enough time to look in their repository implementation, but
I think it is worthwhile.
Comment 14 Andreas Hartmann 2007-05-24 06:08:21 UTC
(In reply to comment #13)

> Javier and I talked about it and a possible fix is to introduce a cue. 
> admin 1 and 2 are trying to create content at the time. Instead directly
> allowing it, each creation goes into a cue and lenya will process it one after
> the other.

The problem with queues is that they're not suitable for interactive scenarios
when the same resources are affected. Imagine that two people would like to
create a document with the same URL. When using a queue, you have no chance to
give feedback to the user. IMO pessimistic offline lock is more suitable in this
situation.
Comment 15 Andreas Hartmann 2007-05-24 06:48:26 UTC
With the pessimistic usecases, I still get two types of exceptions when running
the app in the debugger:

- indexer is busy (we have to avoid this)
- document not referenced in site structure (apparently inconsistent data)
Comment 16 Thorsten Scherler 2007-05-24 07:07:50 UTC
(In reply to comment #14)
...
> The problem with queues is that they're not suitable for interactive scenarios
> when the same resources are affected. Imagine that two people would like to
> create a document with the same URL. When using a queue, you have no chance to
> give feedback to the user. IMO pessimistic offline lock is more suitable in this
> situation.

Actually there is a chance but it costs a call more. Before sending to the cue
the cue needs to be validated that the same URL is not already in it. If not in
the cue the user gets a message that his document is in the cue. If it is in the
cue then she gets the message to choose another name.

The only thing that needs changes is the forward after creation. I think a nice
ajax dialog would be nice, like telling the user to stand by or do other stuff
meanwhile. Using notifications as soon the new doc is out of the cue the user
gets feedback.

Comment 17 Andreas Hartmann 2007-05-24 07:30:55 UTC
(In reply to comment #16)

You mean you'd combine a queue (asynchronous) with request-based user
interaction (synchronous)? I'm not quite sure how this should be implemented in
a reasonable way - would you add a loop to the usecase which waits for the
queued task to be executed?
Comment 18 Thorsten Scherler 2007-05-24 08:14:45 UTC
(In reply to comment #17)
> You mean you'd combine a queue (asynchronous) with request-based user
> interaction (synchronous)? I'm not quite sure how this should be implemented in
> a reasonable way - would you add a loop to the usecase which waits for the
> queued task to be executed?

The "flow" would be:
a) user requests ...?lenya.usecase=sitemanagement.create and submits
b) doCheckExecutionConditions() checks whether the url is in the cue, if not it
locks it in the cue and we hit doExecute() (-> go on with c)). If in the cue
addErrorMessage("path-already-exists", params);
c) Instead doing setDefaultTargetURL(document.getCanonicalWebappURL()); like
right now we can send him to an ajax based cue overview (which gets updated in
real time from the cue worker). In combination (or alternatively) we can use a
checkbox to request a notification from the worker as soon he has done the document.

The cue must be serializable and further keep state of the operation (log
error/success). After a cue item is finished we write to the sitetree.

To answer: "would you add a loop to the usecase which waits for the queued task
to be executed" No, since the execution is to put it in the cue. The only
downside is that the user is not be able to see directly the new page.
Comment 19 Andreas Hartmann 2007-05-24 08:17:57 UTC
When setting the create usecase to pessimistic and applying the patch 20269 for
bug 42510, I can run the test with a clean log. Some of the documents aren't
created, though (when the sitetree repo node is locked by another thread).
Comment 20 Andreas Hartmann 2007-05-24 08:22:07 UTC
(In reply to comment #18)
> (In reply to comment #17)
> > You mean you'd combine a queue (asynchronous) with request-based user
> > interaction (synchronous)? I'm not quite sure how this should be implemented in
> > a reasonable way - would you add a loop to the usecase which waits for the
> > queued task to be executed?
> 
> The "flow" would be:
> a) user requests ...?lenya.usecase=sitemanagement.create and submits
> b) doCheckExecutionConditions() checks whether the url is in the cue, if not it
> locks it in the cue and we hit doExecute() (-> go on with c)). If in the cue
> addErrorMessage("path-already-exists", params);
> c) Instead doing setDefaultTargetURL(document.getCanonicalWebappURL()); like
> right now we can send him to an ajax based cue overview (which gets updated in
> real time from the cue worker). In combination (or alternatively) we can use a
> checkbox to request a notification from the worker as soon he has done the
document.

OK, I see what you mean.

> The cue must be serializable and further keep state of the operation (log
> error/success). After a cue item is finished we write to the sitetree.

That would mean we'd have to implement a full featured command pattern and a
persistent queue, which is serious business. Certainly nothing for 1.4, I'd even
say we should postpone it to 1.5 or later.
Comment 21 Andreas Hartmann 2007-05-24 08:25:15 UTC
I'd suggest we display a sane error message instead of the "repository node was
changed" exception and add an entry to the FAQ, advicing to set the usecase to
pessimistic to avoid the error. Any changes to the command execution can be
discussed in a different issue or mailing list thread. WDYT?
Comment 22 Thorsten Scherler 2007-05-24 08:40:04 UTC
Why do not set it by default to pessimistic?

I agree with your comments, display a message that explains that the doc could
not create out of high traffic (something in this direction) sounds good.

Yeah, regarding command execution that should go in a bug after discussing it on
dev.

Nice working with you. :)
Comment 23 Andreas Hartmann 2007-05-24 08:49:41 UTC
(In reply to comment #22)
> Why do not set it by default to pessimistic?

The problem with pessimistic usecases is that they create persistent locks,
which lead to the infamous "checked out by another user" issue. This makes sense
for editing (because the losing the changes is expensive), but IMO for creating
docs optimistic behaviour is sufficient. But fortunately it's is up to the
integrator to decide that. Maybe you'd like to raise the question on the dev or
user list.

> I agree with your comments, display a message that explains that the doc could
> not create out of high traffic (something in this direction) sounds good.

OK, I'll try to take care of this.

> Yeah, regarding command execution that should go in a bug after discussing it on
> dev.
> 
> Nice working with you. :)

With you too :)

BTW, the test case is great, it also brought the "indexer is busy" issue up.
Thanks a lot to Javier!

Comment 24 Richard Frovarp 2007-10-18 12:34:22 UTC
*** Bug 43136 has been marked as a duplicate of this bug. ***