Bug 62773 - Change DeltaManager to handle session deserialization problems more robustly
Summary: Change DeltaManager to handle session deserialization problems more robustly
Status: NEW
Alias: None
Product: Tomcat 9
Classification: Unclassified
Component: Cluster (show other bugs)
Version: 9.0.12
Hardware: PC Linux
: P2 enhancement (vote)
Target Milestone: -----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-28 19:02 UTC by Mitch Claborn
Modified: 2018-10-23 22:55 UTC (History)
1 user (show)



Attachments
patch that implements the described changes (20.19 KB, patch)
2018-10-23 22:55 UTC, Mitch Claborn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mitch Claborn 2018-09-28 19:02:12 UTC
See discussion at http://mail-archives.apache.org/mod_mbox/tomcat-dev/201809.mbox/%3C694b5737-fb87-a976-440f-2ed0719c1e1a%40claborn.net%3E

Problems:
1. When processing a received "all sessions" message, and there is an Exception when deserializing a session, there is no logging of what session field or attribute caused the error.  This makes it very difficult for the application developer to diagnose and correct problems in the application that caused the error. 

2. When such an exception occurs, the session in error and all remaining sessions in that chunk (sendAllSessionsSize) are discarded by the receiving node. There would be significantly less user impact if only the session in error was discarded.  

I am working on a patch that will address these issues in DeltaManager. Changes in that patch:

1. Update
   org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[])
   and
   org.apache.catalina.ha.session.DeltaSession.doReadObject(ObjectInput)
   to produce a more detailed error message when a session is in
   error.  New error message includes: the session index in the list of
   sessions, the session ID, the last field or attribute that was
   attempted to be read.
2. Introduce new XML attribute verifySerializedSessions for DeltaManager with a default value of false (since the true setting will result in significant overhead when sending all sessions). 
3. If verifySerializedSessions="true",
   org.apache.catalina.ha.session.DeltaManager.serializeSessions(Session[])
   will first serialize each session then immediately deserialize it.
   If all is good, send the session as usual.  If any errors are
   encountered, create and send a dummy session with a known session ID
   instead. (This keeps the session count, which has already been put
   in the output stream, correct for the receiving node.)
4. Update
   org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[])
   to discard any received session that has the known dummy session ID.

I'll will attach the patch here when it is ready.
Comment 1 Mitch Claborn 2018-10-23 22:55:25 UTC
Created attachment 36208 [details]
patch that implements the described changes

This code has been running in our moderately busy website for several weeks with no problems.