Server crash on changeset problem (mismatched apply)

[Gared]

My server crashes nearly daily with the following error message:

[2013-12-16 22:40:37.845] [ERROR] console - Error: Failed assertion: mismatched apply: 46262 / 46261
    at Object.error (/home/eplite/etherpad-lite/src/static/js/Changeset.js:39:11)
    at Object.assert (/home/eplite/etherpad-lite/src/static/js/Changeset.js:53:13)
    at Object.exports.applyToText (/home/eplite/etherpad-lite/src/static/js/Changeset.js:907:11)
    at Object.exports.applyToAText (/home/eplite/etherpad-lite/src/static/js/Changeset.js:1599:19)
    at Pad.getInternalRevisionAText (/home/eplite/etherpad-lite/src/node/db/Pad.js:206:27)
    at async.series.results (/home/eplite/etherpad-lite/src/node_modules/async/lib/async.js:486:21)
    at _asyncMap (/home/eplite/etherpad-lite/src/node_modules/async/lib/async.js:185:13)
    at async.forEachSeries.iterate (/home/eplite/etherpad-lite/src/node_modules/async/lib/async.js:108:13)
    at async.forEachSeries.iterate (/home/eplite/etherpad-lite/src/node_modules/async/lib/async.js:119:25)
    at _asyncMap (/home/eplite/etherpad-lite/src/node_modules/async/lib/async.js:187:17)
RESTART!

The changeset versions of the mismatched apply differs. So it seems there are differnent pads causing this error.

Etherpad is on latest develop: 7d47d91

[marcelklehr]

Mmmh. I thought I had wrapped all critical spots where this might occur
with a try-catch..

[marcelklehr]

occurrences of a call to getInternalRevisionAText:

src/node/db/API.js:257:      pad.getInternalRevisionAText(rev, function(err, atext)
src/node/handler/PadMessageHandler.js:1449:        pad.getInternalRevisionAText(revNum, function(err, _atext)
src/node/utils/ExportDokuWiki.js:34:      pad.getInternalRevisionAText(revNum, function (err, revisionAtext)
src/node/utils/ExportHelper.js:29:  var atext = ((revNum !== undefined) ? pad.getInternalRevisionAText(revNum) : pad.atext());
src/node/utils/ExportHtml.js:39:      pad.getInternalRevisionAText(revNum, function (err, revisionAtext)
src/node/utils/ExportTxt.js:43:      pad.getInternalRevisionAText(revNum, function (err, revisionAtext)
src/node/utils/padDiff.js:75:  this._pad.getInternalRevisionAText(rev, function(err, atext){
src/node/utils/padDiff.js:93:  this._pad.getInternalRevisionAText(rev, function(err, atext){

This is either
triggered by a CHANGESET_REQ message (used in timeslider).
or triggered by an export / API:getText request
or triggered by a call to padDiff.

Handling the error should be easy, the question is why it errors. Maybe you can investigate, and find out whether the pads in question are corrupt or intact for starters.

[Gared]

@marcelklehr Is there an easy way on how I know on which padId the error happens?

[marcelklehr]

Might go away with #2038

[marcelklehr]

You'd need to add a few things to teh codez. I'll wrap up a pull request later that will prevent it from crashing...

[marcelklehr]

@Gared with that branch (#2039) it won't crash anymore, if this happens. (+ It'll display a nice log entry for this event including padID and error message and all ;) )

[Gared]

@marcelklehr thanks. After I added your changes to my etherpad it doesn't crash on this error anymore (it crashed on [RangeError: Maximum call stack size exceeded], but probably this is another issue).
I have checked my logs and found the pad that is causing the error. With the checkPad.js I got the following errors for the pad:

[2013-12-19 11:16:27.093] [ERROR] console - Bad changeset at revision 301 - Failed assertion: mismatched apply: 41618 / 41617
[2013-12-19 11:16:27.201] [ERROR] console - Bad changeset at revision 401 - Failed assertion: mismatched apply: 41786 / 41785
[2013-12-19 11:16:27.946] [ERROR] console - Bad changeset at revision 1101 - Failed assertion: mismatched apply: 44373 / 44372
[2013-12-19 11:16:30.524] [ERROR] console - Bad changeset at revision 2901 - Failed assertion: mismatched apply: 46136 / 46135
[2013-12-19 11:16:32.625] [ERROR] console - Bad changeset at revision 4101 - Failed assertion: mismatched apply: 47600 / 47599
[2013-12-19 11:16:32.704] [ERROR] console - Bad changeset at revision 4201 - Failed assertion: mismatched apply: 47953 / 47952
[2013-12-19 11:16:57.210] [ERROR] console - Bad changeset at revision 12701 - Failed assertion: mismatched apply: 57613 / 57612
[2013-12-19 11:17:04.905] [ERROR] console - Bad changeset at revision 15301 - Failed assertion: mismatched apply: 48253 / 48252

I think there is no way to fix the wrong changesets, but as long as this error doesn't crash my server again I'm happy with it.