[BUG] Uncaught exception in RunLogger plugin when running elemental rifts

[deirdresm]

Describe the bug
Uncaught exception in elemental rifts just as the (successful) battle completes. This will happen several times without creating a huge problem, but sometimes the proxy hangs or the app silently crashes. (I do not have a crash log, apparently, or I'd attach that.)

To Reproduce
Steps to reproduce the behavior:

1. Enable RunLogger
2. Run SW on your phone/tablet through the proxy
3. Go to the elemental rift dungeons (e.g. Wind Beast)
4. When the battle completes, I'm getting the error I pasted below.

I was getting this on a hand-built version, so I uninstalled that and downloaded the pre-built to ensure that it wasn't something wonky with my own build. Nope. I also tested this with other plugins turned off. I have no non-default plugins enabled by default.

Expected behavior
No error.

Screenshots

Uncaught Exception:
Error: Unexpected Error: column header mismatch expected: 26 columns got: 27
at ParserStream.self.__transform (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:138:48)
at ParserStream.__parseLineData [as __handleLine] (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:65:18)
at /Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:90:22
at asyncIterator (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/extended.js:13:17)
at Function.asyncEach (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/extended.js:31:10)
at ParserStream.__processRows (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:88:18)
at /Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:186:34
at headerHandler (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:157:13)
at ParserStream.__processHeaders (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:163:17)
at ParserStream._parseLine [as _parse] (/Applications/Summoners War Exporter.app/Contents/Resources/app.asar/node_modules/fast-csv/lib/parser/parser_stream.js:182:26)

Desktop (please complete the following information):

• OS: MacOS 10.15.6
• Version 0.3.5

Smartphone (please complete the following information):

• Device: iPad Mini 5th gen
• OS: iOS 13.6.1

Additional context
This has been happening for several weeks, it's just that I usually see it when I'm in another room. Because sitting up has literally been making me sick, it's taken a while to get to the diagnosis phase.

[deirdresm]

Going through it a few times, it's specifically happening at BattleRiftDungeonResult, and I've added a log excerpt on a gist, both with the request in raw form from the log and then prettified in JSON (which naturally it put first, ugh). I put that there so I could wipe the gist after this was solved:

https://gist.github.com/deirdresm/a2a7425fec4c96b3fbd163ff8ada29b4

[deirdresm]

I've narrowed down what the issue is. I've gone through every sw-exporter commit and every piece of data and it doesn't seem to be one of those. It started between upgrades of sw-exporter.

When it did start (cue Exorcist head turn) is when I installed a later version of node to run a site converter app for an entirely different reason, and the issue started that day. So v14.8.0 on my system (or one of its ancillary files) is no doubt the issue, which I didn't have installed before. It seems related to the fast-csv version, which was upgraded from 2.x to later, and the concommitant changes there.

[deirdresm]

Apologies for taking so long to dig into this, as it's involved delving into node issues that are new to me.

Possibly related: webpack no longer has an official release of @3.12.0, which has been nuked in favor of 4.0.0, which has significant architectural changes.

https://github.com/Xzandro/sw-exporter/blob/master/package.json

https://github.com/webpack/webpack/tags?after=v4.0.0-alpha.4

So once I accidentally updated for other reasons, I was unable to downgrade back to 3.12.0.

Anyhow, I have a one-line fix that works for me, but because of the library change, I can't test it on previous versions of webpack and verify it doesn't break them. It does also have the possibly unwanted side effect of tossing unmapped variables. But…it does prevent it from crashing on a setup with 4.0.0 webpack in my experience. Also, I've only been able to test on MacOS.

I'll submit a pull request.

[deirdresm]

I have a more correct fix now that I've dug deeper, though it doesn't fix the other issue: monster names don't appear in the columns (like they do for dungeon runs).

This turns out to be because the info sw-exporter is relying on is not, in fact, in the request. (Weirdly, given how verbose their requests normally are.)

The ultimate reason it was failing was that it was looking for "unit_list" in the response like for dungeon runs, which did not exist for rift runs, so those columns would not get added. That created a mismatch in the number of columns between the header and the run rows, hence the error and occasional crash.

For the BattleRiftDungeonResult command, the response where the units are listed is "my_unit_deck_list", which doesn't give full info about the units, unfortunately. But just changing that at least fixes the hairball.

I'll submit a pull request.

[ghispi]

Hi, I dont think its related to any software update that you mentioned.
I've started getting the issue sometime ago and I haven't been changing anything.
I'm getting the issue virtually in any campaign, beast or essence runs, doesn't matter 10x or single.

I'm running sw-exporter v0.0.35 with Nox 6.6.1.5 with Android 7 on Win10.

[Window Title]
Error

[Main Instruction]
A JavaScript error occurred in the main process

[Content]
Uncaught Exception:
Error: Unexpected Error: column header mismatch expected: 25 columns got: 28
    at ParserStream.self.__transform (C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:138:48)
    at ParserStream.__parseLineData [as __handleLine] (C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:65:18)
    at C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:90:22
    at asyncIterator (C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\extended.js:13:17)
    at C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\extended.js:23:29
    at C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:103:25
    at C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:73:29
    at ParserStream.__validate (C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:278:16)
    at C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:69:26
    at __transform (C:\Users\ghispi\AppData\Local\Temp\1fT1pG13V2znzMyGVqEu4Brb9Nv\resources\app.asar\node_modules\fast-csv\lib\parser\parser_stream.js:282:16)

[OK]

The funny part though is that I've friend running same setup whos do not have the issue.
The only difference I can come up with is that I had temp ban once.

[ghispi]

Still happening in v0.0.36.
I'm running with HTTPS Mode unchecked.

[Xzandro]

I can't reproduce. Would be cool if you could provide data from the full logger plugin to see what data is causing the issue.

[ghispi]

full_log.txt

[deirdresm]

Was it the last part of the log? That was a dungeon run fwiw.

[ghispi]

yep, #333 (comment)

I'm getting the issue virtually in any campaign, beast or essence runs, doesn't matter 10x or single.

[deirdresm]

Okay, will look into it. I think this is the same issue that I had a fix for, but I only fixed it in one context as that's how it was hitting me.

[ghispi]

This is definitely related to RunLogger plugin, it never occured to me before that disabling it prevents the crashes.
So if anyone comes here looking for a solution this is the workaround.

[deirdresm]

Here's what I think's happening: you and I have an different-than-specified version of the csv library because there was a change in one library and that had to be bumped…and the format is different.

You've got: "infocsv":"4.02.32"
Mine is: "infocsv":"4.02.04"

But what it's not is "fast-csv": "^2.5.0", which is what's specified in package.json.

Why? Ultimately, it's because the version of webpack that's specified ("webpack": "^3.12.0") no longer exists (it went poof), so the new dependencies of the newer version get met another way and that has different expectations about the format.

I don't know why the specified version was revoked, but having worked on a browser team, I expect that it was for a Very Good Reason™.

I've been time slicing looking into it more deeply to see if there's a way forward that's cleaner without breaking legacy builds.

[Xzandro]

See here: #338 (comment)

[Dalzhel]

Hi,
i get a similar error

Describe the bug

To Reproduce
Steps to reproduce the behavior:

1. get rune when farming

Expected behavior
The error pop, swex stop working proprely

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Version 10

Smartphone (please complete the following information):

• Device: MuMu
• OS: mumu running android

Additional context
Add any other context about the problem here.

[dvildebrand]

Hi, i get a similar error

Describe the bug

To Reproduce Steps to reproduce the behavior:

1. get rune when farming

Expected behavior The error pop, swex stop working proprely

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Version 10

Smartphone (please complete the following information):

• Device: MuMu
• OS: mumu running android

Additional context Add any other context about the problem here.

Pretty sure this is related to the run csv file containing corrupt data in some form, and it keeps happening on my end.
Workarounds for me:

• disable the run logger in settings
• delete the -run.csv file.

If I had to guess, the -run.csv column number was changed between versions and an older file still present cannot be appended to due to column mismatch.

In any case, probably not the same issue as the exception in the raid rift run reported by OP.

[EytanMorg]

I'm having the same issue as deirdresm, but this accounts for all dungeons when having Runlogger enabled.
attached: log and error notification

full_log.txt