Fix #54 misbehaving integers #97

tims-bsquare · 2023-11-20T11:10:04Z

Modify build.sh to use 64 bit floats but 32 bit ints wasm supports this but not 64 bit ints
When pushing a number see if it's within 32 bits as well as being an int if it's not then push it as a 64 bit float
Add tests for negative numbers and numbers greater than 32 bit

* Modify build.sh to use 64 bit floats but 32 bit ints wasm supports this but not 64 bit ints * When pushing a number see if it's within 32 bits as well as being an int if it's not then push it as a 64 bit float * Add tests for negative numbers and numbers greater than 32 bit * Fixed build issue with current emscripten emscripten-core/emscripten#20721 * Updated docs with edge cases to hopefully close off ceifa#22 ceifa#23 and ceifa#54

Sapu94 · 2023-11-20T22:34:50Z

I was just testing wasmoon out today and hit some unexpected behavior caused by 32-bit ints, so I am excited to see this issue being worked on. Unfortunately, I'm hitting a segfault on your branch (which doesn't occur on main). I'm still working on trying to find a minimal script that reproduces this, but let me know if you have any ideas or suggestions on how to get a more useful stack trace. I have bisected it down to being caused by the LUA_INT_DEFAULT change (and not the thread.ts change or other build.sh changes) at least.

Aborted(segmentation fault)
/home/wasmoon/dist/index.js:1663
     /** @suppress {checkTypes} */ var e = new WebAssembly.RuntimeError(what);
                                           ^
RuntimeError: Aborted(segmentation fault)
    at abort (/home/wasmoon/dist/index.js:1663:44)
    at segfault (/home/wasmoon/dist/index.js:1474:6)
    at glue.wasm.SAFE_HEAP_STORE_i32_4_4 (wasm://wasm/glue.wasm-006c350e:wasm-function[1373]:0xc2972)
    at glue.wasm.emmalloc_memalign (wasm://wasm/glue.wasm-006c350e:wasm-function[1282]:0xbf8cd)
    at glue.wasm.emmalloc_aligned_realloc (wasm://wasm/glue.wasm-006c350e:wasm-function[1286]:0xbfddc)
    at glue.wasm.emmalloc_realloc (wasm://wasm/glue.wasm-006c350e:wasm-function[1288]:0xc0155)
    at glue.wasm.l_alloc (wasm://wasm/glue.wasm-006c350e:wasm-function[185]:0x13d7d)
    at glue.wasm.luaM_malloc_ (wasm://wasm/glue.wasm-006c350e:wasm-function[667]:0x55311)
    at glue.wasm.luaC_newobjdt (wasm://wasm/glue.wasm-006c350e:wasm-function[490]:0x3b43a)
    at glue.wasm.luaC_newobj (wasm://wasm/glue.wasm-006c350e:wasm-function[491]:0x3b5b6)

Sapu94 · 2023-11-20T22:56:46Z

Turns out that crash is triggered by me wrapping require in a JS function. Feel free to ignore that as it's probably not a recommended thing to do, and without this hook, everything is working perfectly for me, but here's a simplified version of what I was doing for reference (along with some reasonably-complex lua code that does a bunch of require's):

const origRequire = lua.global.get('require');
lua.global.set('require', path => origRequire(path));

Sapu94 · 2023-11-20T23:04:16Z

build.sh

While testing this, I made the following update to the if/else block at the bottom to avoid leaving local changes to the lua submodule.:

if [[ "$OSTYPE" == "darwin"* ]]; then sed -E -i '' "s/#define LUA_INT_DEFAULT\s+LUA_INT_INT/#define LUA_INT_DEFAULT\t\tLUA_INT_LONGLONG/" ./lua/luaconf.h else sed -E -i "s/#define LUA_INT_DEFAULT\s+LUA_INT_INT/#define LUA_INT_DEFAULT\t\tLUA_INT_LONGLONG/" ./lua/luaconf.h fi

tims-bsquare · 2023-11-21T09:23:22Z

That's a funky error, I'll take a look

tims-bsquare · 2023-11-21T09:35:11Z

I've tried to write a test to reproduce it but it passes, could you try and tweak it @Sapu94 until it fails please?

    it('wrap require function', async () => {
      const factory = getFactory()
      await factory.mountFile('test.lua', 'return 10')
      const engine = await factory.createEngine()

      const origRequire = engine.global.get('require');
      engine.global.set('require', path => origRequire(path));

      await engine.doString(`require('test')`)
    })

tims-bsquare · 2023-11-21T09:38:30Z

It could also be related to this emscripten-core/emscripten#13131

Sapu94 · 2023-11-29T16:58:37Z

Just getting back to this after being away for the past week. It seems like this can only be reproduced by getting the Lua VM / wasm stack into some specific state. Sorry in advance for the huge can of worms this seems to be 😅. Here's as far as I've gotten in coming up with a minimal set of reproduction steps (I'm doing all this in WSL for what it's worth):

Clone your fork into ~/wasmoon and switch to the branch for this PR
Build wasmoon with npm run build:wasm:docker:dev && npm run build
Clone https://github.com/Sapu94/wasmoon-test into ~/wasmoon-test and install dependencies with npm install (the wasmoon dependency is set as file:../wasmoon so it uses the local clone)
Run main.ts: npx ts-node main.ts

Here's the latest error:

$ npx ts-node main.ts
Aborted(segmentation fault)
/home/sapu/wasmoon/dist/index.js:1663
     /** @suppress {checkTypes} */ var e = new WebAssembly.RuntimeError(what);
                                           ^
RuntimeError: Aborted(segmentation fault)
    at abort (/home/sapu/wasmoon/dist/index.js:1663:44)
    at segfault (/home/sapu/wasmoon/dist/index.js:1474:6)
    at glue.wasm.SAFE_HEAP_STORE_i32_1_1 (wasm://wasm/glue.wasm-006bc9de:wasm-function[1358]:0xc15d3)
    at glue.wasm.luaV_execute (wasm://wasm/glue.wasm-006bc9de:wasm-function[1070]:0xaedc4)
    at glue.wasm.ccall (wasm://wasm/glue.wasm-006bc9de:wasm-function[435]:0x33fe4)
    at glue.wasm.luaD_callnoyield (wasm://wasm/glue.wasm-006bc9de:wasm-function[436]:0x340be)
    at glue.wasm.f_call (wasm://wasm/glue.wasm-006bc9de:wasm-function[105]:0xa4b5)
    at invoke_vii (/home/sapu/wasmoon/dist/index.js:6285:31)
    at glue.wasm.luaD_rawrunprotected (wasm://wasm/glue.wasm-006bc9de:wasm-function[416]:0x2fc40)
    at glue.wasm.luaD_pcall (wasm://wasm/glue.wasm-006bc9de:wasm-function[447]:0x35745)

Let me know if you're able to reproduce the issue with these steps or not.

Here are a few of the things I've tried in my test repo that make me the crash go away:

Removing seemingly any single remaining line of lua code.
Renaming the mounted TestLuaCode folder to A and update the require()'s accordingly
Moving IncludeB.lua up one level (to TestLuaCode/Lib) and updating the require() accordingly
Configuring the engine with {enableProxy: true} instead
Removing the hook of require as mentioned above

I don't actually think any of the above are super meaningful, other than to point to the issue being caused by things getting into a specific (corrupted) state as opposed to just one faulty function / line of code somewhere.

tims-bsquare · 2023-11-30T10:03:30Z

That's a great repro repo, thank you for putting that together! :) I think what you encountered was an actual proper segfault where the library was trying to reference a free'd value and I suspect sometimes the memory may not yet have been corrupted depending on those small changes you listed.

I've pushed another change to this branch, unrelated to the misbehaving integers hopefully fixing this for you (it does for me locally).

Something to be aware of is that there's a potential memory leak with doFile/callByteCode that's necessary to avoid segfaults, which was and is done by moving the result value from the execution thread to the global. The code was returning a reference to the value in the thread which had been cleaned up rather than the copy in the global state.
Anyway that means if you intend to call doFile regularly with results being returned then it's important to either reset your global state or pop the result from the top of the stack with lua.global.pop()

Sapu94 · 2023-11-30T13:44:28Z

That fix works great, thanks! Good to know about the memory leak.

ceifa · 2023-11-30T14:37:11Z

build.sh

 else
-    extension="$extension -O3 --closure 1"
+    # ASSERTIONS=1 required with optimisations and strict mode. https://github.com/emscripten-core/emscripten/issues/20721


Actually this is a bug on emscripten? Assertions are probably unnecessary and just increase the code size and slow down a little bit. Should have a TODO to remove it later, wdyt?

As far as I can tell it's a bug in emscripten, the one linked. I'd be quite happy if you found time to take a second look though just in case. I'll add that TODO in

ceifa · 2023-11-30T14:38:40Z

Great job @tims-bsquare! Can I merge it or are you still working on this PR?

tims-bsquare · 2023-11-30T14:41:35Z

No problem, should be ready to merge @ceifa :) Let me add that TODO first though

tims-bsquare force-pushed the bugfix/misbehaving-integers branch from 5abe229 to 6272005 Compare November 20, 2023 11:49

Sapu94 reviewed Nov 20, 2023

View reviewed changes

tims-bsquare requested a review from Sapu94 November 28, 2023 11:57

tims-bsquare added 2 commits November 30, 2023 10:10

Fix callByteCode can segfault when returning a Lua referenced value

5534d35

Run linter on README

6bcda76

tims-bsquare force-pushed the bugfix/misbehaving-integers branch from 2b77a59 to 6bcda76 Compare November 30, 2023 10:11

ceifa reviewed Nov 30, 2023

View reviewed changes

Added TODO for disabling assertions in prod build

baebf66

ceifa merged commit fc9fbd4 into ceifa:main Nov 30, 2023
3 checks passed

Sapu94 mentioned this pull request Dec 5, 2023

Undesired tostring()/format() results for large numbers #99

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #54 misbehaving integers #97

Fix #54 misbehaving integers #97

tims-bsquare commented Nov 20, 2023

Sapu94 commented Nov 20, 2023

Sapu94 commented Nov 20, 2023 •

edited

Loading

Sapu94 Nov 20, 2023

tims-bsquare commented Nov 21, 2023

tims-bsquare commented Nov 21, 2023

tims-bsquare commented Nov 21, 2023

Sapu94 commented Nov 29, 2023

tims-bsquare commented Nov 30, 2023

Sapu94 commented Nov 30, 2023

ceifa Nov 30, 2023

tims-bsquare Nov 30, 2023

ceifa commented Nov 30, 2023

tims-bsquare commented Nov 30, 2023

Fix #54 misbehaving integers #97

Fix #54 misbehaving integers #97

Conversation

tims-bsquare commented Nov 20, 2023

Sapu94 commented Nov 20, 2023

Sapu94 commented Nov 20, 2023 • edited Loading

Sapu94 Nov 20, 2023

Choose a reason for hiding this comment

tims-bsquare commented Nov 21, 2023

tims-bsquare commented Nov 21, 2023

tims-bsquare commented Nov 21, 2023

Sapu94 commented Nov 29, 2023

tims-bsquare commented Nov 30, 2023

Sapu94 commented Nov 30, 2023

ceifa Nov 30, 2023

Choose a reason for hiding this comment

tims-bsquare Nov 30, 2023

Choose a reason for hiding this comment

ceifa commented Nov 30, 2023

tims-bsquare commented Nov 30, 2023

Sapu94 commented Nov 20, 2023 •

edited

Loading