Faciliating Debug Visualizations

[bratter]

@mfedderly and @smallsaucepan you've mentioned a couple of times that it would be useful to have some better debug visualization tooling. I felt the same thing and have started banging out an MVP of something that might help.

What I had in mind was the following:

1. A hook script that can be --imported into anything run with tsx (and compiled to be imported in node). The hook script:
   • Provides the entire turf library and a DebugViz global with some convenient functions to exfiltrate debug data.
   • Can be referenced either from the file under test or the console in a debugger.
   • Completely synchronous, so doesn't have to wait for next tick in the debugger to send a message
2. A super basic web socket relay server that forwards messages from the hook script to the front end.
3. A mapbox-driven visualizer front-end that receives messages from the hook that contain geojson data and displays them, with some ability to play around with the map.

The main functions to get data out of the system under test would be:

• DebugViz.send(geojson: GeoJSON, label?: string)
• DebugViz.diff(geojson1: GeoJSON, geojson2: GeoJSON, label?: string)

Then the front end would have some different view options including a diff mode to see the GeoJSON.

Workflow would be:

• Start the server
• If desired can put a /// <reference... in the file you are playing with to get typing on the hook
• Add debug viz statements into the test (e.g., DebugViz.diff(result, expectedOutput).
• Run a test with the hook (this is what I've been doing): npx tsx --import ../../../turf-debug-viz/src/debug-hook.ts test.ts
• Explore in the visualizer
• If doing something thorny can just send any in-scope variable from a breakpoint in the debugger, can also send reruns from different branches, etc.

Example screenshot of the very first version is below. This something you would be interested in?

[smallsaucepan]

Oh yes please! That would be a huge help. Amaze.

Are you thinking diff mode would show only the differences between geojson1 and 2? Or display both coloured differently like a text diff tool?

Did you say you would welcome feature requests? No? Well I'll submit some anyway 😆

If we could paste geojson and have the tool highlight incorrectly wound rings and overly precise coords that would be a boon for triaging bugs.

[bratter]

Lol, of course!

• Winding for sure.
• Precision I was wanting to somehow notate or visualize where differences were only due to precision above a threshold - is that what you had in mind?
• For diffs (for polygons at least), was thinking of making a diff(a, b) and diff(b, a) shape with different colors for visual inspection (obviously relies on clipping working for the inputs). Still thinking through lines and points...

Will share an early version when I have something!

[mfedderly]

There's some existing prior art on these which have been helpful to me.

https://geojson.io/#map=2/0/20 has good viewing and some geojson validation.
https://geojsonlint.com/ is another validator that also catches wind order.

The biggest thing I'm missing (well I have a vibecoded version, but its not checked in anywhere) is just the ability to quickly look at a test/out/x.geojson file and compare the current version to the one from the master branch to see if I broke output. Right now I tend to just have two tabs of geojson.io open and just click back and forth between them to visually check things.

When trying to reason about a PR's before/after state, it is hard to see from the text diff if the overall structure of the output is the same (nesting of FeatureCollection/GeometryCollection/Feature/Geometry) is the same or not. Sometimes polygons get reordered, or the ordering of the points within a ring gets reordered.

[smallsaucepan]

Taking @mfedderly's comments in mind, maybe a radio button like approach that lets someone quickly flick on and off in the same window two variations of what should be the same, and also display in lurid, unmistakable fluro green any differences.

So even if the structure ordering of the output has changed, the green diffs view should be clear. As mentioned though relies very much on us having a bullet proof diff function.

Regarding extreme precision, yes subtly highlighting values that are above a threshold. Not necessarily in the visual view though. More thinking in the text view.

[bratter]

Yeah, I use both of those tools on occasion (and they are great), but the workflow is challenging. Before/after comparisons + enabling an easy workflow from within the test harness + a decent suite of checks are what I'm aiming for - so quite aligned with what you say above. Will keep you posted!

[bratter]

I've made progress on this. You can see the in-progress, but generally working alpha version here: https://github.com/bratter/turf-debug-viz.

The basic use case is that you set it up locally while you are developing turf (or anything else really that produces geojson), hook into it using node/typscript imports, then send geojson to visualize. Instructions are in the readme on the repo.

The visualizer has a map and a js viewer with two main modes:

1. A view mode that shows geojson objects and detailed lints
2. A diff mode that shows both visual and js object level diffs

The linter is decent, although needs some refinement to specific lint cases (what should be error vs warning, add more lints, etc.)

The shape diffing is fine but relies on turf.difference() for its clipping. The js diff is... pre-alpha at best... but gives an idea of what I'm shooting for.

Here is a viewing/linting example:

And here is a diff example:

Keep me posted on any thoughts or improvements if you choose to take it for a spin!