Add pure-Java gesture typing fallback (SwipeGestureEngine)

[simon-liesinger]

Context

HeliBoard's gesture typing infrastructure — touch tracking, trail rendering, the settings toggle — has always been present, but word matching requires a native .so. Without the proprietary Google libjni_latinimegoogle.so (or the open replacement being built under the NLnet grant, #2226), users who swipe get no suggestions at all.

This PR adds a self-contained Java fallback that makes gesture typing work for everyone, without any native library.

Algorithm

SwipeGestureEngine uses arc-length resampling + L2 distance scoring, a technique closely related to SHARK2 (documented in #668):

Index build (once per dictionary/layout, background thread):

• For each dictionary word, look up the centre pixel of each letter's key from the live Keyboard object.
• Deduplicate consecutive identical positions (handles double-letters), then arc-length resample to 16 evenly-spaced (x, y) points, normalised to keyboard dimensions.
• Group entries by first letter for fast filtering.

Gesture match (each swipe):

1. Find the nearest letter key at the stroke's start and end points.
2. Filter dictionary candidates by first letter; then by last letter (relaxed to all first-letter candidates if the last-letter filter leaves nothing).
3. Arc-length resample the raw input stroke the same way.
4. Rank by L2 distance to the precomputed word path, with a small log(freq+1) bonus so common words win ties.
5. Return the top N results as SuggestedWordInfo.

Integration

File	Change
gesture/SwipeGestureEngine.java	New file — the entire engine (~270 lines, pure Java)
Suggest.kt	getSuggestedWordsForBatchInput builds/caches the index and calls the engine; never calls the JNI stub with SESSION_ID_GESTURE (which crashes on the AOSP stub)
JniUtils.java	sHaveGestureLib = true unconditionally so the gesture toggle appears in settings and touch input is routed as batch input
Dictionary.java	Default getAllWordsWithFrequency() returning empty map
ReadOnlyBinaryDictionary.java	Override: token-based JNI iteration yielding word + probability
DictionaryCollection.java	Override: delegates to contained dictionaries
DictionaryFacilitator.java / DictionaryFacilitatorImpl.kt	Expose getAllMainDictionaryWordsWithFrequency()

Properties

• Layout-aware: key positions are read from the live Keyboard object at index build time, so QWERTY, Dvorak, custom layouts, and layout changes all work correctly.
• Language-aware: uses HeliBoard's own main dictionary (full word list + frequencies), not a bundled word list.
• Coexists with native library: JniUtils still tries to load the native .so first; if it succeeds, the native engine takes over and this code is dormant.
• Zero new dependencies, pure Java.

Relation to #2226 / #668

This is intentionally a simple baseline — the NLnet open native library will eventually supersede it with higher accuracy (SHARK2-level matching, multi-language, etc). Until that ships, this gives every HeliBoard user working gesture typing out of the box. The two can coexist during the transition.

Testing

Tested on a Pixel 6a (Android 14) with the default English dictionary and QWERTY layout. Common short words (hello, game, world, the, because, …) resolve correctly with a casual swipe. Index build takes ~1–2 s on first gesture; subsequent gestures are instant.

[theeclecticdyslexic]

This is super cool @simon-liesinger I'm going to look over the code you wrote soon! I'm on my weekend at the moment though.

It's very cool that you got something working here. :-) It's a good idea to have a fall back in the interim.

Do you, perchance, use long press key labels in this to handle diatrics and ligatures? (As I said, Haven't read it over yet).

Edit: I just snuck a peak at some code. You claim 270 lines of Java, but it looks like 370 lines of kotlin and Java to me. It's also unusual the way some of this is worded. It says users who swipe without the library get no suggestions at all. They can't swipe, so there are no suggestions to give. It's a small distinction, but one I find a bit awkward. This feels a bit off to me. It's also all in one commit...

Can you confirm you wrote this, and not an LLM? Cheers.

[simon-liesinger]

Yes, an LLM wrote this (writing by hand now, though). Sorry about the errors in the description (although in it's defense, some of them may have been caused by the force-pushes).
I work with claude code, and started this as a private project because I wanted to use Heliboard with swipe typing. It didn't occur to me that you might not want it because it was written by an LLM. But I guess if you don't want it, then you don't want it.

code: 100% claude code
PR description: 100% claude code

Is there a tag or something I should use in future? Again, sorry, I should have said something at the start.

Edit: I feel like I should explain some of the mistakes it made. gesture/SwipeGestureEngine.java has +277 lines of pure java. I'm pretty sure it's just talking about that file, not the changes as a whole. the 'swipe without library' thing was probably a reference to when it enabled swiping before implementing the word matching. Still not reasonable to put in the PR, though. The singular commit was actually not it's fault, as I mentioned this was going to be a personal project, not a contribution here, and so all the work was done in a local clone.

Edit 2: After your comment, it can now handle diacritics.