Scanner.scan utf8ByteIndexesMapping Array Out Of Bounds

[crusse54]

final byte[] utf8bytes = input.getBytes(StandardCharsets.UTF_8) encodes the unknown character as a single byte ASCII question mark. When Util.utf8ByteIndexesMapping(input, bytesLength); creates the byteIndexes array of size bytesLength (the size of utf8bytes), it creates an array that is 2 smaller than it should be since the unknown character is 3 bytes. The set of if statements then references the original string and decide that the unknown character is 3 bytes, filling 3 array spots with the character index. Eventually Array.fill goes out of bounds and an exception is thrown.

[gliwka]

Interesting, invalid UTF-8 characters are not handled properly and are replaced with an ASCII "?" instead of with U+FFFD. I will look into a different API that handles that correctly or a lower-level API that give me more control on how invalid chars are processed.

[gliwka]

Alternatively, I might also find a way to directly operate on the strings without the mapping. Let me know if you also have any ideas.

[yenuka78]

We also encountered this bug. When is a fix expected?

[gliwka]

Fix will be released in the upcoming days