Incorrect positions might be recorded after ORC-1711

[nickitat]

Hi 👋🏼
It seems we stepped on a bug after updating ORC submodule on 360c061.

In the corresponding PR we start seeing test failure.
The failing test: select count(), sum(number) from file('02892.orc') where indexHint(sometimes_null is NULL);.
The exception: orc::ParseError, e.what() = Bad seek to (chunkHeader=0, posInChunk=65536) in zstd(ORCInputStream from 1401 for 12)

While looking at this test I noticed that we indeed read wrong positions here:

orc/c++/src/Reader.cc

Lines 457 to 462 in f919fab

	if (pbStream.kind() == proto::Stream_Kind_ROW_INDEX) {
	proto::RowIndex rowIndex;
	if (!rowIndex.ParseFromZeroCopyStream(inStream.get())) {
	throw ParseError("Failed to parse the row index");
	}
	rowIndexes_[colId] = rowIndex;

And then I found that we actually also write incorrect values here:

orc/c++/src/ByteRLE.cc

Lines 190 to 194 in f919fab

	if (outputStream->isCompressed()) {
	// start of the compression chunk in the stream
	recorder->add(flushedSize);
	// number of decompressed bytes that need to be consumed
	recorder->add(unflushedSize);

And then I found that the difference between previously written values and the values we're getting after the update is only that in the positions when we previously had zeroes we now have 65536. And this number is equal to MemoryBlockSize in the PR. It happens when all three unusedBufferSize, bufferLength, bufferPosition were equal to 0.

I'm not familiar with the code, so all I did is just implemented a small crutch to work around that issue: ClickHouse@e77893c and at least it fixed our tests.
Now I'd like to ask the actual experts in this code to have a look and implement a proper fix.

Thanks!

[nickitat]

cc @luffy-zh @ffacs @wgtmac

[luffy-zh]

Thanks for the report, I'll take a look.

[nickitat]

i checked on our tests and everything looks fine now - big thanks @luffy-zh