Update OSITrace to be used in osi-validation

[ClemensLinnhoff]

Reference to a related issue in the repository

#761

Add a description

Re-add cache_messages_in_index_range() and some minor variables that were removed in #783 to ensure compatibility with osi-validation.

Take this checklist as orientation for yourself, if this PR is ready for the Change Control Board:

• My suggestion follows the style and contributors guidelines.
• I have taken care about the message documentation and the fields and enums documentation.
• I have done the DCO signoff.
• My changes generate no errors when passing CI tests.
• I have successfully implemented and tested my fix/feature locally.
• Appropriate reviewer(s) are assigned.

[pmai]

Except for the bug fix in the if condition, I think this is all a bad idea, and rather osi-validation message reading should be rewritten from scratch to be more efficient, cleaner and more transparent. Given the little that is actually needed, this should not be too hard to do; I'll maybe give it a try after the OSI release.

[pmai]

This is redundant to the length of message_offsets, and only valid once a file has been fully read in, so is not an apropriate API to the outside.

[pmai]

This turns a simple open operation into a very expensive operation, as it must read through the whole file to extract all the offsets. Sometimes this is needed, but in most instances you want to actually read and process a file sequentially, either step-by-step, or you need all messages anyway, in which case you also do not want to retrieve the offsets but rather read the messages at the same time. So that is a big no-no.

[pmai]

This is a duplication of line 40.

[pmai]

The bugfix is important,

It just makes me regret that we added black reformatting checks that this is convoluted by the arbitrary re-indentations needed. We probably should increase the line length to avoid stupidity like this.

[pmai]

This whole method is a Bad Idea(tm): A cache is something that stores things that have already been accessed.
If you just want to read everything into memory, this is not a cache, and should not reside inside OSI Trace, but rather inside the recipient that wants to work with the data, i.e. osi-validation. I actually hesitated in re-implementing the cacheing behavior because of the ripe potential for mis-use, as shown here.

In any case, the iterator returned from get_messages_in_index_range will already place all read messages into the cache, so the implementation is also needlessly redundant.

If one just wants to have all messages available in memory, then this really is not cacheing, it is just buffer reading and should be handled as such; i.e. if I want all messages anyway, then the simple one-liner suffices:

   trace = OSITrace(path, type) # Note: No cacheing in OSITrace
   messages = [*trace.get_messages()]
   trace.close()

And then access the messages by index via messages[i]. This is both the most efficient, and straightforward. If one wants to read in batches, something very similar can be done:

   from itertools import islice
   from functools import partial

   trace = OSITrace(path, type) # Note: No cacheing in OSITrace
   for batch in iter(partial(lambda n,i: list(islice(i,n)),batch_size,iter(trace.get_messages())), []):
     # Process batch here
   trace.close()

In Python 3.12 and later itertools.batched could be used directly.

In summary I would say osi-validation code that handles messages is currently a mess and should be rewritten to more clearly do the batch-processing it needs without multiple reads/re-reads of the file and deep interaction with internal mechanisms of the reader.

[ClemensLinnhoff]

Current OSITrace implementation will be used. We are going to make changes to osi-validation to use it: OpenSimulationInterface/osi-validation#70