Nitrite/Java: Asynchronous indexes

[neroux]

I'd have two questions regarding asynchronous indexes

1. I could not find a way to mark an annotation index asynchronous. Could you confirm that this is currently not possible and setting up such an index requires createIndex()?
2. How (and when) do asynchronous indexes exactly work? I set an asynchronous fulltext index on a field but inserts still take just as long as with a synchronous one. I'd expect them to be faster and have the indexing done in parallel in the background. Is my understanding incorrect?

repository.createIndex("field", IndexOptions.indexOptions(IndexType.Fulltext, true));

Thanks a lot!

[anidotnet]

Asynchronous indexing can only be initiated by createIndex() method. It is useful only when you already have data in your collection and you want to introduce new index and don't want to wait for the indexing to complete. It has no effect on updating index while inserting/modifying data.

[anidotnet]

Latter actually does not make much senses. When you modify a data which is indexed, then the indexed data must also be synchronized, otherwise you will encounter data races. So it is always better to do the index modification synchronously with the data modification.

[neroux]

That's a fair point, if you index asynchronously you certainly could have a temporarily inconsistent fulltext index, but that can happen with the current implementation as well.

The reason why I was asking is that, in my use-case, I get X new records which I am adding to the database and subsequently show to the user based on what's in the database. Right now I'd either have to forgo the builtin fulltext indexing or wait for the inserts to complete the parsing, which takes quite a lot of time. If that happened asynchronously, I'd simply have the records added, could show it straight away, and the indexing happens in the background.

[anidotnet]

I can't comment precisely on your use case based on limited knowledge, but I think you might consider introducing asynchrony between displaying the data and inserting the data. Nitrite has built-in events. You can subscribe to the insert event. Once the insert and indexing is complete, you get notified, then you can update the display.

[neroux]

Knowing when the record has been added is not the issue, that I could cover without events as well. The issue rather is that the logic is based on what's in the database and if inserting takes significantly longer, so will displaying the individual record. The only solution would be to decouple that from the database and show data based on what I already got versus on what I stored. I'd rather want to avoid that and would probably opt for my own basic fulltext indexer in that case.

I understand your concerns about potential inconsistency, but that's always a possibility with anything asynchronous (also with the current implementation) and I - personally - would be happy to accept that the fulltext index might be briefly inconsistent, should I really happen to make such an update.

[anidotnet]

If insertion is taking long time, it is better to decouple the indexing from the insertion and use third-party full-text indexing like lucene. You will have better control on indexing.