Update indexes within a transaction

[davidfou]

Hello community 👋

Just bumped into a road block regarding using transactions. From what I understood, the transaction are useful for:

• calling listeners only once
• avoid listeners to be called with potential inconsistent data (because a write is still missing)

We rely on them a lot. Recently, we tackled some performance issues by using the indexes. I was surprised that the indexes are not updated within a transaction:

import { createIndexes, createStore } from 'tinybase';

const store = createStore().setTable('pets', {
  fido: { species: 'dog' },
  felix: { species: 'cat' },
  cujo: { species: 'dog' },
});

const indexes = createIndexes(store);
indexes.setIndexDefinition('bySpecies', 'pets', 'species');

store.setRow('pets', 'felix', { species: 'dog' });
console.log(indexes.getSliceRowIds('bySpecies', 'dog')); // logs [ 'fido', 'cujo', 'felix' ] as expected

store.transaction(() => {
  store.setRow('pets', 'rex', { species: 'dog' });
  console.log(indexes.getSliceRowIds('bySpecies', 'dog')); // I would expect rex to be logged here
});

Would you say that's a bug or a misuse of the library? Any idea how to keep the transaction and have indexes to be updated?

Thanks in advance for the help 🙏

[davidfou]

I managed to patch the lib to ensure that "internal" listeners are called even within a transaction -> davidfou#1

The code is at the moment more a quick and dirty patch but might be enough to spark the conversation.

Here is my take away:

• At the moment, all indexes are refreshed even within a transaction. Maybe this could new behavior could be controlled by adding an extra parameter to setIndexDefinition.
• If one transaction does a massive batch of operations without using the indexes, it might hurt the performance. Maybe adding a new parameter could be added when starting the transaction to not update the indexes.
• I had some difficulties to define a private method on the store to add internal listeners -> the function context is used, that was just to unlock me.

@jamesgpearce do you have any opinion on that one?