Re-evaluate expressions at runtime issue

[kubatatami]

Describe the bug

We are trying to optimize syncing process in our app. We try to sync only limited amount of data using sync expressions. Later in runtime we modify properties to always return QueryPredicateConstant.all and we try to resync by stop-start as described in documentation(In order to have your expressions re-evaluated, you can execute Amplify.DataStore.clear() or Amplify.DataStore.stop() followed by Amplify.DataStore.start().)
Unfortunately it doesn't work as expected. New data will not show up in the app until you call Amplify.DataStore.clear(). The only workaround we found is set syncInterval to 0 what always force a fullSync but it's a bad solution for other reasons. It looks like deltaSync doesn't support changes in sync expressions. I found a PR with possible solution but there are no updates since February.

Steps To Reproduce

Steps to reproduce the behavior:
1. Create a table, fill with data
2. Create a sync expression that will limit some of data
3. Modify behaviour of sync expression in runtime. 
4. Call Amplify.DataStore.stop()
5. Call Amplify.DataStore.start()
6. When syncing done do a query. There will be no new data.
7. Call Amplify.DataStore.clear()
8. Call Amplify.DataStore.start()
9. When syncing done do a query. New data will show up.

Expected behavior

Re-evaluate expressions at runtime should work as described in documentation. Calling Amplify.DataStore.stop() and Amplify.DataStore.start() should re-evaluate expressions and pull new data.

Amplify Framework Version

2.22.0

Amplify Categories

DataStore

Dependency manager

Swift PM

Swift version

5.9

CLI version

12.7.1

Xcode version

15.0.1

Relevant log output

<details>
<summary>Log Messages</summary>


INSERT LOG MESSAGES HERE

```

Is this a regression?

No

Regression additional context

No response

Platforms

iOS

OS Version

all

Device

all

Specific to simulators

No response

Additional context

No response

[ruisebas]

Hi @kubatatami, thanks for opening this issue.

The team will take a look and post here once we have an update.

[5d]

Hi @kubatatami,
For the expected behavior, when stop and start to re-evaluate the new predicate expression, do you want to keep the old data which is evaluated by the previous predicate expression?

[kubatatami]

@5d Yes. Let's say we have a user with data from the last few months. User signs in and I want to let him use the app once the app synced data from the last 24h. Then I want to re-evaluate expressions to sync rest of data.

[5d]

@kubatatami Thanks for confirming. Therefore, the desired behavior is to preserve the data filtered by the previous predicate and apply the new predicate exclusively to the newly generated data upon triggering a restart on the datastore.

[kubatatami]

@5d @ruisebas I was also thinking about different strategy to observe syncing progress and let users use the app once the app synced data from the last 24h but I noticed that query Sync[tableName] requests(made internally by Amplify sdk) always returns data in pages from oldest to newest. Is there any way to change it on local or backend side and syncing data from newest to oldest?

[lawmicha]

Please try out the fix in lawmicha.ds-full-sync-expression-change #2757 and let us know if you are able to successfully change your sync expression and restart DataStore to do a full sync

[kubatatami]

Hey @lawmicha fix in lawmicha.ds-full-sync-expression-change looks good. I am able to successfully change sync expression and restart DataStore to do a full sync. We continue tests but so far it looks very promising. Thanks 🙏

[lawmicha]

Hi @kubatatami, thanks for trying out the fix and letting us know.

Is there any way to change it on local or backend side and syncing data from newest to oldest?

We don't expose a way to change the sort order for the Sync Query API call performed internally but that sounds like it might be possible and would not impact the integrity of the data. What are some use cases you have in mind for sorting by newest to oldest? Is it so that you will display the most recently created/updated items first and let the oldest items eventually come in?

[kubatatami]

@lawmicha Yes, exactly. Now on lawmicha.ds-full-sync-expression-change we are able to sync last X days of data, let user to use the app and then we re-evaluate expressions to sync all of data in the background. It works ok but if we would be able to change the sort order for the Sync Query API we would be able to observe sync progress and let user to use the app when we synced last X days of data. This way we would not have to pull data for last X days twice and older data would be available in the app from newest to oldest what in most cases is more important for user.

[lawmicha]

This has been released in https://github.com/aws-amplify/amplify-swift/releases/tag/2.25.1

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.