Offline Support (v5.0.0-beta)

[vanGalilea]

Hi everyone,

Our team has rolled out offline support today, and we would like to gather your thoughts and feedback in a manageable manner. Currently we have released a beta version of v5 of the RN Chat SDK.

This discussion will contain any idea, piece of feedback, or advice with regards to offline support.

You can find more info in our migration guide from v4 to v5 about how to upgrade the SDK and how to opt-in for offline support.

P.S. Docs are a little raw at the moment, but we will improve them as we go. As a priority we wanted to release beta version of offline support and gather as much feedback as possible.

[nandorojo]

Is there a reason to initialize this state as false?

[vishalnarkhede]

So you don't need to wait for connectUser to succeed, but it's important to call connectUser before rendering any component.

So state is initialized as false, and set to true right after calling connectUser(). But no need to await it

[vishalnarkhede]

@nandorojo does that make sense? Or am I missing something

[nandorojo]

I'm on my phone but I'll try to share some code.

It seems like the best move is then to either 1) call the promise without awaiting it in the effect:

const connectPromise = chatClient.connectUser(user, tokenOrTokenProvider)() // notice it's called
setIsClientReady(true)
await connectPromise

or, 2) the promise itself could be initialized in useState

const [promise] = useState(() => 
  chatClient.connectUser(user, tokenOrTokenProvider)()
)

And then if you need to await the connection for something else, you could in an effect:

const [promise] = useState(() => 
  chatClient.connectUser(user, tokenOrTokenProvider)()
)

useEffect(() => {
  await promise 

  doSomethingElse()
}, [promise])

[nandorojo]

The first snippet is functionally the same as the docs have, except that it more accurately sets state to ready after connecting user.

The benefit of the second approach is that there is no flash of empty content.

[vishalnarkhede]

Thats correct indeed, connectUser can be called during initialization of useState itself. But generally in real applications, you need to get the id of current user from some local storage as well and there might some more logic to be executed before calling connectUser. Thats why I decided to mention 1st approach in sample code. But approach 2 can be used as well.

We will mention it within our docs soon. Thanks for bringing this up.

[andreibarabas]

Hey guys,

Any issues that we come across, should they be added here, or to the issues tab and linked here? What do you prefer?

[vishalnarkhede]

We would prefer here!!

[andreibarabas]

One issue is related to the location of the sqlite file on Android. currently, it's under the files/databases folder, while you would expect it to simply by under the database. I know the location you specified is only databases, but it was created in the files folder.

What this means is that android context won't know of it, and thus flipper requires manual setup.

      client.addPlugin(new DatabasesFlipperPlugin(new SqliteDatabaseDriver(context, new SqliteDatabaseProvider() {
          @Override
          public List<File> getDatabaseFiles() {
              List<File> databaseFiles = new ArrayList<>();
              for (String databaseName : context.databaseList()) {
                databaseFiles.add(context.getDatabasePath(databaseName));
              }
              
              //stream-chat-react-native does not save the database file in the regular databases folder, but in the files one
              databaseFiles.add(new File(context.getFilesDir(), "databases/stream-chat-react-native"));
              return databaseFiles;
          }
      })));

for example, this android call will not work in order to setup Flipper databaseFiles.add(context.getDatabasePath("stream-chat-react-native"))

[vishalnarkhede]

You are right!! I am guessing this is where its coming from - https://github.com/ospfranco/react-native-quick-sqlite/blob/127b0dec4d34aa6530d1430ea4ff08ef60592aab/android/src/main/java/com/reactnativequicksqlite/QuickSQLiteBridge.java#L16

I am not sure though if we can do anything about this at the moment, but its better that we mention it in our docs since it might help other people. Thanks for bringing it up :)

[vishalnarkhede]

Is this a blocker for offline support integration in some way, besides flipper plugin?

[andreibarabas]

no, it's not. offline support was something we have been working in tandem with you guys, so now that it's available in the SDK, we will make all the efforts needed to adopt it. I'm just giving feedback on what we find so that you guys can make a decision if it might be a blocker, or not, for others

[andreibarabas]

You are right!! I am guessing this is where its coming from - https://github.com/ospfranco/react-native-quick-sqlite/blob/127b0dec4d34aa6530d1430ea4ff08ef60592aab/android/src/main/java/com/reactnativequicksqlite/QuickSQLiteBridge.java#L16

I am not sure though if we can do anything about this at the moment, but its better that we mention it in our docs since it might help other people. Thanks for bringing it up :)

exactly. at least mention the solution above in the docs, so others don't spend the time debugging

[vishalnarkhede]

I'm just giving feedback on what we find so that you guys can make a decision if it might be a blocker, or not, for others

Thanks alot. Really appreciate it :)

[andreibarabas]

Found another issue.

as you can see in the source, the result can be a empty array (in case there is no rows), but then the next line you expect result[0] to be defined.

[vishalnarkhede]

How did you reproduce this issue though? Was it on first app load?

[vishalnarkhede]

I would ideally expect pragma user_version to be available by default, but maybe I am wrong.

[vishalnarkhede]

But regardless, I will include the fix in next beta release which should be next week. Is the testing blocked because this?

[andreibarabas]

I reproduced it by mistake. I executed a query in Flipper (a truncate) and then the database was locked on that process, so the app could not write anymore.

Nonetheless, it's good to add the ? just to be safe.

[andreibarabas]

Hi @vishalnarkhede, I saw you did not add the ? safety check, but if I understand the code correctly, if there are no results, the line const result = rows ? rows._array : []; will return the empty array, thus result[0].user_version will fail

https://github.com/GetStream/stream-chat-react-native/blob/develop/package/src/store/QuickSqliteClient.ts#L137

[andreibarabas]

We've spent a lot of time polishing the initial app load, and we continuously find situations like this

but if you would press it

The root cause is this line

https://github.com/GetStream/stream-chat-react-native/blob/develop/package/src/components/ChannelPreview/ChannelPreview.tsx#L43-L45

the lastMessage has not yet been loaded (undefined) but the translatedMessage is {}. Then, by the time it get's to useLatestMessagePreview, the state is loaded and thus channelLastMessageString populated with the final message, but empty text. It's not something we can reproduce 100% reproduce, but it blocks the user when it happens

The working patch is this, but it's not efficient (Object.keys). It might be better to look at supporting null as the translatedMessage, but I leave that up to you

diff --git a/src/components/ChannelPreview/hooks/useLatestMessagePreview.ts b/src/components/ChannelPreview/hooks/useLatestMessagePreview.ts
index d4b47b9ad480d1643e8143ec66392750e74f6441..3a48ce095eff3387c10518f15b9be5d6934e5b1a 100644
--- a/src/components/ChannelPreview/hooks/useLatestMessagePreview.ts
+++ b/src/components/ChannelPreview/hooks/useLatestMessagePreview.ts
@@ -244,6 +244,11 @@ export const useLatestMessagePreview = <
   const messages = channel.state.messages;
   const message = messages.length ? messages[messages.length - 1] : undefined;
 
+  //workarround for "Empty message..." preview
+  if(lastMessage && Object.keys(lastMessage).length === 0) {
+    lastMessage = message;
+  }
+
   const channelLastMessageString = `${lastMessage?.id || message?.id}${
     lastMessage?.updated_at || message?.updated_at
   }`;

[vishalnarkhede]

I will create a separate issue for this, since its an existing bug (as far as I can read from description) in current stable version of SDK

[andreibarabas]

might be, but have never experienced it before. I think the offline capabilities surfaced it

[vishalnarkhede]

Ohhh. Sorry I misunderstood it then. Let me investigate and get back to you asap

[vishalnarkhede]

the lastMessage has not yet been loaded (undefined) but the translatedMessage is {}.

I am curious how come lastMessage is undefined, since we initiate the value within useState itself

  const [lastMessage, setLastMessage] = useState<
    | ReturnType<ChannelState<StreamChatGenerics>['formatMessage']>
    | MessageResponse<StreamChatGenerics>
    | undefined
  >(channel.state.messages[channel.state.messages.length - 1]);

So technically lastMessage should already have a value in first render of the component.

[andreibarabas]

@vishalnarkhede - it might be nice to have, but was curious what you think of this pseudo loading state. It's most noticeable at a cold start, when an entire channel list looks like this for a few frames

I would say this is bad because it provides an inconsistent state (i.e: the channel has loaded, but the messages have not). My first reaction would be to continue displaying the skeleton until we can safely display the last message preview, instead of the "Nothing yet..." array, but wanted to see what you guys think would be best

[andreibarabas]

See it in action here, even if the final channel list is empty. I have to check if this might be related to FlashList (we have integrated it instead of FlatList as per the steps I laid out here #1594).

Untitled.mov

[andreibarabas]

Nope. it's actually a problem from the createSelectQuery as in case the channelIds is an empty list (due to the no channels matching the filters), it actually returns the entire database of channels :)

A test that you could add to the createSelectQuery.test.ts

  it('REGRESSION: should support empty channel list, and not return the entire database', () => {
    const query = createSelectQuery('channels', ['*'], {
      cid: [],
    });

    expect(query).toStrictEqual([
      'SELECT cid, id FROM channels WHERE cid in ()',
      [],
    ]);
  });

[andreibarabas]

I see you specifically ignored this case though
https://github.com/GetStream/stream-chat-react-native/blob/v5.0.0-beta.0/package/src/store/sqlite-utils/appendWhereCluase.ts#L18

Curious what is the use case for that skipping?

[andreibarabas]

One last thing for today. For channel deep linking, we intentionally do not load the entire channel list, so that we can make it super fast.

Based on the usePaginatedChannels example, we did something like this

      const channelsFromDB = getChannel<HiBeam.StreamChatType>({
        currentUserId: chatClient.userID!,
        channelId: channel.cid,
      });

      chatClient.hydrateActiveChannels(channelsFromDB, {
        staticState: true,
      });

and we temporarily created a getChannel function, but I think it should be in the SDK

export const getChannel = <
  StreamChatGenerics extends DefaultStreamChatGenerics = DefaultStreamChatGenerics,
>({
  currentUserId,
  channelId,
}: {
  currentUserId: string;
  channelId: string;
}): Omit<ChannelAPIResponse<StreamChatGenerics>, 'duration'>[] => {
  const channelIds = [channelId];

  const channels = selectChannels({channelIds});

  const cidVsMembers = getMembers<StreamChatGenerics>({channelIds});
  const cidVsReads = getReads<StreamChatGenerics>({channelIds});
  const cidVsMessages = getMessages<StreamChatGenerics>({
    channelIds,
    currentUserId,
  });

  // Enrich the channels with state
  // @ts-expect-error we are using StrictMode, thus members could be undefined and raise a missmatch error
  // todo: this is a TEMP WORKARROUND
  return channels.map(c => ({
    ...mapStorableToChannel<StreamChatGenerics>(c),
    members: cidVsMembers[c.cid],
    messages: cidVsMessages[c.cid],
    pinned_messages: [],
    read: cidVsReads[c.cid],
  }));
};

At the end, we call watch to get the latest details, and this is how we save the latest details

//load the channel details
    channel.watch().then(
      channelSnapshot => {
        //save the latest snapshot, so the next deepLink will have the latest messages
        upsertChannels({
          channels: [channelSnapshot],
          flush: true, //flush the data
          isLatestMessagesSet: true, //we have everything
        });
      },
      reason => {
        console.error(reason);
      },
    );

what do you think?

[vishalnarkhede]

and we temporarily created a getChannel function, but I think it should be in the SDK

I agree, and will export this function from SDK.

Regarding following piece of code, its actually not necessary to upsertChannels here on your (application) side. Anytime you query or watch a channel, js client emits an event named channels.queries. And within our RN SDK, we listen for this event and upsert the channels https://github.com/GetStream/stream-chat-react-native/blob/v5.0.0-beta/package/src/components/Chat/hooks/handleEventToSyncDB.ts#L111

    channel.watch().then(
      channelSnapshot => {
        //save the latest snapshot, so the next deepLink will have the latest messages
        upsertChannels({
          channels: [channelSnapshot],
          flush: true, //flush the data
          isLatestMessagesSet: true, //we have everything
        });
      },
      reason => {
        console.error(reason);
      },
    );

[andreibarabas]

are you sure It works? I thought so, but when I actually tested out, the "getChannel" was returning empty result. Will test it again later and see, maybe there is a bug in the JS SDK then

[vishalnarkhede]

Could you check if the version of stream-chat in yarn.lock file is 7.0.0-offline-support.3?

[vishalnarkhede]

Also what you can do is to add following listener and see if you receive the event (just for debugging):

client.on('channels.queried', (e) => { console.log(e) })

[andreibarabas]

confirmed. it works as expected with the latest stream-chat 7.0, so no need to upset

[andreibarabas]

So, for cases where the filters applied return an empty channel list, it will wait for the channelList to get the same empty list from the server before displaying anything. This is a problem when in offline mode because it will fail the call, and the user will know as he was on that particular screen a few minutes ago and it worked (was empty).

This is mainly due to the condition

channels.length === 0 - https://github.com/GetStream/stream-chat-react-native/blob/v5.0.0-beta.0/package/src/components/ChannelList/hooks/usePaginatedChannels.ts#L208

and the fact that selectChannelIdsForFilterSort returns an empty array if there are no rows for a specific filter.

To solve this, the proposed steps are:

1. selectChannelIdsForFilterSort to return an array or NULL
2. getChannels to return an array or NULL
3. To not add a new state (i.e: isRehydrated) in the usePaginatedChannels hook, we could use a reference check condition
   channels === initialChannelsStateRef.current

here is a potential patch

diff --git a/src/components/ChannelList/hooks/usePaginatedChannels.ts b/src/components/ChannelList/hooks/usePaginatedChannels.ts
index 8e021a90434c28b87bf7543d5e0e139100dc3c91..412255345cc7dc2388a00e785ddeceffdb1eb472 100644
--- a/src/components/ChannelList/hooks/usePaginatedChannels.ts
+++ b/src/components/ChannelList/hooks/usePaginatedChannels.ts
@@ -45,7 +45,8 @@ export const usePaginatedChannels = <
 }: Parameters<StreamChatGenerics>) => {
   const { client } = useChatContext<StreamChatGenerics>();
 
-  const [channels, setChannels] = useState<Channel<StreamChatGenerics>[]>([]);
+  const initialChannelsStateRef = useRef([]);
+  const [channels, setChannels] = useState<Channel<StreamChatGenerics>[]>(initialChannelsStateRef.current);
   const [staticChannelsActive, setStaticChannelsActive] = useState<boolean>(true);
   const activeChannels = useActiveChannelsRefContext();
 
@@ -182,11 +183,13 @@ export const usePaginatedChannels = <
       if (enableOfflineSupport) {
         try {
           const channelsFromDB = getChannels({ currentUserId: client.user.id, filters, sort });
+          if(channelsFromDB){
           setChannels(
             client.hydrateActiveChannels(channelsFromDB, {
               staticState: true,
             }),
           );
+          }
         } catch (e) {
           console.warn('Failed to get channels from database: ', e);
         }
@@ -205,7 +208,7 @@ export const usePaginatedChannels = <
     loadingChannels:
       activeQueryType === 'queryLocalDB'
         ? true
-        : activeQueryType === 'reload' && channels.length === 0,
+        : activeQueryType === 'reload' && channels === initialChannelsStateRef.current, //if we have updated the array, then we can show it, even if it is empty
     loadingNextPage: activeQueryType === 'loadChannels',
     loadNextPage,
     refreshing: activeQueryType === 'refresh',
diff --git a/src/store/apis/getChannels.ts b/src/store/apis/getChannels.ts
index ef32154b09fe0b8729673f68f0d064b25bd6711a..5a9f26122285926a7d97f8f09522b5c376cbe044 100644
--- a/src/store/apis/getChannels.ts
+++ b/src/store/apis/getChannels.ts
@@ -19,13 +19,17 @@ export const getChannels = <
   currentUserId: string;
   filters?: ChannelFilters<StreamChatGenerics>;
   sort?: ChannelSort<StreamChatGenerics>;
-}): Omit<ChannelAPIResponse<StreamChatGenerics>, 'duration'>[] => {
+}): Omit<ChannelAPIResponse<StreamChatGenerics>, 'duration'>[] | null => {
   if (!filters && !sort) {
     console.warn('Please provide the query (filters/sort) to fetch channels from DB');
     return [];
   }
 
   const channelIds = selectChannelIdsForFilterSort({ filters, sort });
+  if(!channelIds) {
+    return null;
+  }
+
   const channels = selectChannels({ channelIds });
 
   const cidVsMembers = getMembers<StreamChatGenerics>({ channelIds });
diff --git a/src/store/apis/queries/selectChannelIdsForFilterSort.ts b/src/store/apis/queries/selectChannelIdsForFilterSort.ts
index 89489f7f151cb7622fcc1382feb2775e5f013c3b..82872af3092ba49e46069a25695c2507a0b16992 100644
--- a/src/store/apis/queries/selectChannelIdsForFilterSort.ts
+++ b/src/store/apis/queries/selectChannelIdsForFilterSort.ts
@@ -14,7 +14,7 @@ export const selectChannelIdsForFilterSort = <
 }: {
   filters?: ChannelFilters<StreamChatGenerics>;
   sort?: ChannelSort<StreamChatGenerics>;
-}): string[] => {
+}): string[] | null => {
   const query = convertFilterSortToQuery({ filters, sort });
   const results = QuickSqliteClient.executeSql.apply(
     null,
@@ -24,5 +24,5 @@ export const selectChannelIdsForFilterSort = <
   );
 
   const channelIdsStr = results?.[0]?.cids;
-  return channelIdsStr ? JSON.parse(channelIdsStr) : [];
+  return channelIdsStr ? JSON.parse(channelIdsStr) : null;
 };

[vishalnarkhede]

@andreibarabas could I see the usage of FlashList on your side? There seems to be some differences between SDK implementation of flatlist and your implementation of flashlist. Because I am not able to reproduce this issue unfortunately.

I do see the error indicator at the end of this use case. But we keep showing loading indicator until we get final response from network.

screen-recording.mov

[andreibarabas]

Sure, the "flashlist" we are using is this https://shopify.github.io/flash-list/, which I described how to implement here

#1594

But the bug is not related to the FlashList, but the filters retuning 0 results. i.e: add a filter for "dummy": "none" and you should repro it if you turn on airplane mode.

[vishalnarkhede]

Let me check

[vishalnarkhede]

@andreibarabas you are correct, I am able to reproduce the issue. Let me add a fix for it. Thanks once again.

[andreibarabas]

Sounds good. I look forward to seeing what fix you choose (so i can remove our patch after that)

[andreibarabas]

This is all the feedback for now. The integration is complete and we will roll it out internally and to some of our beta users to see real life usage and get back to you guys.

Thanks for the hard work and look forward to the polishes

[vishalnarkhede]

Thanks alot @andreibarabas for all the feedback. I am going to go through it today and triage them and do the necessary actions.

[andreibarabas]

Use the latest react-native-quick-sqlite@4.x will allow for people to upgrade their app to RN 0.69. Since the offline mode is still in BETA, I would suggest you upgrade the code to make use of this from the start.

The patch to support this is simple

diff --git a/src/store/QuickSqliteClient.ts b/src/store/QuickSqliteClient.ts
index d429e4fbbf7169d02c77939be03477a4fbc718fb..b4e4a5685d9276e0825eaa38c133125b08bd687e 100644
--- a/src/store/QuickSqliteClient.ts
+++ b/src/store/QuickSqliteClient.ts
@@ -5,7 +5,8 @@
 import types from 'react-native-quick-sqlite';
 
 try {
-  require('react-native-quick-sqlite');
+  //@ts-ignore
+  window.sqlite = require('react-native-quick-sqlite').QuickSQLite;
 } catch (e) {
   // Failed for one of the reason
   // 1. Running on expo, where we don't support offline storage yet.

[vanGalilea]

@andreibarabas thanks for your suggestions, we will take a look at it!

[vishalnarkhede]

@andreibarabas we have published https://github.com/GetStream/stream-chat-react-native/releases/tag/v5.0.0-beta.1

[andreibarabas]

Awesome! I'm away for the next 2 weeks, so I won't be able to provide feedback sooner, but will definitely upgrade asap and come back to you guys

[vishalnarkhede]

Quick update, we published 5.0.0-beta.2 ... so when you upgrade please update to this version :)

[vishalnarkhede]

And just to help the case, we have upgraded following dependencies to given version

• stream-chat@7.0.0-offline-support.4
• react-native-quick-sqlite@4.0.5

[andreibarabas]

Found one issue in the stable version, namely: if you have a filter with 0 results saved locally, then go in airplane mode and open the app, after the MAX_QUERY_CHANNELS_LIMIT is hit, the ui will error (although we did load from local database 0 channels). see recording

issue.mp4

The issue comes from this line, as the channels.length is obviously 0

stream-chat-react-native/package/src/components/ChannelList/ChannelListMessenger.tsx

Line 125 in 53a9c41

if (error && !refreshing && !loadingChannels && !channels?.length) {

[vishalnarkhede]

This has been fixed in latest release :)

[billnbell]

I am still having issues with SQL Lite for the caching. I get referential integrity issues.