Crash in AtomicDictionary "Error _ContiguousArrayBuffer.init - EXC_BREAKPOINT (SIGTRAP) Could not allocate memory"

[fzy-github]

Describe the bug

We have observed a crash coming from Amplify SDK using Sentry

OS Version: iOS 17.1.2 (21B101)
Report Version: 104

Exception Type: EXC_BREAKPOINT (SIGTRAP)
Crashed Thread: 13

Application Specific Information:
Could not allocate memory.

Thread 13 Crashed:
0   libswiftCore.dylib              0x304e18f64         swift::swift_slowAllocTyped
1   libswiftCore.dylib              0x304d5916c         swift::swift_slowAllocTyped
2   libswiftCore.dylib              0x304d5dcf8         _swift_allocObject_
3   SampleApp                       0x2011c5164         _ContiguousArrayBuffer.init
4   SampleApp                       0x2011c51f0         [inlined] _ContiguousArrayBuffer.init
5   SampleApp                       0x2011c51f0         [inlined] _copyCollectionToContiguousArray<T>
6   SampleApp                       0x2011c51f0         [inlined] Collection._copyToContiguousArray
7   SampleApp                       0x2011c51f0         [inlined] [A : B].Values
8   SampleApp                       0x2011c51f0         Array.init<T>
9   SampleApp                       0x2011c4cec         [inlined] Array.init<T>
10  SampleApp                       0x2011c4cec         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
11  SampleApp                       0x2011c4cec         AtomicDictionary.values.getter
12  SampleApp                       0x2011c4d24         AtomicDictionary.values.getter
13  SampleApp                       0x2011be1f4         [inlined] thunk for closure
14  SampleApp                       0x2011be1f4         [inlined] thunk for closure
15  SampleApp                       0x2011be1f4         [inlined] NSLocking.execute<T> (NSLocking+Execute.swift:47)
16  SampleApp                       0x2011be1f4         NSLocking.execute<T> (NSLocking+Execute.swift:38)
17  SampleApp                       0x2011c3810         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
18  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher.listeners.getter (HubChannelDispatcher.swift:78)
19  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher (<compiler-generated>:77)
20  SampleApp                       0x2011c3810         HubDispatchOperation.main (HubChannelDispatcher.swift:122)
21  SampleApp                       0x2011c3c0c         HubDispatchOperation.main
22  Foundation                      0x3100aa9c0         __NSOPERATION_IS_INVOKING_MAIN__
23  Foundation                      0x3100a9ab0         -[NSOperation start]
24  Foundation                      0x3100ac5c8         __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__
25  Foundation                      0x3100ac334         __NSOQSchedule_f
26  libdispatch.dylib               0x32207924c         _dispatch_block_async_invoke2
27  libdispatch.dylib               0x32206a2fc         _dispatch_client_callout
28  libdispatch.dylib               0x32206d7b4         _dispatch_continuation_pop
29  libdispatch.dylib               0x32206ce64         _dispatch_async_redirect_invoke
30  libdispatch.dylib               0x32207bbe0         _dispatch_root_queue_drain
31  libdispatch.dylib               0x32207c3e8         _dispatch_worker_thread2
32  libsystem_pthread.dylib         0x3e55ad924         _pthread_wqthread

Steps To Reproduce

No steps to reproduce - caught by crash reporting tools

Expected behavior

No crash

Amplify Framework Version

2.22.0

Amplify Categories

API, DataStore, Storage

Dependency manager

Swift PM

Swift version

5.8

CLI version

12.2.3

Xcode version

Xcode 15.0.1 Build version 15A507

Relevant log output

n/a

Is this a regression?

No

Regression additional context

n/a

Platforms

iOS

OS Version

17.1.2

Device

iPhone XR

Specific to simulators

no

Additional context

No response

[atierian]

Thanks for opening this @fzy-spyro
Please include the full crash report; it has important information that will help us diagnose this. Thanks!

[fzy-github]

@atierian Here's the full crash log:

OS Version: iOS 17.1.2 (21B101)
Report Version: 104

Exception Type: EXC_BREAKPOINT (SIGTRAP)
Crashed Thread: 13

Application Specific Information:
Could not allocate memory.

Thread 13 Crashed:
0   libswiftCore.dylib              0x304e18f64         swift::swift_slowAllocTyped
1   libswiftCore.dylib              0x304d5916c         swift::swift_slowAllocTyped
2   libswiftCore.dylib              0x304d5dcf8         _swift_allocObject_
3   SampleApp                       0x2011c5164         _ContiguousArrayBuffer.init
4   SampleApp                       0x2011c51f0         [inlined] _ContiguousArrayBuffer.init
5   SampleApp                       0x2011c51f0         [inlined] _copyCollectionToContiguousArray<T>
6   SampleApp                       0x2011c51f0         [inlined] Collection._copyToContiguousArray
7   SampleApp                       0x2011c51f0         [inlined] [A : B].Values
8   SampleApp                       0x2011c51f0         Array.init<T>
9   SampleApp                       0x2011c4cec         [inlined] Array.init<T>
10  SampleApp                       0x2011c4cec         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
11  SampleApp                       0x2011c4cec         AtomicDictionary.values.getter
12  SampleApp                       0x2011c4d24         AtomicDictionary.values.getter
13  SampleApp                       0x2011be1f4         [inlined] thunk for closure
14  SampleApp                       0x2011be1f4         [inlined] thunk for closure
15  SampleApp                       0x2011be1f4         [inlined] NSLocking.execute<T> (NSLocking+Execute.swift:47)
16  SampleApp                       0x2011be1f4         NSLocking.execute<T> (NSLocking+Execute.swift:38)
17  SampleApp                       0x2011c3810         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
18  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher.listeners.getter (HubChannelDispatcher.swift:78)
19  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher (<compiler-generated>:77)
20  SampleApp                       0x2011c3810         HubDispatchOperation.main (HubChannelDispatcher.swift:122)
21  SampleApp                       0x2011c3c0c         HubDispatchOperation.main
22  Foundation                      0x3100aa9c0         __NSOPERATION_IS_INVOKING_MAIN__
23  Foundation                      0x3100a9ab0         -[NSOperation start]
24  Foundation                      0x3100ac5c8         __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__
25  Foundation                      0x3100ac334         __NSOQSchedule_f
26  libdispatch.dylib               0x32207924c         _dispatch_block_async_invoke2
27  libdispatch.dylib               0x32206a2fc         _dispatch_client_callout
28  libdispatch.dylib               0x32206d7b4         _dispatch_continuation_pop
29  libdispatch.dylib               0x32206ce64         _dispatch_async_redirect_invoke
30  libdispatch.dylib               0x32207bbe0         _dispatch_root_queue_drain
31  libdispatch.dylib               0x32207c3e8         _dispatch_worker_thread2
32  libsystem_pthread.dylib         0x3e55ad924         _pthread_wqthread

Thread 0
0   libswiftCore.dylib              0x304d5dd54         swift::ConcurrentReadableHashMap<T>::find<T>
1   SwiftUI                         0x31a274620         OUTLINED_FUNCTION_101
2   SwiftUI                         0x31b354a58         OUTLINED_FUNCTION_2
3   SwiftUI                         0x31b350d14         OUTLINED_FUNCTION_2
4   SwiftUI                         0x31b3c1090         OUTLINED_FUNCTION_9
5   SwiftUI                         0x31b3c10f0         OUTLINED_FUNCTION_9
6   UIKitCore                       0x31656e690         -[UIView(CALayerDelegate) layoutSublayersOfLayer:]
7   QuartzCore                      0x314d651bc         CA::Layer::layout_if_needed
8   QuartzCore                      0x314d64d44         CA::Layer::layout_and_display_if_needed
9   QuartzCore                      0x314d6b274         CA::Context::commit_transaction
10  QuartzCore                      0x314d64570         CA::Transaction::commit
11  QuartzCore                      0x314d64218         CA::Transaction::flush_as_runloop_observer
12  CoreFoundation                  0x312129a28         __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__
13  CoreFoundation                  0x312128430         __CFRunLoopDoObservers
14  CoreFoundation                  0x312127b4c         __CFRunLoopRun
15  CoreFoundation                  0x312127664         CFRunLoopRunSpecific
16  GraphicsServices                0x3988ed5e8         GSEventRunModal
17  UIKitCore                       0x3167682b0         -[UIApplication _run]
18  UIKitCore                       0x3167678ec         UIApplicationMain
19  SwiftUI                         0x31ac360f8         OUTLINED_FUNCTION_31
20  SwiftUI                         0x31ac35f3c         OUTLINED_FUNCTION_31
21  SwiftUI                         0x31a8a7864         OUTLINED_FUNCTION_26
22  SampleApp                       0x200d30bf0         [inlined] AppLauncher.main
23  SampleApp                       0x200d30bf0         [inlined] AppLauncher.$main (<compiler-generated>:6)
24  SampleApp                       0x200d30bf0         main
25  <unknown>                       0x1ace82dcc         <redacted>

Thread 1
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 2
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 3
0   libsystem_kernel.dylib          0x3a0b0b9d8         __semwait_signal
1   libsystem_c.dylib               0x3220fdf1c         nanosleep
2   libsystem_c.dylib               0x32210c1d8         sleep
3   SampleApp                       0x2016b92cc         monitorCachedData (SentryCrashCachedData.c:145)
4   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 4 name: SentryCrash Exception Handler (Secondary)
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0f774         thread_suspend
3   SampleApp                       0x2016b4f70         handleExceptions (SentryCrashMonitor_MachException.c:306)
4   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 5 name: SentryCrash Exception Handler (Primary)
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0ae84         mach_msg_overwrite
3   libsystem_kernel.dylib          0x3a0b0acc4         mach_msg
4   SampleApp                       0x2016b4f9c         handleExceptions (SentryCrashMonitor_MachException.c:314)
5   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 6 name: io.sentry.app-hang-tracker
0   libsystem_kernel.dylib          0x3a0b0b9d8         __semwait_signal
1   libsystem_c.dylib               0x3220fdf1c         nanosleep
2   Foundation                      0x3107806c8         +[NSThread sleepForTimeInterval:]
3   SampleApp                       0x20164d8dc         -[SentryANRTracker detectANRs] (SentryANRTracker.m:103)
4   Foundation                      0x3100e5180         __NSThread__start__
5   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 7
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0ae84         mach_msg_overwrite
3   libsystem_kernel.dylib          0x3a0b0acc4         mach_msg
4   Instabug                        0x106ce7d38         exception_server_thread
5   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 8 name: com.apple.NSURLConnectionLoader
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0ae84         mach_msg_overwrite
3   libsystem_kernel.dylib          0x3a0b0acc4         mach_msg
4   CoreFoundation                  0x312129d08         __CFRunLoopServiceMachPort
5   CoreFoundation                  0x312127c00         __CFRunLoopRun
6   CoreFoundation                  0x312127664         CFRunLoopRunSpecific
7   CFNetwork                       0x3144ea77c         _CFURLStorageSessionCopyIdentifier
8   Foundation                      0x3100e5180         __NSThread__start__
9   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 9 name: AwsLogger
0   libsystem_kernel.dylib          0x3a0b0bb7c         __psynch_cvwait
1   libsystem_pthread.dylib         0x3e55acfd0         _pthread_cond_wait
2   SampleApp                       0x201329380         aws_condition_variable_wait (condition_variable.c:74)
3   SampleApp                       0x20131e504         aws_condition_variable_wait_pred (condition_variable.c:16)
4   SampleApp                       0x201326818         aws_background_logger_thread (log_channel.c:157)
5   SampleApp                       0x20132b4f0         thread_fn (thread.c:177)
6   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 10 name: AwsEventLoop 1
0   libsystem_kernel.dylib          0x3a0b18f6c         kevent
1   SampleApp                       0x2013362ac         aws_event_loop_thread (kqueue_event_loop.c:876)
2   SampleApp                       0x20132b4f0         thread_fn (thread.c:177)
3   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 11 name: com.apple.uikit.eventfetch-thread
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0ae84         mach_msg_overwrite
3   libsystem_kernel.dylib          0x3a0b0acc4         mach_msg
4   CoreFoundation                  0x312129d08         __CFRunLoopServiceMachPort
5   CoreFoundation                  0x312127c00         __CFRunLoopRun
6   CoreFoundation                  0x312127664         CFRunLoopRunSpecific
7   Foundation                      0x310060548         -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
8   Foundation                      0x31008e278         -[NSRunLoop(NSRunLoop) runUntilDate:]
9   UIKitCore                       0x3166c9fc4         -[UIEventFetcher threadMain]
10  Foundation                      0x3100e5180         __NSThread__start__
11  libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 12
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 13 Crashed:
0   libswiftCore.dylib              0x304e18f64         swift::swift_slowAllocTyped
1   libswiftCore.dylib              0x304d5916c         swift::swift_slowAllocTyped
2   libswiftCore.dylib              0x304d5dcf8         _swift_allocObject_
3   SampleApp                       0x2011c5164         _ContiguousArrayBuffer.init
4   SampleApp                       0x2011c51f0         [inlined] _ContiguousArrayBuffer.init
5   SampleApp                       0x2011c51f0         [inlined] _copyCollectionToContiguousArray<T>
6   SampleApp                       0x2011c51f0         [inlined] Collection._copyToContiguousArray
7   SampleApp                       0x2011c51f0         [inlined] [A : B].Values
8   SampleApp                       0x2011c51f0         Array.init<T>
9   SampleApp                       0x2011c4cec         [inlined] Array.init<T>
10  SampleApp                       0x2011c4cec         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
11  SampleApp                       0x2011c4cec         AtomicDictionary.values.getter
12  SampleApp                       0x2011c4d24         AtomicDictionary.values.getter
13  SampleApp                       0x2011be1f4         [inlined] thunk for closure
14  SampleApp                       0x2011be1f4         [inlined] thunk for closure
15  SampleApp                       0x2011be1f4         [inlined] NSLocking.execute<T> (NSLocking+Execute.swift:47)
16  SampleApp                       0x2011be1f4         NSLocking.execute<T> (NSLocking+Execute.swift:38)
17  SampleApp                       0x2011c3810         [inlined] AtomicDictionary.values.getter (AtomicDictionary.swift:28)
18  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher.listeners.getter (HubChannelDispatcher.swift:78)
19  SampleApp                       0x2011c3810         [inlined] HubChannelDispatcher (<compiler-generated>:77)
20  SampleApp                       0x2011c3810         HubDispatchOperation.main (HubChannelDispatcher.swift:122)
21  SampleApp                       0x2011c3c0c         HubDispatchOperation.main
22  Foundation                      0x3100aa9c0         __NSOPERATION_IS_INVOKING_MAIN__
23  Foundation                      0x3100a9ab0         -[NSOperation start]
24  Foundation                      0x3100ac5c8         __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__
25  Foundation                      0x3100ac334         __NSOQSchedule_f
26  libdispatch.dylib               0x32207924c         _dispatch_block_async_invoke2
27  libdispatch.dylib               0x32206a2fc         _dispatch_client_callout
28  libdispatch.dylib               0x32206d7b4         _dispatch_continuation_pop
29  libdispatch.dylib               0x32206ce64         _dispatch_async_redirect_invoke
30  libdispatch.dylib               0x32207bbe0         _dispatch_root_queue_drain
31  libdispatch.dylib               0x32207c3e8         _dispatch_worker_thread2
32  libsystem_pthread.dylib         0x3e55ad924         _pthread_wqthread

Thread 14
0   libsystem_kernel.dylib          0x3a0b0e6b0         __guarded_open_dprotected_np
1   libsystem_kernel.dylib          0x3a0b0e698         guarded_open_dprotected_np
2   libsqlite3.dylib                0x364090cf0         sqlite3_sourceid
3   libsqlite3.dylib                0x364007e08         sqlite3_snprintf
4   libsqlite3.dylib                0x36404a8c8         sqlite3_free_table
5   libsqlite3.dylib                0x36404e300         sqlite3_free_table
6   libsqlite3.dylib                0x36404db98         sqlite3_free_table
7   libsqlite3.dylib                0x364032824         sqlite3_step
8   libsqlite3.dylib                0x36402d640         sqlite3_step
9   SampleApp                       0x2014c3084         Statement.step (Statement.swift:185)
10  SampleApp                       0x2014c455c         Statement.step
11  libswiftDispatch.dylib          0x3249a3590         thunk for closure
12  libswiftDispatch.dylib          0x3249a356c         thunk for closure
13  libswiftDispatch.dylib          0x3249a34ac         OS_dispatch_queue._syncHelper<T>
14  libswiftDispatch.dylib          0x3249a33e4         thunk for closure
15  libswiftDispatch.dylib          0x3249a33bc         thunk for closure
16  libdispatch.dylib               0x32206a2fc         _dispatch_client_callout
17  libdispatch.dylib               0x3220796b0         _dispatch_lane_barrier_sync_invoke_and_complete
18  libswiftDispatch.dylib          0x3249a4074         OS_dispatch_queue.asyncAndWait<T>
19  libswiftDispatch.dylib          0x3249a3fac         OS_dispatch_queue.sync<T>
20  libswiftDispatch.dylib          0x3249a3e2c         OS_dispatch_queue._syncHelper<T>
21  libswiftDispatch.dylib          0x3249a3c70         OS_dispatch_queue.asyncAndWait<T>
22  libswiftDispatch.dylib          0x3249a3bdc         OS_dispatch_queue.sync<T>
23  SampleApp                       0x2014c2a9c         [inlined] Connection.sync<T> (Connection.swift:694)
24  SampleApp                       0x2014c2a9c         [inlined] Statement.step (Statement.swift:185)
25  SampleApp                       0x2014c2a9c         Statement.run (Statement.swift:133)
26  SampleApp                       0x2014c1ed0         Statement.run (Statement.swift:143)
27  SampleApp                       0x20122986c         SQLiteStorageEngineAdapter.save<T> (StorageEngineAdapter+SQLite.swift:169)
28  SampleApp                       0x2012291e8         SQLiteStorageEngineAdapter.save<T> (StorageEngineAdapter+SQLite.swift:138)
29  SampleApp                       0x20122d3b0         SQLiteStorageEngineAdapter
30  SampleApp                       0x201297334         ReconcileAndLocalSaveOperation.saveMetadata (ReconcileAndLocalSaveOperation.swift:400)
31  Combine                         0x3249e331c         Future.init
32  Combine                         0x3249e31a8         Future.init
33  SampleApp                       0x201295bc0         [inlined] Future.__allocating_init
34  SampleApp                       0x201295bc0         [inlined] ReconcileAndLocalSaveOperation.saveMetadata (ReconcileAndLocalSaveOperation.swift:394)
35  SampleApp                       0x201295bc0         ReconcileAndLocalSaveOperation.applyRemoteModelsDisposition (ReconcileAndLocalSaveOperation.swift:287)
36  Combine                         0x3249d56a0         Publishers.FlatMap.Outer.receive
37  Combine                         0x3249d5570         Publishers.FlatMap<T>.Outer<T>
38  Combine                         0x3249e0868         Future.Conduit.request
39  Combine                         0x3249de280         ConduitBase<T>
40  Combine                         0x3249e1ce4         Publishers.FlatMap.Outer.receive
41  Combine                         0x3249e1b90         Publishers.FlatMap<T>.Outer<T>
42  Combine                         0x3249e9290         Future.Conduit.subscribe
43  Combine                         0x3249e8cd4         Future.receive<T>
44  Combine                         0x3249e88d8         Future<T>
45  Combine                         0x3249d68b4         Publishers.FlatMap.receive<T>
46  Combine                         0x3249d66dc         Publishers.Map.receive<T>
47  Combine                         0x3249d70c0         Publishers.Catch.receive<T>
48  Combine                         0x3249d5880         PublisherBox.receive<T>
49  Combine                         0x3249d5808         AnyPublisher.receive<T>
50  Combine                         0x3249db344         Publishers.MergeMany.receive<T>
51  Combine                         0x3249efb20         Publishers.Collect.receive<T>
52  Combine                         0x3249f1150         Publisher.sink
53  SampleApp                       0x2012963d0         ReconcileAndLocalSaveOperation.applyRemoteModelsDispositions (ReconcileAndLocalSaveOperation.swift:321)
54  Combine                         0x3249e331c         Future.init
55  Combine                         0x3249e31a8         Future.init
56  SampleApp                       0x201296048         ReconcileAndLocalSaveOperation.applyRemoteModelsDispositions
57  SampleApp                       0x201294670         ReconcileAndLocalSaveOperation.reconcile (ReconcileAndLocalSaveOperation.swift:145)
58  SampleApp                       0x20129ba24         [inlined] thunk for closure
59  SampleApp                       0x20129ba24         thunk for closure
60  Combine                         0x3249d56a0         Publishers.FlatMap.Outer.receive
61  Combine                         0x3249d5570         Publishers.FlatMap<T>.Outer<T>
62  Combine                         0x3249ed644         Publishers.FlatMap.Outer.receiveInner
63  Combine                         0x3249ed480         Publishers.FlatMap.Outer.Side.receive
64  Combine                         0x3249e0868         Future.Conduit.request
65  Combine                         0x3249de280         ConduitBase<T>
66  Combine                         0x3249e9448         Publishers.FlatMap.Outer.receiveInner
67  Combine                         0x3249e9338         Publishers.FlatMap.Outer.Side.receive
68  Combine                         0x3249e9290         Future.Conduit.subscribe
69  Combine                         0x3249e8cd4         Future.receive<T>
70  Combine                         0x3249e88d8         Future<T>
71  Combine                         0x3249d5780         Publishers.FlatMap.Outer.receive
72  Combine                         0x3249d5570         Publishers.FlatMap<T>.Outer<T>
73  Combine                         0x3249d5544         Publishers.SubscribeOn.Inner.receive
74  Combine                         0x3249edec8         Publishers.SubscribeOn<T>.Inner<T>
75  Combine                         0x3249f1b08         Future.Conduit.fulfill
76  Combine                         0x3249f1350         Future.Conduit.offer
77  Combine                         0x3249f11f0         Future.promise
78  Combine                         0x3249f6b8c         ConduitList.forEach
79  Combine                         0x3249f56fc         Future.promise
80  Combine                         0x3249f51ec         Future.init
81  SampleApp                       0x201294f40         ReconcileAndLocalSaveOperation.queryPendingMutations
82  SampleApp                       0x2012a6d84         MutationEvent.pendingMutationEvents (MutationEvent+Query.swift:95)
83  libswift_Concurrency.dylib      0x328551c28         swift::runJobInEstablishedExecutorContext

Thread 15
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 16
0   libsystem_kernel.dylib          0x3a0b0bb7c         __psynch_cvwait
1   libsystem_pthread.dylib         0x3e55acfd0         _pthread_cond_wait
2   SampleApp                       0x201329438         aws_condition_variable_wait_for (condition_variable.c:102)
3   SampleApp                       0x20131e564         aws_condition_variable_wait_for_pred (condition_variable.c:31)
4   SampleApp                       0x201340f5c         aws_host_resolver_thread (host_resolver.c:1082)
5   SampleApp                       0x20132b4f0         thread_fn (thread.c:177)
6   libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 17
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 18
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 19
0   libsystem_pthread.dylib         0x3e55ad9fc         start_wqthread

Thread 20
0   libsystem_kernel.dylib          0x3a0b0b1d8         mach_msg2_trap
1   libsystem_kernel.dylib          0x3a0b0af6c         mach_msg2_internal
2   libsystem_kernel.dylib          0x3a0b0ae84         mach_msg_overwrite
3   libsystem_kernel.dylib          0x3a0b0acc4         mach_msg
4   CoreFoundation                  0x312129d08         __CFRunLoopServiceMachPort
5   CoreFoundation                  0x312127c00         __CFRunLoopRun
6   CoreFoundation                  0x312127664         CFRunLoopRunSpecific
7   Foundation                      0x310060548         -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
8   Foundation                      0x31005e940         -[NSRunLoop(NSRunLoop) run]
9   SwiftUI                         0x31a2774c8         OUTLINED_FUNCTION_101
10  SwiftUI                         0x31a2775fc         OUTLINED_FUNCTION_101
11  Foundation                      0x3100e5180         __NSThread__start__
12  libsystem_pthread.dylib         0x3e55ae4d0         _pthread_start

Thread 21
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 22
0   libsystem_kernel.dylib          0x3a0b13730         __workq_kernreturn
1   libsystem_pthread.dylib         0x3e55ad9ac         _pthread_wqthread

Thread 13 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000000   x1: 0x0000000000000005   x2: 0x0000000000000000   x3: 0x0000000000800000
    x4: 0x0000000002000001   x5: 0x0000000000000003   x6: 0x0000000000000000   x7: 0x0000000000000000
    x8: 0x00000001dc0a8000   x9: 0x0000000183d79a5f  x10: 0xffffffffffffc000  x11: 0x0000000000000000
   x12: 0x0000000000000001  x13: 0x00000000ffffffff  x14: 0x0000000000000001  x15: 0x0000000000000000
   x16: 0xfffffffffffffff1  x17: 0x0000000000000238  x18: 0x0000000000000000  x19: 0x0110080012525e61
   x20: 0x0000000000001ce0  x21: 0x0000000000000007  x22: 0x0110080012525e61  x23: 0x0000000000000020
   x24: 0x000000016fe8abc8  x25: 0x0000000000000007  x26: 0x0000000000001ce0  x27: 0x00000001029404f0
   x28: 0x0000000000000040   fp: 0x000000016fe8a7e0   lr: 0x0000000183c00170   sp: 0x000000016fe8a7b0
    pc: 0x0000000183cbff64 cpsr: 0x0000000040000000



EOF

[lawmicha]

Hi @fzy-spyro, it appears to be happening during DataStore's reconciliation process. Do you have more details such as

• Your GraphQL schema used for DataStore
• The number of models for each model type you are expecting to be syncing down and if you have a sync expression set up
• Any custom DataStore configuration, especially the sync expressions, syncMaxRecords, syncPageSize, syncInterval
• Do you know which devices are running out of memory? and percentage of overall customers impacted?
• Are you able to determine that among the full crash reports, they're have stacks traces including ReconcileAndLocalSaveOperation ? Reconciliation happens at the initial start-up of the sync process and as new data comes in, so it could be that the device ran out of memory and it just happened to be performing a reconciliation and it could be unrelated to DataStore.

[fzy-github]

Hey @lawmicha

Please find responses below:

Your GraphQL schema used for DataStore

yes. Given this is proprietary project I have renamed table names and properties names. Plesee see attached schema file.
schema.txt

It's rather "simple" schema. Its collection of tables where we mostly store pairs of timestaps and values. We do not have any relations set up etc

The number of models for each model type you are expecting to be syncing down and if you have a sync expression set up

Some tables hold only 1 data row per user in dynamo but for some tables they can hold more than 10k rows (without real max number). Data for those tables are coming from external IoT sensores hence the big amout of rows.
Re. sync expressions - yes, we have started using them recently (some use cases are described in related ticket - #3383) and observed this crash.

Any custom DataStore configuration, especially the sync expressions, syncMaxRecords, syncPageSize, syncInterval
Here's the plugin initialization code:

try Amplify.add(plugin: AWSDataStorePlugin(
            modelRegistration: DataStoreModels(),
            configuration: .custom(errorHandler: onError, syncMaxRecords: UInt.max, syncExpressions: syncExpressions)
        ))
        
        // onError -> this is logger only, no additional logic: 

Sync expressions are configured depending of fact if we expect lot of data to be stored in particular table - in such case we try to limit amout of data that is synced at first to only sync last x days of data and then we re-evaluate experssions at runtme to fetch remaining data:

// for "data-heavy" tables we have following expressions set up
DataStoreSyncExpression.syncExpression(Table2.schema, where: {
                if let dateLimit = dateLimit {
                    Table2.keys.user_id.eq(userId).and(Table2.keys.data_timestamp.gt(dateLimit.temporalDateTime))
                } else {
                    Table2.keys.user_id.eq(userId)
                }
            }),

other tables simply use :

DataStoreSyncExpression.syncExpression(Table7.schema, where: {
                Table7.keys.user_id.eq(userId)
            }),

Do you know which devices are running out of memory? and percentage of overall customers impacted?

We have quite low user base for now (~10-15) and following crash happend for ~20%. looking at device breakdown it happened mostly on iPhone XR, iPhone SE running iOS 17.

Are you able to determine that among the full crash reports, they're have stacks traces including ReconcileAndLocalSaveOperation ? Reconciliation happens at the initial start-up of the sync process and as new data comes in, so it could be that the device ran out of memory and it just happened to be performing a reconciliation and it could be unrelated to DataStore.

unfortuantely I do not have such logs to confirm it. If anything changes I'll update you with logs.

[lawmicha]

Thanks for the info @fzy-spyro, are you able to capture any data related to the memory usage? With your dataset, are you able to check the memory usage during

1. DataStore.clear(), DataStore.start(), re-evaluate sync expression, stop, start. To perform a full sync of your data set.
2. Repeat 1 without relaunching the app. This is to understand if memory usage is sustainable and freed properly.

Are you able to implement any low on memory notifications and correlate that with the crash instances?

[lawmicha]

I tried to create a repro using your schema, and populating about 10k items in Table1, Table2, Table3, and 1000 in the rest (except for I think one or two tables that were using the userId as primary key). I was testing the memory usage around syncing down 35k+ items and it appears to operate fine. It spikes up to around 100MB in memory when the data is pulled into DataStore, and reconciles them into the local database. Each reconciliation operation operates on about 1k items at a time, which takes about 2-3s, so syncing everything (my dataset of 35k items) took approx 2mins.

Could it be possible that the app has been running for some time, and another process has a memory leak leading to up to this crash?

[fzy-github]

Yes, it is possible that app has been running for some time either in foreground or background.

We were not yet able to reproduce this issue on our test devices unfortunately. As crash in the description happened on customer devices it will be quite hard for us to get the actual devices to run memory profiler.

I will add more logging and low-mem events to see if this happens around start/stop calls etc and come back to you once we have more insights.

[lawmicha]

@fzy-spyro, were you able to gather any more information regarding low-mem events?