Update to core 14

CHANGELOG.md

@@ -1,7 +1,14 @@
 ## vNext (TBD)
 
+**File format version bumped. Old files will be automatically upgraded but cannot be downgraded and opened with older versions of the .NET SDK.**
+
 ### Breaking Changes
 * Added automatic serialization and deserialization of Realm classes when using methods on `MongoClient.Collection`, without the need to annotate classes with `MongoDB.Bson`attributes. This feature required to change the default serialization for various types (including `DateTimeOffset`). If you prefer to use the previous serialization, you need to call `Realm.SetLegacySerialization` before any kind of serialization is done, otherwise it may not work as epxected. [#3459](https://github.com/realm/realm-dotnet/pull/3459)
+* Support for upgrading from Realm files produced by RealmCore v5.23.9 (Realm .NET v5.0.1) or earlier is no longer supported. (Core 14.0.0-beta.0)
+* `String` and `byte[]` are now strongly typed for comparisons and queries. This change is especially relevant when querying for a string constant on a `RealmValue` property, as now only strings will be returned. If searching for binary data is desired, then that type must be specified by the constant. In RQL (`.Filter()`) the new way to specify a binary constant is to use `RealmValueProp = bin('xyz')` or `RealmValueProp = binary('xyz')`. (Core 14.0.0-beta.0)
+* Sorting order of strings has changed to use standard unicode codepoint order instead of grouping similar english letters together. A noticeable change will be from "aAbBzZ" to "ABZabz". (Core 14.0.0-beta.0)
+* In RQL (`Filter()`), if you want to query using `@type` operation, you must use `objectlink` to match links to objects. `object` is reserved for dictionary types. (Core 14.0.0)
+* Opening realm with file format 23 or lower (Realm .NET versions earlier than 12.0.0) in read-only mode will crash. (Core 14.0.0)
 
 ### Enhancements
 * Add support for passing a key paths collection (`KeyPathsCollection`) when using `IRealmCollection.SubscribeForNotifications`. Passing a `KeyPathsCollection` allows to specify which changes in properties should raise a notification.
@@ -11,7 +18,6 @@
   - building it implicitly with the conversion from a `List` or array of `KeyPath` or strings;
   - getting one of the static values `Full` and `Shallow` for full and shallow notifications respectively.
 
-
   For example:
   ```csharp
   var query = realm.All<Person>();
@@ -26,15 +32,49 @@
   ```
   (PR [#3501 ](https://github.com/realm/realm-dotnet/pull/3501))
 * Added the `MongoClient.GetCollection<T>` method to get a collection of documents from MongoDB that can be deserialized in Realm objects. This methods works the same as `MongoClient.GetDatabase(dbName).GetCollection(collectionName)`, but the database name and collection name are automatically derived from the Realm object class.  [#3414](https://github.com/realm/realm-dotnet/pull/3414)
+* Improved performance of RQL (`.Filter()`) queries on a non-linked string property using: >, >=, <, <=, operators and fixed behaviour that a null string should be evaulated as less than everything, previously nulls were not matched. (Core 13.27.0)
+* Updated bundled OpenSSL version to 3.2.0. (Core 13.27.0)
+* Storage of Decimal128 properties has been optimised so that the individual values will take up 0 bits (if all nulls), 32 bits, 64 bits or 128 bits depending on what is needed. (Core 14.0.0-beta.0)
+* Add support for collection indexes in RQL (`Filter()`) queries.
+  For example:
+  ```csharp
+  var people = realm.All<Person>();
+
+  //People whose first dog is called "Fluffy"
+  var query1 = people.Filter("ListOfDogs[FIRST].Name = $0", "Fluffy")
+
+  //People whose last dog is called "Fluffy"
+  var query2 = people.Filter("ListOfDogs[LAST].Name = $0", "Fluffy")
+
+  //People whose second dog is called "Fluffy"
+  var query3 = people.Filter("ListOfDogs[2].Name = $0", "Fluffy")
+
+  //People that have a dog called "Fluffy"
+  var query4 = people.Filter("ListOfDogs[*].Name = $0", "Fluffy")
+
+  //People that have 3 dogs
+  var query5 = people.Filter("ListOfDogs[SIZE] = $0", 3)
+  ```
+  (Core 14.0.0)
 
 ### Fixed
-* None
+* Fixed RQL (`.Filter()`) queries like `indexed_property == NONE {x}` which mistakenly matched on only x instead of not x. This only applies when an indexed property with equality (==, or IN) matches with `NONE` on a list of one item. If the constant list contained more than one value then it was working correctly. (Core 13.27.0)
+* Uploading the changesets recovered during an automatic client reset recovery may lead to 'Bad server version' errors and a new client reset. (Core 13.27.0)
+* Fixed crash in fulltext index using prefix search with no matches. (Core 13.27.0)
+* Fixed a crash with Assertion `failed: m_initiated` during sync session startup. (Core 13.27.0)
+* Fixed a TSAN violation where the user thread could race to read m_finalized with the sync event loop. (Core 13.27.0)
+* Fix a minor race condition when backing up Realm files before a client reset which could have lead to overwriting an existing file. (Core 13.27.0)
+* Boolean property `ChangeSet.IsCleared` that is true when the collection gets cleared is now also raised for `IDictionary`, aligning it to `ISet` and `IList`. (Core 14.0.0-beta.0)
+* Fixed equality queries on `RealmValue` properties with an index. (Core 14.0.0-beta.0)
+* Fixed a crash that would happen when more than 8388606 links were pointing to a specific object.
+* Fixed wrong results when querying for `NULL` value in `IDictionary`. (Core 14.0.0-beta.0)
+* A Realm generated on a non-apple ARM 64 device and copied to another platform (and vice-versa) were non-portable due to a sorting order difference. This impacts strings or binaries that have their first difference at a non-ascii character. These items may not be found in a set, or in an indexed column if the strings had a long common prefix (> 200 characters). (Core 14.0.0-beta.0)
 
 ### Compatibility
-* Realm Studio: 13.0.0 or later.
+* Realm Studio: 15.0.0 or later.
 
 ### Internal
-* Using Core x.y.z.
+* Using Core 14.1.0.
 
 ## 11.7.0 (2024-02-05)
 

Tests/Realm.Tests/Database/CollectionTests.cs

@@ -589,6 +589,73 @@ public void ListFilter_CanBeFilteredWithStringPredicate()
             Assert.That(rDogs.ElementAt(2).Name, Is.EqualTo("Rick"));
         }
 
+        [Test]
+        public void ListFilter_SupportsCollectionIndexes()
+        {
+            var joe = new Owner
+            {
+                Name = "Joe",
+                ListOfDogs =
+                {
+                    new Dog { Name = "Fluffy" },
+                    new Dog { Name = "Plumpy" },
+                }
+            };
+
+            var mark = new Owner
+            {
+                Name = "Mark",
+                ListOfDogs =
+                {
+                    new Dog { Name = "Plumpy" },
+                    new Dog { Name = "Cuddly" },
+                    new Dog { Name = "Fluffy" },
+                }
+            };
+
+            var anton = new Owner
+            {
+                Name = "Anton",
+            };
+
+            _realm.Write(() =>
+            {
+                _realm.Add(joe);
+                _realm.Add(mark);
+                _realm.Add(anton);
+            });
+
+            var owners = _realm.All<Owner>();
+
+            var query = owners.Filter("ListOfDogs[0].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(joe.Name));
+
+            query = owners.Filter("ListOfDogs[FIRST].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(joe.Name));
+
+            query = owners.Filter("ListOfDogs[LAST].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(mark.Name));
+
+            query = owners.Filter("ListOfDogs[LAST].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(mark.Name));
+
+            query = owners.Filter("ListOfDogs[*].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(2));
+            Assert.That(query.ToArray().Select(p => p.Name).OrderBy(p => p), Is.EqualTo(new[] { joe.Name, mark.Name }));
+
+            query = owners.Filter("ListOfDogs[SIZE] = $0", 0);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(anton.Name));
+
+            query = owners.Filter("ListOfDogs[SIZE] = $0", 3);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(mark.Name));
+        }
+
         [Test]
         public void ListFilter_WhenNotRealmList_Throws()
         {
@@ -903,6 +970,66 @@ public void SetFilter_CanBeFilteredWithStringPredicate()
             Assert.That(rDogs.ElementAt(2).Name, Is.EqualTo("Rick"));
         }
 
+        [Test]
+        public void SetFilter_SupportsCollectionIndexes()
+        {
+            var joe = new Owner
+            {
+                Name = "Joe",
+                SetOfDogs =
+                {
+                    new Dog { Name = "Fluffy" },
+                    new Dog { Name = "Plumpy" },
+                }
+            };
+
+            var mark = new Owner
+            {
+                Name = "Mark",
+                SetOfDogs =
+                {
+                    new Dog { Name = "Plumpy" },
+                    new Dog { Name = "Cuddly" },
+                    new Dog { Name = "Fluffy" },
+                }
+            };
+
+            var anton = new Owner
+            {
+                Name = "Anton",
+            };
+
+            _realm.Write(() =>
+            {
+                _realm.Add(joe);
+                _realm.Add(mark);
+                _realm.Add(anton);
+            });
+
+            var owners = _realm.All<Owner>();
+
+            var query = owners.Filter("SetOfDogs[SIZE] = $0", 0);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(anton.Name));
+
+            query = owners.Filter("SetOfDogs[SIZE] = $0", 3);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(mark.Name));
+
+            query = owners.Filter("SetOfDogs[*].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(2));
+            Assert.That(query.ToArray().Select(p => p.Name).OrderBy(p => p), Is.EqualTo(new[] { joe.Name, mark.Name }));
+
+            Assert.That(() => owners.Filter("SetOfDogs[FIRST].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[FIRST] not expected"));
+
+            Assert.That(() => owners.Filter("SetOfDogs[LAST].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[LAST] not expected"));
+
+            Assert.That(() => owners.Filter("SetOfDogs[2].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[2] not expected"));
+        }
+
         [Test]
         public void SetFilter_WhenNotRealmSet_Throws()
         {
@@ -1204,7 +1331,9 @@ public void QueryFilter_WithWrongArguments_ShouldThrow(StringQueryTestData data)
             });
 
             var ex = Assert.Throws<ArgumentException>(() => _realm.All<AllTypesObject>().Filter($"{data.PropertyName} = $0", data.NonMatchingValue))!;
-            Assert.That(ex.Message, Does.Contain($"Unsupported comparison"));
+            Assert.That(ex.Message, Does.Contain($"Cannot compare argument")
+                .Or.Contain("Unsupported comparison")
+                .Or.Contain("Cannot convert"));
         }
 
         [Test]

Tests/Realm.Tests/Database/NotificationTests.cs

@@ -1593,7 +1593,7 @@ public void DictionaryOfObjects_SubscribeForNotifications_DoesntReceiveModificat
                 dict.Clear();
             });
 
-            VerifyNotifications(changesets, expectedDeleted: new[] { 0 }, expectedCleared: false);
+            VerifyNotifications(changesets, expectedDeleted: new[] { 0 }, expectedCleared: true);
         }
 
         [Test]
@@ -1641,7 +1641,7 @@ public void DictionaryOfPrimitives_SubscribeForNotifications_ShallowHasNoEffect(
                 dict.Clear();
             });
 
-            VerifyNotifications(changesets, expectedDeleted: new[] { 0 }, expectedCleared: false);
+            VerifyNotifications(changesets, expectedDeleted: new[] { 0 }, expectedCleared: true);
         }
 
         #region Keypath filtering

Tests/Realm.Tests/Database/RealmDictionaryTests.cs

@@ -1313,6 +1313,66 @@ public void DictionaryFilter_NoArguments_Throws()
                 Throws.TypeOf<ArgumentException>().And.Message.Contains("Request for argument at index 0 but no arguments are provided"));
         }
 
+        [Test]
+        public void DictionaryFilter_SupportsCollectionIndexes()
+        {
+            var joe = new Owner
+            {
+                Name = "Joe",
+                DictOfDogs =
+                {
+                    { "a", new Dog { Name = "Fluffy" } },
+                    { "b", new Dog { Name = "Plumpy" } },
+                }
+            };
+
+            var mark = new Owner
+            {
+                Name = "Mark",
+                DictOfDogs =
+                {
+                    { "a", new Dog { Name = "Plumpy" } },
+                    { "b", new Dog { Name = "Cuddly" } },
+                    { "c", new Dog { Name = "Fluffy" } },
+                }
+            };
+
+            var anton = new Owner
+            {
+                Name = "Anton",
+            };
+
+            _realm.Write(() =>
+            {
+                _realm.Add(joe);
+                _realm.Add(mark);
+                _realm.Add(anton);
+            });
+
+            var owners = _realm.All<Owner>();
+
+            var query = owners.Filter("DictOfDogs[SIZE] = $0", 0);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(anton.Name));
+
+            query = owners.Filter("DictOfDogs[SIZE] = $0", 3);
+            Assert.That(query.Count, Is.EqualTo(1));
+            Assert.That(query.Single().Name, Is.EqualTo(mark.Name));
+
+            query = owners.Filter("DictOfDogs[*].Name = $0", "Fluffy");
+            Assert.That(query.Count, Is.EqualTo(2));
+            Assert.That(query.ToArray().Select(p => p.Name).OrderBy(p => p), Is.EqualTo(new[] { joe.Name, mark.Name }));
+
+            Assert.That(() => owners.Filter("DictOfDogs[FIRST].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[FIRST] not expected"));
+
+            Assert.That(() => owners.Filter("DictOfDogs[LAST].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[LAST] not expected"));
+
+            Assert.That(() => owners.Filter("DictOfDogs[2].Name = $0", "Fluffy"), Throws.TypeOf<ArgumentException>()
+                .And.Message.Contains("[2] not expected"));
+        }
+
         private static void RunUnmanagedTests<T>(Func<DictionariesObject, IDictionary<string, T>> accessor, TestCaseData<T> testData)
         {
             TestHelpers.RunAsyncTest(async () =>

Tests/Realm.Tests/Database/RealmResults/SortingTests.cs

@@ -348,7 +348,7 @@ public void SortsByAcceptedOrder()
                 }
             });
             var sortedCities = _realm.All<Cities>().OrderBy(c => c.Name).ToList().Select(c => c.Name);
-            Assert.That(sortedCities, Is.EqualTo(new[] { "A-Place", "A Place", "Santo Domingo", "São Paulo", "Shanghai", "Sydney", "Åby" }));
+            Assert.That(sortedCities, Is.EqualTo(new[] { "A Place", "A-Place", "Santo Domingo", "Shanghai", "Sydney", "São Paulo", "Åby" }));
         }
 
         [TestCase(true)]

Tests/Realm.Tests/Database/RealmValueTests.cs

@@ -1161,7 +1161,7 @@ private static string ConvertRealmValueTypeToFilterAttribute(RealmValueType rvt)
                 RealmValueType.Double => "double",
                 RealmValueType.Decimal128 => "decimal",
                 RealmValueType.ObjectId => "objectid",
-                RealmValueType.Object => "object",
+                RealmValueType.Object => "objectlink",
                 RealmValueType.Guid => "uuid",
                 _ => throw new NotImplementedException(),
             };

Tests/Realm.Tests/Sync/SyncTestBase.cs

@@ -275,7 +275,7 @@ protected PartitionSyncConfiguration GetFakeConfig(App? app = null, string? user
 
             if (setFakeSyncRoute)
             {
-                SetFakeSyncRoute(app);
+                SetFakeSyncRoute(user.App);
             }
 
             return UpdateConfig(new PartitionSyncConfiguration(Guid.NewGuid().ToString(), user, optionalPath));
@@ -288,7 +288,7 @@ protected FlexibleSyncConfiguration GetFakeFLXConfig(App? app = null, string? us
 
             if (setFakeSyncRoute)
             {
-                SetFakeSyncRoute(app);
+                SetFakeSyncRoute(user.App);
             }
 
             return UpdateConfig(new FlexibleSyncConfiguration(user, optionalPath));

wrappers/src/app_cs.cpp

@@ -296,26 +296,20 @@ extern "C" {
     }
 
     REALM_EXPORT void shared_app_reset_for_testing(SharedApp& app) {
+
+        // If the logger is empty then tear_down_for_testing has been called already
+        if (!app->sync_manager()->get_logger()) {
+            return;
+        }
+
         auto users = app->all_users();
         for (size_t i = 0; i < users.size(); i++) {
             auto &user = users[i];
             user->log_out();
         }
 
-        while (app->sync_manager()->has_existing_sessions()) {
-            sleep_ms(5);
-        }
-
-        bool did_reset = false;
-        while (!did_reset) {
-            try {
-                app->sync_manager()->reset_for_testing();
-                did_reset = true;
-            }
-            catch (...) {
-
-            }
-        }
+        // This method will crash the application if called more than once for the same app.
+        app->sync_manager()->tear_down_for_testing();
 
         App::clear_cached_apps();
     }