Co-authored-by: Christian Melchior <christian.melchior@mongodb.com>
Co-authored-by: James Stone <james.stone@mongodb.com>
Co-authored-by: realm-ci <ci@realm.io>
Co-authored-by: Kirill Burtsev <kirill.burtsev@mongodb.com>
Co-authored-by: Daniel Tabacaru <96778637+danieltabacaru@users.noreply.github.com>
Co-authored-by: Thomas Goyne <tg@realm.io>
Co-authored-by: Thomas Goyne <thomas.goyne@mongodb.com>
Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com>
Co-authored-by: Michael Wilkerson-Barker <michael.wilkersonbarker@mongodb.com>
Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com>
fix entries that went to the wrong change version (#6632)
fix a race in a test (#6651)
Fix a lock order inversion in tests (#6666)
Fix an assertion failure if an async write callback ran during a write transaction (#6661)

* Add initial support for a Bindgen opt-in list.

* Update CMakeLists with opt-in option.

* Move calling of 'bindModel()' to Bindgen.

* Remove 'Readonly' return types from getters.

* Add comment to 'opt in' command option.

* Move logic of applying the opt-in list and require consumers to invoke it.

* Update doc comments.

* Validate parsed yaml file.

* Update name in 'OptInSpec' type.

* Revert 'Validate parsed yaml file'.

We'll instead have the binding generator generate the opt-in spec schema file from spec.yaml in order to list all classes and records with their corresponding methods and fields. The validation would then also provide autocomplete in the opt-in spec file.

* Update pointer to 'external/catch'.

* Update error message for missing method/field.

Release 13.15.1

* Updated release notes

* Update catch2 v3.3.2 (#6605)

* Make core infer platform and cpu_arch, while bundle_id must be provided by SDK's (#6612)

* platform and cpu_arch are inferred by core, bundle_id must be provided by SDK's

* update changelog

* Return proper value for X86_64 arch

Co-authored-by: Christian Melchior <christian.melchior@mongodb.com>

* Get fine-grained platform for Apple devices

* Fix tests

* small fixes

* fix more tests

* Fix mistake in changelog

---------

Co-authored-by: Christian Melchior <christian.melchior@mongodb.com>

* use consistent rounding, following SERVER-75392 (#6477)

* fix entries that went to the wrong change version (#6632)

* Special-case main thread runloop scheduler

* Improve SectionedResults performance

SectionedResults used a std::map in a few places where the keys are a dense
range (i.e. always [0..map.size())) and so they can be std::vector instead. The
maps keyed on Mixed are now std::unordered_map.

Change notifications now report changes as a `std::vector<IndexSet>` rather
than `std::map<size_t, IndexSet>`. This is slower and uses more memory when the
only sections that changed are near the end of a SectionedResults with a large
number of sections, but is much faster if all sections changed or if the
sections which changed are early in the SectionedResults. Change notifications
now reuse buffers, which increases persistent memory usage slightly but
significant reduces allocations.

Change notifications for a single section now only compute the changes for that
section rather than computing the full changes and then filtering out the
changes for other sections.

* use static_assert rather than a old home rolled one

* fix warning of redefine of CHECK macro

* fix unused function warning

* silence warnings in bid128_to_string

* Introduce BPlusTree::for_all

* Prevent program from crashing when removing backlinks

* Fix broken snapshot of collection of objects

* Fix importing Results with deleted collection

The result should be an empty result, not the whole table.

* geospatial validation of polygons (#6607)

* geospatial validation of polygons

* Loop->Ring, added tests

* use std::unique

* changelog

* Benchmark for full-text search

* Allow to filter benchmark and run only list of specified names
* Add simple benchmark for fulltext search with index

* Filter out unresolved links in Dictionary::get_any()

* Add support for early exit in BPlusTree::for_all()

* Geospatial feedback (#6645)

* verify local results match a server query

* disallow geowithin on top level tables

* fix geo queries with ANY/ALL/NONE

* geospatial validation of points

* rename GeoCenterSphere -> GeoCircle

* review feedback

* better testing and fix any/all/none geospatial

* format

* Geospatial basic queries benchmarks (#6621)

* Add basic benchmarks for Geospatial type and queries

* Less copying in GeoWithinCompare

* Bring back caching of s2 region into Geospatial

* remove transaction overhead from measurements

* a couple small optimizations

* formatting

* simplify geospatial query evaluations

* changelog

---------

Co-authored-by: James Stone <james.stone@mongodb.com>

* Updated baas server tag for CI (#6650)

* Prepare release

* Updated release notes

* Access token refresh for websockets was not updating the location metadata (#6631)

* Always refresh metadata on app login
* Updated changelog
* Always update location when requested; fix c_api test
* Update test to properly evaluate websocket redirections; added one more test
* Updated changelog and fixed compile warning
* Added location checks back to test
* added mutex locking around location updated state and reworked requesting location update to use flag
* clang format and fix incorrect timeout value
* Reworked update location logic a bit and removed unused function
* Free mutex before calling completion on early exit in init_app_metadata

* maybe fix a race in a test (#6651)

* Use std::optional to store cached leaves in query nodes (#6653)

Our use of aligned_storage was basically a complicated manual version of this.
I was hoping this'd have binary size benefits, but it ended up making the
library 100 bytes larger instead. Nonetheless, it greatly simplifies things.

* Fix a few UBSan failures hit by tests

* Avoid performing unaligned reads in Array::get_chunk()

* Fix a lock order inversion in tests (#6666)

The cycle was DaemonThread::m_running_on_change_mutex =>
RealmCoordinator::m_realm_mutex  => SyncManager::m_mutex  =>
RealmCoordinator::s_coordinator_mutex  =>
DaemonThread::m_running_on_change_mutex, and it happened due to
DaemonThread::remove() being called inside RealmCoordinator::clear_cache()
while holding s_coordinator_mutex. Fortunately we don't actually need to be doing that.

As the cycle required RealmCoordinator::clear_all_caches(), this was only
applicable to tests.

* Allow geo coordinate numeric argument substitutions (#6663)

* allow geo coordinate numeric argument substitutions

* review feedback

* explicit cast to address warning

* Remove catch() clause to prevent truncating stack trace in AsyncOper::do_recycle_and_execute() (#6667)

* Fix an assertion failure if an async write callback ran during a write transaction (#6661)

Between when the callback after acquiring the write lock is scheduled and when
it's invoked a synchronous write transaction can be begun, and if it's not
ended before the next time the scheduler gets to run, the scheduled callback
will be invoked inside the write. When this happens we want to just do nothing.
Ending the synchronous write transaction will take care of rescheduling the
async write it preempted.

* core release 13.13.0

* Updated release notes

* Allocate arguments for lists (#6674)

* Small documentation and code fixes (#6672)

* Fix crash when opening FLX realm after client reset failure (#6671)

* Fix crash when opening FLX realm after client reset failure

* Update changelog

* Don't superceed pending subscriptions in case of a client reset failure

* Add test

* Changes after code review

* Support sorting based on values from a dictionary (#5311)

Co-authored-by: Sebastian Valle <svalle@gascaribe.com>
Co-authored-by: James Stone <james.stone@mongodb.com>

* Filter out external sources from Eclipse (#6682)

Indexer has a hard time dealing with Catch2

* Use cross-compilers instead of CentOS image (#6559)

* Use cross-compilers instead of CentOS image

* changelog

* fix bad merge

* refactor toolchain files

* clarify useToolchain exception circumstances

* Remap github URL to ssh to fix BAAS dependency using https:// (#6685)

* core v13.14.0

* Updated release notes

* Switch to building with Xcode 14 (#6647)

* better fix explanation in the changelog for list of args in the query parser (#6692)

* Remove constructor for GeoPoint and GeoPolygon (#6679)

Co-authored-by: Mathias Stearn <redbeard0531@gmail.com>

* Fix failing "sync: non-synced metadata table doesn't result in non-additive schema change" tests (#6697)

* Reporting correct error message on HTTP errors for Browser target

* User/Server API key provider becomes a single 'API key' provider (#6696)

* Allow frozen Realms to be opened with additive schema changes (#6693)

* allow frozen Realms to be opened with additive schema changes

* lint

* strengthen tests and comments

* Update src/realm/object-store/shared_realm.cpp

Co-authored-by: Thomas Goyne <thomas.goyne@mongodb.com>

---------

Co-authored-by: Thomas Goyne <thomas.goyne@mongodb.com>

* Reverted minimum swift version to fix failing CI tests (#6706)

* core release v13.15.0

* Updated release notes

* Fix client reset test with invalid query (#6711)

* Fix SessionWrapper use-after-free crash when tearing down sessions (#6676)

* Changed SessionWrapper pointer to bind_ptr; added session ident history
* Fix teardown if client is destroyed before session
* Session no longer holds bind_ptr to SessionWrapper; reverted some changes
* Fixed return and updated some comments
* Don't process errors if session is shutting down
* Added extra checks for session state
* Updates from review
* Updated some finalized checks
* Rolled back some changes
* Added output to ASSERTS and moved session history to unordered_set
* Remove session history entry on normal close
* Updated comment in sync tests

* Add [baas] and [local] tags to object store sync tests to identify the tests that rely on BAAS or not (#6710)

* Use Columns<Link> when property is Dictionary of links (#6705)

If a Dictionary property has links as value type, we can use Columns<Link> to handle
the links instead of the basic Columns<Dictionary>. This has the effect that when we
compare with a single value, we will optimize to use LinksToNode. So we need to make
LinksToNode handle the Dictionary case.

When we compare with a list of links, we must ensure that the list is converted to
a list obj ObjKeys - which is the type that Column<Link> evaluates to.

 Use LinksToNode for lists in QueryParser

* better changelog message for the fix related to queries with list of arguments (#6717)

* Fixes for Emscripten target (Passing header from fetch response. Using Config.path for inMemory Realm) (#6716)

* Fixes for Emscripten target: Passing header for fetch response. Passing the RealmConfig.path to be used for inMemory Realm, this is needed for registering SyncSession

Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com>

* release 13.15.1

* Updated spec.yml to remove User & Server prefix from ApiKey credentials

---------

Co-authored-by: James Stone <james.stone@mongodb.com>
Co-authored-by: realm-ci <ci@realm.io>
Co-authored-by: Kirill Burtsev <kirill.burtsev@mongodb.com>
Co-authored-by: Daniel Tabacaru <96778637+danieltabacaru@users.noreply.github.com>
Co-authored-by: Christian Melchior <christian.melchior@mongodb.com>
Co-authored-by: Thomas Goyne <tg@realm.io>
Co-authored-by: Thomas Goyne <thomas.goyne@mongodb.com>
Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com>
Co-authored-by: Michael Wilkerson-Barker <michael.wilkersonbarker@mongodb.com>
Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com>
Co-authored-by: Sebastian Valle <svalle@gascaribe.com>
Co-authored-by: Yavor Georgiev <fealebenpae@users.noreply.github.com>
Co-authored-by: Ferdinando Papale <4850119+papafe@users.noreply.github.com>
Co-authored-by: Mathias Stearn <redbeard0531@gmail.com>
Co-authored-by: Nabil Hachicha <nabil.hachicha@gmail.com>
Co-authored-by: Finn Schiermer Andersen <finn.schiermer.andersen@gmail.com>

* fix slightly different behaviour of GeoBox

* Add back GeoBox to a contained type of Geospatial

* fix invalid variant access in tests

* lint

* add GeoBox::from_polygon

* avoid creating/setting a full text index for primary key columns and vice versa

Replace TableView and limit with a single QueryStateBase object. This will
in principle allow us to call do_find_all with an alternative QueryState
object with some enhanced functionality.

This change triggered some other small refactorings.

* Added pre-v8 protocol support for Sync Server
* Simplified changes based on review comments

This error was introduced in 601805b, where the destructor on
Mixed was removed. It was not possible to actually prove that Valgrind
was right in reporting a read from uninitialized memory, but changing
the logic a bit could make the error go away. The new code is
probably slightly slower, but it is not on the critical path.

* Expose sync proxy configuration

Make a single implementation that will work for all types. This is to
ensure consistent behavior.

Merge Release/13.15.2

Add support for building for visionOS

#6722)

* Added `subscription_initializer` and `rerun_init_subscription_on_open ` 
* Testing

* a more detailed error message for SystemError exceptions

* update error message

* fix windows message

realm core v13.16.0

* bump baas version and fix test

* compare lower case error messages

* open frozen before Realm with correct schema

* fix client reset/async open with added columns and recover schema additions

* changelog

* relax assertion and handle non-notify-before case for sync tests

* fix issues revealed in tests

* flag is required, fix benchmarks

* lint

* change app id and partition

* debug on CI help; log test file cleanup

* fix race to delete file in test

* CI debug open sessions path

* try early shutdown of sync sessions

* possibly fix lingering sync sessions in tests

* review feedback addressed

* Use a thread safe reference in callback

* don't use a scheduler when opening a Realm during client reset

* touch up tests

* touch up tests

* don't copy construct a TestFile

* fix a use after free error in tests

* don't run a notifier that uses out of scope variables

* fix UB on async open error

core v13.16.1

* Update query error assertion

* Update baas version

SystemError is used in two places: as a base class for SyncError, and for
filesystem errors. Filesystem errors don't include the actual error message and
need it appended, but sync errors do and appending the message from the error
code just results in it being duplicated.

…lm-core into nc/fix_timestamp_multiplatform

* first pass at merging changes from PR #5616
* Fixed shellcheck errors in other scripts
* Additional script updates to run baas on remote host
* Additional script updates
* Fixed missing object store tests and baas scripts improvements
* Updated all variants to use baas on a remote host
* ssh options on linux seem to be case sensitive...
* Formatting updates and wait for 2 ssh "hello"'s before continuing
* Set up /data directory using secondary data device
* Fix baas-remote path for 'upload test results'
* Removed build_libuv.sh script since libuv is now built by CMake
* Added comments to disabled windows tests; updated github known hosts
* Rolled back pinned baas version due to CI failures with updated version

For complicated performance reasons, Cocoa doesn't pass the schema in the
RealmConfig and instead calls update_schema() later. This means that when a
client reset happens during an async open, the object store code doesn't have
access to the actual schema yet, and the write to update the schema needs to
happen in the SDK code.

Co-authored-by: James Stone <james.stone@mongodb.com>

* fix timestamp string representation on different platforms

* deprecate Object::obj for Object::get_obj

* benchmark

core v13.17.0

* More object store test tag updates
* Fix malformed tag and cleaned up other tags
* missed a [baas] tag
* Removed a few missed [local] tags

…#6803)

Say we are checking for cycles starting from object A and traverse our
way down through  A -> B -> C -> D -> E and A -> F -> C -> D -> E.
Prior to this change, we'd traverse every single property, but if we've
already checked C -> D -> E via object B, we can avoid checking it via
object F. Cumulatively, this can save a bunch of redundant checks

We also stop keeping track of all the object types we've seen. This is
because if there is a cycle A -> E -> B -> C -> D -> E, we can identify
this when E is the object being considered as the start point (seeing as
the caller goes through all embedded objects defined in the schema
anyway).

If it is detected when A is the object being considered, we're not really
adding any additional useful information. Instead, the do_check loop
becomes much simpler (and we avoid copying) if we check all child
objects are not embedding the 'start' object only

* Update error message when breaking changes are detected

* Fix assertion in test

* Code review

…quire them (#6823)

…sks and allow custom test specification (#6805)

* Separate baas/local object store tests - add custom tests specification
* Updated changelog
* Minor updates to object store tests main
* Don't generate baas ctest entry if sync is disabled
* Updated evergreen task names for object store tests
* Updated name of object-store-tests.lst and added PR merge check
* Turned off remote baas verbose logging; fixed -v flags when not provided
* Updates from review
* Removed old local baas references from evergreen config

* Missing [baas] Catch2 tag for new test
* Updated baas commit to include the context race fix (PR #10335) that was causing some test failures
* Reverted baas commit due to errors during migration to PBS
* Removed extra line from changelog

…gly (#6828)

* Update sync progress to the progress of the download message when there are no changes to integrate anymore

* Improve checks around server versions received in a download message

* Fix check

* Add integration test

* Update changelog

* Documentation fix

…o debug (#6832)

* Moved object store sync util files to util/ directory
* Moved obj store test sync util files to test/object-store/util/sync/

We changed (in v12.12.0) how we attach files in order to give better errors. Unfortunately this can lead to scenarios in which a file on streaming format (as produced by compact on launch or copy to file) is truncated or expanded before conversion to normal (non-streaming) format. If the app is then terminated (for whatever reason) we can no longer find the file footer and do the conversion later. The end result is a corrupted file - either the footer is lost or it is no longer at the end of the file, as expected.

This PR retains the better error reporting, while only doing file size changing operations after the conversion to normal format.