RCORE-2232 Actually check for unuploaded changes in no_pending_local_changes()

[tgoyne]

We can have local changesets stored which have already been uploaded and acknowledged by the server, so checking all of the changesets is incorrect. We need to instead only check changesets for versions after the current position of the upload cursor.

I wrote a basic test since we didn't have any tests at all which hit the case where we threw this exception, but I couldn't figure out how to write a standalone test for this scenario (one of the tests I wrote for #7921 hit it very indirectly). For things to break we need to have the server send us a DOWNLOAD with upload_client_version > download_client_version (i.e. it's acknowledged the upload of some version, but needs us to store some older version still for merging purposes) and then try to write a copy before we receive a DOWNLOAD message which bumps download_client_version. In practice this requires that we be uploading while receiving non-bootstrap sync downloads, and even then there's a pretty small window where things are broken.

[coveralls-official]

Pull Request Test Coverage Report for Build thomas.goyne_491

Details

• 37 of 38 (97.37%) changed or added relevant lines in 2 files are covered.
• 42 unchanged lines in 8 files lost coverage.
• Overall coverage decreased (-1.5%) to 89.577%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/realm/sync/noinst/client_history_impl.cpp	5	6	83.33%

Files with Coverage Reduction	New Missed Lines	%
src/realm/util/serializer.cpp	1	90.43%
src/realm/table_view.cpp	2	92.99%
test/object-store/util/test_file.cpp	2	86.47%
src/realm/mixed.cpp	3	86.46%
src/realm/unicode.cpp	3	83.83%
src/realm/bplustree.cpp	6	72.55%
src/realm/table.cpp	7	90.42%
src/realm/sync/noinst/server/server.cpp	18	74.02%

Totals
Change from base Build 2554:	-1.5%
Covered Lines:	145956
Relevant Lines:	162940

---

💛 - Coveralls

[danieltabacaru]

nit: size_t base_version = std::max(size_t(upload_client_version - m_sync_history_base_version), 0);

[tgoyne]

version_type is unsigned so upload_client_version - m_sync_history_base_version will never be smaller than 0.

[danieltabacaru]

For things to break we need to have the server send us a DOWNLOAD with upload_client_version > download_client_version

I think you mean upload_client_version > last_integrated_client_version

couldn't figure out how to write a standalone test for this scenario

I wrote a test that I think can be used as a base. You can use on_sync_client_event_hook to inspect download messages and write copy at the right time.

TEST_CASE("flx: bundled realm", "[app][flx][baas]") {
    FLXSyncTestHarness harness("flx_bundled_realm");
    auto config = harness.make_test_file();
    auto [promise, future] = util::make_promise_future<void>();
    config.sync_config->on_sync_client_event_hook =
        [promise = util::CopyablePromiseHolder<void>(std::move(promise))](std::weak_ptr<SyncSession>,
                                                                          const SyncClientHookData& data) mutable {
            if (data.event != SyncClientHookEvent::SessionSuspended) {
                return SyncClientHookAction::NoAction;
            }
            promise.get_promise().emplace_value();
            return SyncClientHookAction::NoAction;
        };

    auto realm = Realm::get_shared_realm(config);
    auto table = realm->read_group().get_table("class_TopLevel");
    auto new_subs = realm->get_latest_subscription_set().make_mutable_copy();
    new_subs.insert_or_assign(Query(table));
    auto subs = new_subs.commit();
    subs.get_state_change_notification(sync::SubscriptionSet::State::Complete).get();
    wait_for_upload(*realm);

    nlohmann::json error_body = {
        {"tryAgain", true},           {"message", "fake error"},
        {"shouldClientReset", false}, {"isRecoveryModeDisabled", false},
        {"action", "Transient"},      {"backoffIntervalSec", 900},
        {"backoffMaxDelaySec", 900},  {"backoffMultiplier", 1},
    };
    nlohmann::json test_command = {{"command", "ECHO_ERROR"},
                                   {"args", nlohmann::json{{"errorCode", 229}, {"errorBody", error_body}}}};

    // First we trigger a retryable transient error that should cause the client to try to resume the
    // session in 5 minutes.
    auto test_cmd_res =
        wait_for_future(SyncSession::OnlyForTesting::send_test_command(*realm->sync_session(), test_command.dump()))
            .get();
    REQUIRE(test_cmd_res == "{}");
    future.get();

    realm->begin_transaction();
    CppContext c(realm);
    Object::create(c, realm, "TopLevel",
                   util::Any(AnyDict{
                       {"_id", ObjectId::gen()},
                       {"queryable_str_field", std::string{"foo"}},
                   }));
    realm->commit_transaction();

    harness.load_initial_data([&](SharedRealm realm) {
        CppContext c(realm);
        Object::create(c, realm, "TopLevel",
                       std::any(AnyDict{{"_id", ObjectId::gen()}, {"queryable_str_field", "bar"s}}));
    });

    // Once we're suspended, immediately tell the sync client to resume the session. This should cancel the
    // timer that would have auto-resumed the session.
    realm->sync_session()->handle_reconnect();
    wait_for_download(*realm);
}

[danieltabacaru]

Nice catch!

[tgoyne]

You can use on_sync_client_event_hook to inspect download messages and write copy at the right time.

The problem is getting the server to produce the DOWNLOAD message that causes problems in the first place. It requires concurrent uploads and downloads so that the client receives a non-empty download which acknowledges a write but has a changeset based on the version before the write. This seems like it shouldn't be hard to do but every time I try to reduce a complex scenario that causes it to something understandable it stops happening.