rpc v2: backpressure chainhead_v1_follow

[pkhry]

Description

closes #5871

The chainHead_v1_follow is using unbounded channels to send out messages on the JSON-RPC connection which may use lots of memory if the client is slow and can't really keep up with server i.e, substrate may keep lots of message in memory

This PR changes the outgoing stream to abort and send a Stop event downstream in the case that client doesn't keep up with the producer.

Integration

In depth notes about how this PR should be integrated by downstream projects. This part is mandatory, and should be
reviewed by reviewers, if the PR does NOT have the R0-Silent label. In case of a R0-Silent, it can be ignored.

Review Notes

• rpc::Subscription::pipe_from_stream - now takes Self param by reference, change was made to allow sending events to the Subscription after calls to pipe_from_stream.
• chainhead_follow::submit_events - now uses Abortable stream to end it early in case
  • connection was closed by the client
  • signal received that subscription should stop
  • error has occured when processing the events
  • client can't keep up with the events produced
• TODO:
  • make the abort logic less hacky

[pkhry]

bot fmt

[command-bot]

@pkhry https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7571594 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh". Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 1-068fee57-0c9c-42ce-9555-9427935c80cd to cancel this command or bot cancel to cancel all commands in this pull request.

[command-bot]

@pkhry Command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh" has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7571594 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7571594/artifacts/download.

[niklasad1]

ok nice catch :)

[lexnv]

IIRC, this was causing the lifetime issue Pavlo mentioned before, nice catch indeed :D

[lexnv]

LGTM! Thanks for handling this @pkhry 🙏

[pkhry]

bot fmt

[command-bot]

@pkhry https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7598335 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh". Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 21-70da5a54-87f4-4db3-966c-444529c72eaf to cancel this command or bot cancel to cancel all commands in this pull request.

[command-bot]

@pkhry Command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh" has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7598335 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7598335/artifacts/download.

[carlosala]

I guess this PR could be backported to 2407 and 2409 as well, similar to #5741 👍🏻

cc @niklasad1

[paritytech-cmd-bot-polkadot-sdk]

Created backport PR for stable2407:

• [stable2407] Backport #6058 #6121 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-6058-to-stable2407
git worktree add --checkout .worktree/backport-6058-to-stable2407 backport-6058-to-stable2407
cd .worktree/backport-6058-to-stable2407
git reset --hard HEAD^
git cherry-pick -x a0aefc6b233ace0a82a8631d67b6854e6aeb014b
git push --force-with-lease

[paritytech-cmd-bot-polkadot-sdk]

Created backport PR for stable2409:

• [stable2409] Backport #6058 #6122 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-6058-to-stable2409
git worktree add --checkout .worktree/backport-6058-to-stable2409 backport-6058-to-stable2409
cd .worktree/backport-6058-to-stable2409
git reset --hard HEAD^
git cherry-pick -x a0aefc6b233ace0a82a8631d67b6854e6aeb014b
git push --force-with-lease