Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiprocess bitcoin #7

Open
wants to merge 23 commits into
base: bitcoin-fresheyes-master-10102
Choose a base branch
from
Open

Multiprocess bitcoin #7

wants to merge 23 commits into from

Commits on Jan 11, 2024

  1. serialization: Support for multiple parameters 

    This commit makes a minimal change to the ParamsStream class to let it retrieve
multiple parameters. Followup commits after this commit clean up code using
ParamsStream and make it easier to set multiple parameters.

Currently it is only possible to attach one serialization parameter to a stream
at a time. For example, it is not possible to set a parameter controlling the
transaction format and a parameter controlling the address format at the same
time because one parameter will override the other.

This limitation is inconvenient for multiprocess code since it is not possible
to create just one type of stream and serialize any object to it. Instead it is
necessary to create different streams for different object types, which
requires extra boilerplate and makes using the new parameter fields a lot more
awkward than the older version and type fields.

Fix this problem by allowing an unlimited number of serialization stream
parameters to be set, and allowing them to be requested by type. Later
parameters will still override earlier parameters, but only if they have the
same type.

This change requires replacing the stream.GetParams() method with a
stream.GetParams<T>() method in order for serialization code to retreive the
desired parameters. This change is more verbose, but probably a good thing for
readability because previously it could be difficult to know what type the
GetParams() method would return, and now it is more obvious.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    0be3913 View commit details
    Browse the repository at this point in the history

  2. test: add ipc test to test multiprocess type conversion code 

    Add unit test to test IPC method calls and type conversion between bitcoin c++
types and capnproto messages.

Right now there are custom type hooks in bitcoin IPC code, so the test is
simple, but in upcoming commits, code will be added to convert bitcoin types to
capnproto messages, and the test will be expanded.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    422bd2d View commit details
    Browse the repository at this point in the history

  3. serialization: Drop unnecessary ParamsStream references 

    Drop unnecessary ParamsStream references from CTransaction and
CMutableTransaction constructors. This just couples these classes unnecessarily
to the ParamsStream class, making the ParamsStream class harder to modify, and
making the transaction classes in some cases (depending on parameter order)
unable to work with stream classes that have multiple parameters set.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    de13c17 View commit details
    Browse the repository at this point in the history

  4. multiprocess: Add type conversion code for serializable types 

    Allow any C++ object that has Serialize and Unserialize methods and can be
serialized to a bitcoin CDataStream to be converted to a capnproto Data field
and passed as arguments or return values to capnproto methods using the Data
type.

Extend IPC unit test to cover this and verify the serialization happens
correctly.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    c9dad8d View commit details
    Browse the repository at this point in the history

  5. serialization: Reverse ParamsStream constructor order 

    Move parameter argument after stream argument so will be possible to accept
multiple variadic parameter arguments in the following commit.

Also reverse template parameter order for consistency.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    9e9dfce View commit details
    Browse the repository at this point in the history

  6. multiprocess: Add type conversion code for UniValue types 

    Extend IPC unit test to cover this and verify the serialization happens
correctly.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    afce182 View commit details
    Browse the repository at this point in the history

  7. serialization: Accept multiple parameters in ParamsStream constructor 

    Before this change it was possible but awkward to create ParamStream streams
with multiple parameter objects. After this change it is straightforward.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    ffc1e9f View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'origin/pull/28921/head'

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    ca82b45 View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'origin/pull/28929/head'

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    d27c337 View commit details
    Browse the repository at this point in the history

  10. doc: multiprocess documentation improvements 

    All suggested by stickies-v <stickies-v@protonmail.com>
bitcoin#28978 (review)

Co-authored-by: stickies-v <stickies-v@protonmail.com>
    @ryanofsky @stickies-v
    ryanofsky and stickies-v committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    ee4b913 View commit details
    Browse the repository at this point in the history

  11. Increase feature_block.py and feature_taproot.py timeouts 

    Needed because BlockConnected notifications are a lot slower with the wallet
running in separate process.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    df27e30 View commit details
    Browse the repository at this point in the history

  12. Add util::Result workaround to be compatible with libmultiprocess 

    Make default constructor more generic so it doesn't only work with void types.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    cbc5e36 View commit details
    Browse the repository at this point in the history

  13. Add capnp serialization code for bitcoin types

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    3427e7e View commit details
    Browse the repository at this point in the history

  14. Add capnp wrapper for Handler interface

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    b6516c4 View commit details
    Browse the repository at this point in the history

  15. Add capnp wrapper for Chain interface

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    5cf30e5 View commit details
    Browse the repository at this point in the history

  16. Add capnp wrapper for Wallet interface

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    b5fd425 View commit details
    Browse the repository at this point in the history

  17. Add capnp wrapper for Node interface

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    329b34c View commit details
    Browse the repository at this point in the history

  18. Make bitcoin-gui spawn a bitcoin-node process 

    Spawn node subprocess instead of running node code internally
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    dde2a22 View commit details
    Browse the repository at this point in the history

  19. Make bitcoin-node spawn a bitcoin-wallet process 

    Spawn wallet subprocess instead of running wallet code internally
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    f3349a2 View commit details
    Browse the repository at this point in the history

  20. multiprocess: Add debug.log .wallet/.gui suffixes 

    Add .wallet/.gui suffixes to log files created by bitcoin-gui and
bitcoin-wallet processes so they don't clash with bitcoin-node log file.
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    6ea638c View commit details
    Browse the repository at this point in the history

  21. doc: Multiprocess misc doc and comment updates

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    622acd0 View commit details
    Browse the repository at this point in the history

  22. combine_logs: Handle multiprocess wallet log files

    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    755ca62 View commit details
    Browse the repository at this point in the history

  23. Update libmultiprocess library 

    Add recent changes to support C++20 and fix some build issues:

chaincodelabs/libmultiprocess#89 pkgconfig: Drop -std=c++17 compile flag
chaincodelabs/libmultiprocess#90 pkgconfig: Use @CMAKE_INSTALL_LIBDIR@ variable
    @ryanofsky
    ryanofsky committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    56ef459 View commit details
    Browse the repository at this point in the history