diff --git a/helm/cron_jobs/README.md b/helm/cron_jobs/README.md new file mode 100644 index 00000000000..edd63378621 --- /dev/null +++ b/helm/cron_jobs/README.md @@ -0,0 +1,18 @@ +Replayer cron jobs +================== + +There are replayer cron jobs for Mainnet, Devnet, and Berkeley. These +jobs are run daily, to replay a day's worth of transactions. + +Each cron job downloads the most recent archive dump corresponding to +a network, and loads the data into PostgreSQL. That results in an +archive database. The most recent replayer checkpoint file is +downloaded, which provides the starting point for the replayer. When +the replayer runs, it creates new checkpoint files every 50 +blocks. When the replayer finishes, it uploads the most recent +checkpoint file, so it can be used in the following day's run. If +there are any errors, the replayer logs are also uploaded. + +There is a separate checkpoint file bucket for each network. Both the +checkpoint files and error files for a given network are uploaded to +the same bucket. diff --git a/src/app/archive_blocks/README.md b/src/app/archive_blocks/README.md new file mode 100644 index 00000000000..718e7a48467 --- /dev/null +++ b/src/app/archive_blocks/README.md @@ -0,0 +1,20 @@ +archive_blocks +============== + +The `archive_blocks` app adds blocks in either "precomputed" or +"extensional" format to the archive database. + +Precomputed blocks are stored in the bucket `mina_network_block_data` +on Google Cloud Storage. Blocks are named NETWORK-HEIGHT-STATEHASH.json. +Example: mainnet-100000-3NKLvMCimUjX1zjjiC3XPMT34D1bVQGzkKW58XDwFJgQ5wDQ9Tki.json. + +Extensional blocks are extracted from other archive databases using +the `extract_blocks` app. + +As many blocks as are available can be added at a time, but all blocks +must be in the same format. + +Except for blocks from the original mainnet, both precomputed and +extensional blocks have a version in their JSON representation. That +version must match the corresponding OCaml type in the code when this +app was built. diff --git a/src/app/archive_blocks/archive_blocks.ml b/src/app/archive_blocks/archive_blocks.ml index caa4203939e..7851ca04518 100644 --- a/src/app/archive_blocks/archive_blocks.ml +++ b/src/app/archive_blocks/archive_blocks.ml @@ -1,4 +1,4 @@ -(* archive_blocks.ml -- archive precomputed or extensional blocks to Postgresql *) +(* archive_blocks.ml *) open Core_kernel open Async diff --git a/src/app/extract_blocks/README.md b/src/app/extract_blocks/README.md new file mode 100644 index 00000000000..bbaecdbb29b --- /dev/null +++ b/src/app/extract_blocks/README.md @@ -0,0 +1,12 @@ +extract_blocks +============== + +The `extract_blocks` app pulls out individual blocks from an archive +database in "extensional" format. Such blocks can be added to other +archive databases using the `archive_blocks` app. + +Blocks are extracted into files with name .json. + +The app offers the choice to extract all canonical blocks, or a +subchain specified with starting state hash, or a subchain specified +with starting and ending state hashes. diff --git a/src/app/extract_blocks/extract_blocks.ml b/src/app/extract_blocks/extract_blocks.ml index f4fa0ac2e77..e7f58718cba 100644 --- a/src/app/extract_blocks/extract_blocks.ml +++ b/src/app/extract_blocks/extract_blocks.ml @@ -1,4 +1,5 @@ (* extract_blocks.ml -- dump extensional blocks from archive db *) + [@@@coverage exclude_file] open Core_kernel diff --git a/src/app/missing_blocks_auditor/README.md b/src/app/missing_blocks_auditor/README.md new file mode 100644 index 00000000000..e5db4ee64a3 --- /dev/null +++ b/src/app/missing_blocks_auditor/README.md @@ -0,0 +1,10 @@ +missing_blocks_auditor +====================== + +The `missing_blocks_auditor` app looks for blocks without parent +blocks in an archive database. + +The app also looks for blocks marked as pending that are lower (have a +lesser height) than the highest (most recent) canonical block. There +can be such blocks if blocks are added when there are missing blocks +in the database. diff --git a/src/app/missing_blocks_auditor/missing_blocks_auditor.ml b/src/app/missing_blocks_auditor/missing_blocks_auditor.ml index 5bce995ea28..e6fd538f1b5 100644 --- a/src/app/missing_blocks_auditor/missing_blocks_auditor.ml +++ b/src/app/missing_blocks_auditor/missing_blocks_auditor.ml @@ -1,4 +1,4 @@ -(* missing_blocks_auditor.ml -- report missing blocks from an archive db *) +(* missing_blocks_auditor.ml *) open Core_kernel open Async