diff --git a/.github/workflows/test-minimal.yml b/.github/workflows/test-minimal.yml new file mode 100644 index 0000000..fe64862 --- /dev/null +++ b/.github/workflows/test-minimal.yml @@ -0,0 +1,43 @@ +name: Test minimal Rust version + +on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + +jobs: + build: + + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Rust and set a specific version + run: | + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y + source $HOME/.cargo/env + rustup install 1.59.0 + rustup default 1.59.0 + + - name: Display Rust version + run: cargo --version + + - name: Build + run: cargo build --verbose + + - name: Test + run: cargo test --verbose + + # one of the shell tests below needs jq + - name: Install jq + run: | + sudo apt-get update + sudo apt-get install -y jq + + - name: Shell tests + run: | + cd tests + ./run_tests.sh diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8d3ab56..3075a63 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -13,7 +13,10 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v2 + uses: actions/checkout@v4 + + - name: Display Rust version + run: cargo --version - name: Build run: cargo build --verbose diff --git a/README.md b/README.md index 252b106..f7ef38a 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ probe the system and reports on its overall configuration. Image: [Midjourney](https://midjourney.com/), [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/legalcode) + ## Subcommands Sonar has two subcommands, `ps` and `sysinfo`. Both collect information about the system and print @@ -35,6 +36,7 @@ Options: -V, --version Print version ``` + ## Versions and release procedures ### Version numbers @@ -51,6 +53,7 @@ itself changes. For example, v0.8.0 both added fields and stopped printing fiel The bugfix version is updated for changes that do not alter the output format per se but that might affect the output nevertheless, ie, most changes not covered by changes to the minor version number. + ### Release branches, uplifts and backports The following branching scheme is new with v0.12.x. @@ -83,19 +86,37 @@ With the branches come some additional rules for how to move patches around: backports. +### Policies for changing Rust edition and minimum Rust version + +At the time of writing we require: +- 2021 edition of Rust +- Rust 1.59.0, released 2022-02-24 (can be found with `cargo msrv find`) + +Policy for changing the minimum Rust version: +- Open a GitHub issue and motivate the change +- Once we reach agreement in the issue discussion: + - Update the version inside the test workflow [test-minimal.yml](.github/workflows/test-minimal.yml) + - Update the documentation (this section) + + +## Changelog + ### Changes in v0.13.x-devel (on `main`) Version in progress, no changes as of yet. + ### Changes in v0.12.x (on `release_0_12`) **System load data introduced**. Added the `load` field which is printed with one of the records per sonar invocation. (v0.12.0) + ### Changes in v0.11.x **Better `ps` data**. More data points. (v0.11.0) + ### Changes in v0.10.x **Less output**. Removed the `cores` and `memtotalkib` fields, as they are supplied by `sonar @@ -105,6 +126,7 @@ sysinfo`. (v0.10.0) --batchless`) has changed from being the pid of the process below the session leader to being the more conventional process group id. In most situations this won't make a difference. (v0.10.1) + ### Changes in v0.9.x **Sysinfo introduced**. The `sonar sysinfo` subcommand was introduced to extract information about @@ -113,6 +135,7 @@ the system itself. **More help when information is missing**. The user name field now includes the UID if the user name can't be obtained from system databases but the UID is known. (v0.9.0) + ### Changes in v0.8.x **Better `ps` data**. More clarifications, more data points. (v0.8.0) @@ -121,6 +144,7 @@ name can't be obtained from system databases but the UID is known. (v0.9.0) **Less `ps` output**. Fields that hold default values are not printed. (v0.8.0) + ### Changes in v0.7.x **Improved `ps` process filtering.** The filters used in previous versions (minimum CPU @@ -144,6 +168,7 @@ of fields and the use of default values. simply-created sonar data, process it and present it in specialized ways, removing those burdens from sonar. + ### Changes in v0.6.0 **This tool focuses on how resources are used**. What is actually running. Its @@ -242,17 +267,20 @@ The base-45 digits of the value `897` are (in little-endian order) 42 and 19, an this value is thus `&J`. As the initial character is from a different character set, no explicit separator is needed in the array - the initial digit acts as a separator. + ### Version 0.11.0 `ps` output format Version 0.11.0 adds one field: `ppid` (optional, default "0"): The parent process ID of the job, a positive integer. + ### Version 0.10.0 `ps` output format The fields `cores` and `memtotalkib` were removed, as they were unused by all clients and are supplied by `sonar sysinfo` for clients that need that information. + ### Version 0.9.0 `ps` output format Version 0.9.0 documents that the `user` field *in previous versions* could have the value @@ -263,6 +291,7 @@ Version 0.9.0 extends the encoding of the `user` field: it can now (also) have t `_noinfo_` where `` is the user ID, if user information was unobtainable for any reason but we have a UID. Clients could be able to handle both this encoding and the older encoding. + ### Version 0.8.0 `ps` output format Fields with default values (zero in most cases, or the empty set of GPUs) are not printed. @@ -278,6 +307,7 @@ a nonnegative integer, with 0 meaning "no data available". Version 0.8.0 also clarifies that the existing `cpukib` field reports virtual data+stack memory, not resident memory nor virtual total memory. + ### Version 0.7.0 `ps` output format Each field has the syntax `name=value` where the names are defined below. Fields are separated by @@ -413,6 +443,7 @@ VMs) once every 24 hours, and to aggregate the information in some database. The `sysinfo` subcommand currently has no options. + ### Version 0.9.0 `sysinfo` format The JSON structure has these fields: @@ -429,6 +460,7 @@ Numeric fields that are zero may or may not be omitted by the producer. Note the v0.9.0 `sysinfo` output does not carry a version number. + ## Collect and analyze results Sonar data are used by two other tools: @@ -439,6 +471,7 @@ Sonar data are used by two other tools: provides dashboards, interactive and batch queries, and reporting of system activity, policy violations, hung jobs, and more. + ## Authors - [Radovan Bast](https://bast.fr) @@ -497,6 +530,7 @@ of processes. Optionally, `sonar` will use a lockfile to avoid a pile-up of processes. + ## Dependencies and updates Sonar runs everywhere and all the time, and even though it currently runs without privileges it @@ -514,6 +548,7 @@ attack. There are some rules: There is a useful discussion of these matters [here](https://research.swtch.com/deps). + ## How we run sonar on a cluster We let cron execute the following script every 5 minutes on every compute node: