Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document minimum required versions and test it #219

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/test-minimal.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Test minimal Rust version

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
build:

runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Rust and set a specific version
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source $HOME/.cargo/env
rustup install 1.59.0
rustup default 1.59.0

- name: Display Rust version
run: cargo --version

- name: Build
run: cargo build --verbose

- name: Test
run: cargo test --verbose

# one of the shell tests below needs jq
- name: Install jq
run: |
sudo apt-get update
sudo apt-get install -y jq

- name: Shell tests
run: |
cd tests
./run_tests.sh
5 changes: 4 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Display Rust version
run: cargo --version

- name: Build
run: cargo build --verbose
Expand Down
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ probe the system and reports on its overall configuration.

Image: [Midjourney](https://midjourney.com/), [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/legalcode)


## Subcommands

Sonar has two subcommands, `ps` and `sysinfo`. Both collect information about the system and print
Expand All @@ -35,6 +36,7 @@ Options:
-V, --version Print version
```


## Versions and release procedures

### Version numbers
Expand All @@ -51,6 +53,7 @@ itself changes. For example, v0.8.0 both added fields and stopped printing fiel
The bugfix version is updated for changes that do not alter the output format per se but that might
affect the output nevertheless, ie, most changes not covered by changes to the minor version number.


### Release branches, uplifts and backports

The following branching scheme is new with v0.12.x.
Expand Down Expand Up @@ -83,19 +86,37 @@ With the branches come some additional rules for how to move patches around:
backports.


### Policies for changing Rust edition and minimum Rust version

At the time of writing we require:
- 2021 edition of Rust
- Rust 1.59.0, released 2022-02-24 (can be found with `cargo msrv find`)

Policy for changing the minimum Rust version:
- Open a GitHub issue and motivate the change
- Once we reach agreement in the issue discussion:
- Update the version inside the test workflow [test-minimal.yml](.github/workflows/test-minimal.yml)
- Update the documentation (this section)


## Changelog

### Changes in v0.13.x-devel (on `main`)

Version in progress, no changes as of yet.


### Changes in v0.12.x (on `release_0_12`)

**System load data introduced**. Added the `load` field which is printed with one of the records
per sonar invocation. (v0.12.0)


### Changes in v0.11.x

**Better `ps` data**. More data points. (v0.11.0)


### Changes in v0.10.x

**Less output**. Removed the `cores` and `memtotalkib` fields, as they are supplied by `sonar
Expand All @@ -105,6 +126,7 @@ sysinfo`. (v0.10.0)
--batchless`) has changed from being the pid of the process below the session leader to being the
more conventional process group id. In most situations this won't make a difference. (v0.10.1)


### Changes in v0.9.x

**Sysinfo introduced**. The `sonar sysinfo` subcommand was introduced to extract information about
Expand All @@ -113,6 +135,7 @@ the system itself.
**More help when information is missing**. The user name field now includes the UID if the user
name can't be obtained from system databases but the UID is known. (v0.9.0)


### Changes in v0.8.x

**Better `ps` data**. More clarifications, more data points. (v0.8.0)
Expand All @@ -121,6 +144,7 @@ name can't be obtained from system databases but the UID is known. (v0.9.0)

**Less `ps` output**. Fields that hold default values are not printed. (v0.8.0)


### Changes in v0.7.x

**Improved `ps` process filtering.** The filters used in previous versions (minimum CPU
Expand All @@ -144,6 +168,7 @@ of fields and the use of default values.
simply-created sonar data, process it and present it in specialized ways, removing those burdens
from sonar.


### Changes in v0.6.0

**This tool focuses on how resources are used**. What is actually running. Its
Expand Down Expand Up @@ -242,17 +267,20 @@ The base-45 digits of the value `897` are (in little-endian order) 42 and 19, an
this value is thus `&J`. As the initial character is from a different character set, no explicit
separator is needed in the array - the initial digit acts as a separator.


### Version 0.11.0 `ps` output format

Version 0.11.0 adds one field:

`ppid` (optional, default "0"): The parent process ID of the job, a positive integer.


### Version 0.10.0 `ps` output format

The fields `cores` and `memtotalkib` were removed, as they were unused by all clients and are
supplied by `sonar sysinfo` for clients that need that information.


### Version 0.9.0 `ps` output format

Version 0.9.0 documents that the `user` field *in previous versions* could have the value
Expand All @@ -263,6 +291,7 @@ Version 0.9.0 extends the encoding of the `user` field: it can now (also) have t
`_noinfo_<uid>` where `<uid>` is the user ID, if user information was unobtainable for any reason
but we have a UID. Clients could be able to handle both this encoding and the older encoding.


### Version 0.8.0 `ps` output format

Fields with default values (zero in most cases, or the empty set of GPUs) are not printed.
Expand All @@ -278,6 +307,7 @@ a nonnegative integer, with 0 meaning "no data available".
Version 0.8.0 also clarifies that the existing `cpukib` field reports virtual data+stack memory, not
resident memory nor virtual total memory.


### Version 0.7.0 `ps` output format

Each field has the syntax `name=value` where the names are defined below. Fields are separated by
Expand Down Expand Up @@ -413,6 +443,7 @@ VMs) once every 24 hours, and to aggregate the information in some database.

The `sysinfo` subcommand currently has no options.


### Version 0.9.0 `sysinfo` format

The JSON structure has these fields:
Expand All @@ -429,6 +460,7 @@ Numeric fields that are zero may or may not be omitted by the producer.

Note the v0.9.0 `sysinfo` output does not carry a version number.


## Collect and analyze results

Sonar data are used by two other tools:
Expand All @@ -439,6 +471,7 @@ Sonar data are used by two other tools:
provides dashboards, interactive and batch queries, and reporting of system activity, policy violations,
hung jobs, and more.


## Authors

- [Radovan Bast](https://bast.fr)
Expand Down Expand Up @@ -497,6 +530,7 @@ of processes.

Optionally, `sonar` will use a lockfile to avoid a pile-up of processes.


## Dependencies and updates

Sonar runs everywhere and all the time, and even though it currently runs without privileges it
Expand All @@ -514,6 +548,7 @@ attack. There are some rules:

There is a useful discussion of these matters [here](https://research.swtch.com/deps).


## How we run sonar on a cluster

We let cron execute the following script every 5 minutes on every compute node:
Expand Down