Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

counters: add --ipcs command for showing IPC counters by interface #452

Merged
merged 31 commits into from
Mar 12, 2024

Conversation

hawkw
Copy link
Member

@hawkw hawkw commented Mar 9, 2024

PR oxidecomputer/idolatry#43 enhanced Idolatry's codegen to support
automatically generating counters of IPC operations in the generated IPC
client and server stubs. By using client-side counters, we can display
both a total count of IPC operations for a particular IPC interface,
and a breakdown of those IPC oeprations by client task. However, the
current humility counters command can only do the latter, as it
displays each instance of a counter variable separately, even when
multiple tasks contain the same counter.

Therefore, this PR adds a new --ipc flag to humility counters. This
flag will sum the IPC client counters by IPC interface, and then display
the total count for each operation followed by a breakdown by response
type, and then by client. This allows us to see which clients called
which IPC methods, what responses were received by each client, and the
fractions of the total number of IPC requests that each client was
responsible for.

Example output:

Displaying counters for a particular IPC interface:
$ humility -d tests/cmd/cores/hubris.core.ipc-counts.0 counters --ipc task_net_apihumility: attached to dump
task_net_api::__NET_CLIENT_COUNTERS
 fn Net::send_packet() ............................................. 1380 calls
    clients:
    task udpbroadcast (0 restarts) ..................... = 0 ........ = 1380 ok

 fn Net::recv_packet() ................................................ 3 calls
    clients:
    task gimlet_inspector (0 restarts) ................................. + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
    task udpecho (0 restarts) .......................................... + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
    task dump_agent (0 restarts) ....................................... + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
                                                         ---             ---
    totals:                                              = 3 err         = 0 ok

 fn Net::get_mac_address() ............................................ 2 calls
    clients:
    task udpbroadcast (0 restarts) ..................... + 0 ........... + 1 ok
    task control_plane_agent (0 restarts) .............. + 0 ........... + 1 ok
                                                         ---             ---
    totals:                                              = 0 err         = 2 ok

 fn Net::get_spare_mac_addresses() .................................... 1 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 1 ok

Using the `--client` flag to only display counts recorded by specific tasks::
$ humility --d tests/cmd/cores/hubris.core.ipc-counts.0 counters --ipc --client net --client gimlet_seq`
humility: attached to dump
drv_gimlet_hf_api::__HOSTFLASH_CLIENT_COUNTERS
 fn HostFlash::set_mux() .............................................. 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok


drv_spi_api::__SPI_CLIENT_COUNTERS
 fn Spi::exchange() ............................................... 67588 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ....... + 67580 ok
    task net (0 restarts) .............................. + 0 ........... + 8 ok
                                                         ---         -------
    totals:                                              = 0 err     = 67588 ok

 fn Spi::write() .................................................... 592 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ......... + 530 ok
    task net (0 restarts) .............................. + 0 .......... + 62 ok
                                                         ---           -----
    totals:                                              = 0 err       = 592 ok

 fn Spi::lock() ....................................................... 4 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 4 ok

 fn Spi::release() .................................................... 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok


drv_stm32xx_sys_api::__SYS_CLIENT_COUNTERS
 fn Sys::gpio_read_input() ........................................ 16796 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ....... = 16796 ok

 fn Sys::gpio_configure_raw() ........................................ 27 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 .......... + 14 ok
    task net (0 restarts) .............................. + 0 .......... + 13 ok
                                                         ---            ----
    totals:                                              = 0 err        = 27 ok

 fn Sys::gpio_set_reset() ............................................ 23 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 .......... + 15 ok
    task net (0 restarts) .............................. + 0 ........... + 8 ok
                                                         ---            ----
    totals:                                              = 0 err        = 23 ok

 fn Sys::enable_clock_raw() ........................................... 4 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 4 ok

 fn Sys::enter_reset_raw() ............................................ 2 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 2 ok

 fn Sys::leave_reset_raw() ............................................ 2 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 2 ok


task_jefe_api::__JEFE_CLIENT_COUNTERS
 fn Jefe::set_state() ................................................. 5 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 5 ok

 fn Jefe::get_state() ................................................. 2 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 2 ok


task_packrat_api::__PACKRAT_CLIENT_COUNTERS
 fn Packrat::set_spd_eeprom() ........................................ 32 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 .......... = 32 ok

 fn Packrat::get_mac_address_block() .................................. 1 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 1 ok

 fn Packrat::set_mac_address_block() .................................. 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok

 fn Packrat::set_identity() ........................................... 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok
By default, all non-zero counters are displayed:
$ humility  --d tests/cmd/cores/hubris.core.ipc-counts.0 counters --ipc 
humility: attached to dump
drv_gimlet_hf_api::__HOSTFLASH_CLIENT_COUNTERS
 fn HostFlash::get_mux() .............................................. 6 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 6 ok

 fn HostFlash::set_mux() .............................................. 2 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ........... + 1 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 1 ok
                                                         ---             ---
    totals:                                              = 0 err         = 2 ok

 fn HostFlash::get_dev() .............................................. 1 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 1 ok


drv_gimlet_seq_api::__SEQUENCER_CLIENT_COUNTERS
 fn Sequencer::get_state() ......................................... 2017 calls
    clients:
    task thermal (0 restarts) .......................... + 0 ........ + 1386 ok
    task power (0 restarts) ............................ + 0 ......... + 626 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 5 ok
                                                         ---          ------
    totals:                                              = 0 err      = 2017 ok


drv_spi_api::__SPI_CLIENT_COUNTERS
 fn Spi::exchange() ............................................... 67589 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ....... + 67580 ok
    task net (0 restarts) .............................. + 0 ........... + 8 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 1 ok
                                                         ---         -------
    totals:                                              = 0 err     = 67589 ok

 fn Spi::write() .................................................... 592 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ......... + 530 ok
    task net (0 restarts) .............................. + 0 .......... + 62 ok
                                                         ---           -----
    totals:                                              = 0 err       = 592 ok

 fn Spi::lock() ....................................................... 4 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 4 ok

 fn Spi::release() .................................................... 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok


drv_stm32xx_sys_api::__SYS_CLIENT_COUNTERS
 fn Sys::gpio_set_reset() ........................................ 135574 calls
    clients:
    task spi2_driver (0 restarts) ...................... + 0 ...... + 135316 ok
    task i2c_driver (0 restarts) ....................... + 0 ......... + 220 ok
    task gimlet_seq (0 restarts) ....................... + 0 .......... + 15 ok
    task hf (0 restarts) ............................... + 0 .......... + 10 ok
    task net (0 restarts) .............................. + 0 ........... + 8 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 3 ok
    task sprot (0 restarts) ............................ + 0 ........... + 1 ok
    task user_leds (0 restarts) ........................ + 0 ........... + 1 ok
                                                         ---        --------
    totals:                                              = 0 err    = 135574 ok

 fn Sys::gpio_read_input() ........................................ 16832 calls
    clients:
    task gimlet_seq (0 restarts) ....................... + 0 ....... + 16796 ok
    task i2c_driver (0 restarts) ....................... + 0 .......... + 36 ok
                                                         ---         -------
    totals:                                              = 0 err     = 16832 ok

 fn Sys::gpio_configure_raw() ...................................... 6351 calls
    clients:
    task i2c_driver (0 restarts) ....................... + 0 ........ + 6284 ok
    task spi2_driver (0 restarts) ...................... + 0 .......... + 24 ok
    task gimlet_seq (0 restarts) ....................... + 0 .......... + 14 ok
    task net (0 restarts) .............................. + 0 .......... + 13 ok
    task hf (0 restarts) ............................... + 0 ........... + 6 ok
    task sprot (0 restarts) ............................ + 0 ........... + 4 ok
    task spd (0 restarts) .............................. + 0 ........... + 2 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 2 ok
    task control_plane_agent (0 restarts) .............. + 0 ........... + 1 ok
    task user_leds (0 restarts) ........................ + 0 ........... + 1 ok
                                                         ---          ------
    totals:                                              = 0 err      = 6351 ok

 fn Sys::enable_clock_raw() .......................................... 14 calls
    clients:
    task net (0 restarts) .............................. + 0 ........... + 4 ok
    task i2c_driver (0 restarts) ....................... + 0 ........... + 3 ok
    task spi2_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task spd (0 restarts) .............................. + 0 ........... + 1 ok
    task hash_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task hf (0 restarts) ............................... + 0 ........... + 1 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 1 ok
    task control_plane_agent (0 restarts) .............. + 0 ........... + 1 ok
    task sprot (0 restarts) ............................ + 0 ........... + 1 ok
                                                         ---            ----
    totals:                                              = 0 err        = 14 ok

 fn Sys::leave_reset_raw() ........................................... 12 calls
    clients:
    task i2c_driver (0 restarts) ....................... + 0 ........... + 3 ok
    task net (0 restarts) .............................. + 0 ........... + 2 ok
    task spi2_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task spd (0 restarts) .............................. + 0 ........... + 1 ok
    task hash_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task hf (0 restarts) ............................... + 0 ........... + 1 ok
    task host_sp_comms (0 restarts) .................... + 0 ........... + 1 ok
    task control_plane_agent (0 restarts) .............. + 0 ........... + 1 ok
    task sprot (0 restarts) ............................ + 0 ........... + 1 ok
                                                         ---            ----
    totals:                                              = 0 err        = 12 ok

 fn Sys::enter_reset_raw() ............................................ 5 calls
    clients:
    task net (0 restarts) .............................. + 0 ........... + 2 ok
    task spi2_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task hash_driver (0 restarts) ...................... + 0 ........... + 1 ok
    task sprot (0 restarts) ............................ + 0 ........... + 1 ok
                                                         ---             ---
    totals:                                              = 0 err         = 5 ok

 fn Sys::disable_clock_raw() .......................................... 1 calls
    clients:
    task hash_driver (0 restarts) ...................... = 0 ........... = 1 ok

 fn Sys::read_uid() ................................................... 1 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 1 ok


task_jefe_api::__JEFE_CLIENT_COUNTERS
 fn Jefe::set_state() ................................................. 5 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 5 ok

 fn Jefe::get_state() ................................................. 4 calls
    clients:
    task net (0 restarts) .............................. + 0 ........... + 2 ok
    task spd (0 restarts) .............................. + 0 ........... + 2 ok
                                                         ---             ---
    totals:                                              = 0 err         = 4 ok

 fn Jefe::set_reset_reason() .......................................... 1 calls
    clients:
    task sys (0 restarts) .............................. = 0 ........... = 1 ok


task_net_api::__NET_CLIENT_COUNTERS
 fn Net::send_packet() ............................................. 1380 calls
    clients:
    task udpbroadcast (0 restarts) ..................... = 0 ........ = 1380 ok

 fn Net::recv_packet() ................................................ 3 calls
    clients:
    task gimlet_inspector (0 restarts) ................................. + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
    task udpecho (0 restarts) .......................................... + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
    task dump_agent (0 restarts) ....................................... + 0 ok
    - Err(QueueEmpty) .................................. + 1 ..................
                                                         ---             ---
    totals:                                              = 3 err         = 0 ok

 fn Net::get_mac_address() ............................................ 2 calls
    clients:
    task udpbroadcast (0 restarts) ..................... + 0 ........... + 1 ok
    task control_plane_agent (0 restarts) .............. + 0 ........... + 1 ok
                                                         ---             ---
    totals:                                              = 0 err         = 2 ok

 fn Net::get_spare_mac_addresses() .................................... 1 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 1 ok


task_packrat_api::__PACKRAT_CLIENT_COUNTERS
 fn Packrat::get_spd_data() ........................................ 9280 calls
    clients:
    task spd (0 restarts) .............................. = 0 ........ = 9280 ok

 fn Packrat::get_spd_present() ..................................... 4656 calls
    clients:
    task spd (0 restarts) .............................. + 0 ........ + 4640 ok
    task host_sp_comms (0 restarts) .................... + 0 .......... + 16 ok
                                                         ---          ------
    totals:                                              = 0 err      = 4656 ok

 fn Packrat::set_spd_eeprom() ........................................ 32 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 .......... = 32 ok

 fn Packrat::get_full_spd_data() ..................................... 16 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 .......... = 16 ok

 fn Packrat::get_identity() ........................................... 8 calls
    clients:
    task host_sp_comms (0 restarts) .................... + 0 ........... + 7 ok
    task udpbroadcast (0 restarts) ..................... + 0 ........... + 1 ok
                                                         ---             ---
    totals:                                              = 0 err         = 8 ok

 fn Packrat::get_mac_address_block() .................................. 1 calls
    clients:
    task net (0 restarts) .............................. = 0 ........... = 1 ok

 fn Packrat::set_mac_address_block() .................................. 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok

 fn Packrat::set_identity() ........................................... 1 calls
    clients:
    task gimlet_seq (0 restarts) ....................... = 0 ........... = 1 ok

 fn Packrat::get_next_boot_host_startup_options() ..................... 1 calls
    clients:
    task host_sp_comms (0 restarts) .................... = 0 ........... = 1 ok


task_sensor_api::__SENSOR_CLIENT_COUNTERS
 fn Sensor::post() ................................................ 76717 calls
    clients:
    task power (0 restarts) ............................ + 0 ....... + 50300 ok
    task thermal (0 restarts) .......................... + 0 ....... + 26417 ok
                                                         ---         -------
    totals:                                              = 0 err     = 76717 ok

 fn Sensor::get_reading() ......................................... 19804 calls
    clients:
    task thermal (0 restarts) ...................................... = 18101 ok
    - Err(NotPresent) ............................... + 1701 ..................
    - Err(DeviceError) ................................. + 2 ..................
                                                      ------         -------
    totals:                                           = 1703 err     = 18101 ok

 fn Sensor::nodata() ............................................... 6225 calls
    clients:
    task power (0 restarts) ............................ + 0 ........ + 3536 ok
    task thermal (0 restarts) .......................... + 0 ........ + 2689 ok
                                                         ---          ------
    totals:                                              = 0 err      = 6225 ok
The `--full` argument includes all counters, including those which are zero:
$ humility -d tests/cmd/cores/hubris.core.ipc-counts.0 counters --ipc --full sensor`
humility: attached to dump
task_sensor_api::__SENSOR_CLIENT_COUNTERS
 fn Sensor::get() ..................................................... 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) ..................... + 0 ...............................
    - Err(DeviceError) .................... + 0 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) ..................... + 0 ...............................
    - Err(DeviceError) .................... + 0 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) ..................... + 0 ...............................
    - Err(DeviceError) .................... + 0 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::get_reading() ......................................... 19804 calls
    clients:
    task thermal (0 restarts) ...................................... + 18101 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) .................. + 1701 ...............................
    - Err(DeviceError) .................... + 2 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                         ------
                                         = 1703 ----> + 1703
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) ..................... + 0 ...............................
    - Err(DeviceError) .................... + 0 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
    - Err(NotPresent) ..................... + 0 ...............................
    - Err(DeviceError) .................... + 0 ...............................
    - Err(DeviceUnavailable) .............. + 0 ...............................
    - Err(DeviceTimeout) .................. + 0 ...............................
    - Err(DeviceOff) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                      ------         -------
    totals:                                           = 1703 err     = 18101 ok

 fn Sensor::get_raw_reading() ......................................... 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::get_last_data() ........................................... 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::get_last_nodata() ......................................... 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::get_min() ................................................. 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::get_max() ................................................. 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok

 fn Sensor::post() ................................................ 76717 calls
    clients:
    task thermal (0 restarts) ...................................... + 26417 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ........................................ + 50300 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---         -------
    totals:                                              = 0 err     = 76717 ok

 fn Sensor::nodata() ............................................... 6225 calls
    clients:
    task thermal (0 restarts) ....................................... + 2689 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ......................................... + 3536 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---          ------
    totals:                                              = 0 err      = 6225 ok

 fn Sensor::get_nerrors() ............................................. 0 calls
    clients:
    task thermal (0 restarts) .......................................... + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task power (0 restarts) ............................................ + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
    task control_plane_agent (0 restarts) .............................. + 0 ok
    - Err(InvalidSensor) .................. + 0 ...............................
    - Err(NoReading) ...................... + 0 ...............................
                                            ---
                                            = 0 -------> + 0
                                                         ---             ---
    totals:                                              = 0 err         = 0 ok


@hawkw hawkw marked this pull request as ready for review March 10, 2024 18:20
cmd/counters/src/lib.rs Outdated Show resolved Hide resolved
@cbiffle
Copy link
Contributor

cbiffle commented Mar 11, 2024

I admit I still find the arrows and brackets confusing, particularly the converging ones, but that might just be me. (I'd use words, personally. I implemented the tasks output, for instance. No sigils.)

e.g.

    1380 Net::send_packet()
    1380 +---> Ok <---+ [udpbroadcast]

I read this as "we called send_packet 1380 times, of which 1380 times we sent Ok and, uh, udpbroadcast?"

I could surely get used to any format we choose, so don't interpret this as a hard no. What I think this is saying is

  • Across all tasks, Net::send_packet was used 1380 times in the current instances of tasks.
  • Of these, all 1380 were from the task udpbroadcast
  • Of these, all 1380 returned Ok.

Off the top of my head, one alternate way of presenting this that I'd find clearer might be (sketchy):

task_net_api::__NET_CLIENT_COUNTERS
- Net::send_packet: 1380 uses, 0 errors
    - task udpbroadcast (gen 0): 1380 uses
        - all returned: Ok
- Net::recv_packet: 3 uses, 3 errors
    - task gimlet_inspector (gen 123): 1 use
        - all returned: Err(QueueEmpty)
    - task udpecho (gen 2): 1 use
        - all returned: Err(QueueEmpty)
    - task dump_agent (gen 6): 1 use
        - all returned: Err(QueueEmpty)

...with a flag for showing only a single task if desired. My implicit assumptions here include, but are likely not limited to:

  1. Tasks will restart, so showing generation number is useful. If the generation number is non-zero, you know that the count may be an underestimate. (The full-width restart counter in the kernel is the best thing to rely upon here, since it overflows at 2**32 restarts, unlike the IPC generation which is ~6 bits.)
  2. The likelihood that I want to know how many times a particular error code is returned across all tasks is probably low, with the exception of knowing that it's >0. (My proposed format doesn't support that super well, but does provide the "0 errors" summary as an approximation.)
  3. Users can read simple English. This assumption is broadly embedded into Humility already.

You get the idea, I hope, and maybe there are things here that are useful to you.

FYI, I am sensitive to dismissal of UX issues as "bikeshedding" and would prefer to avoid that.

@hawkw
Copy link
Member Author

hawkw commented Mar 12, 2024

Okay, I've made a great deal of changes to the output format based on @cbiffle's suggestions, and updated the PR description to show the new output. Also, I've added a --client CLI argument, for filtering by clients. I'd love another review!

Comment on lines +246 to +251
/// when used with `--ipc`, show only IPC counters originating from tasks
/// whose name contain the given substring.
///
/// multiple values may be provided to select more than one client task.
#[clap(long, short, conflicts_with = "list", requires = "ipc")]
client: Vec<String>,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there's now a flag that's only relevant when --ipc is passed, I wonder if it would be a good idea to change --ipc from a flag to an ipc subcommand, so that the --client flag can only be defined for that subcommand? On the other hand, I haven't seen many other nested subcommands in Humility --- do we generally avoid nesting subcommands by convention?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bryan tends not to use more than one level of subcommand; I tend to use subcommands and positional arguments in preference to flags in a lot of situations. Kind of depends on who wrote the command.

I'd enthusiastically support subcommanding this case.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 101 files.

Valid Invalid Ignored Fixed
100 1 0 0
Click to see the invalid file list
  • cmd/counters/src/ipc.rs

cmd/counters/src/ipc.rs Show resolved Hide resolved
cmd/counters/src/ipc.rs Show resolved Hide resolved
let err_str =
if num_important_tasks > 1 { "+ 0" } else { "= 0" };
let pad1 = 80usize
.saturating_sub(err_str.len())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these are almost an argument for using Saturating here, buuuut I think it'd wind up just as verbose and I'm not sure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL about Saturating! That must be new!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it is --- stabilized in 1.74.0!

I think these would be a bit less verbose using Saturating, since we could just wrap the 80 in Saturating and then do the rest of the arithmetic normally, but it would require updating our MSRV to 1.74.0...

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@hawkw hawkw merged commit 99dfbf8 into master Mar 12, 2024
11 checks passed
hawkw added a commit that referenced this pull request Mar 13, 2024
As suggested by @bcantrill in [this comment][1], recent changes from
PRs #452 and # 453 have added new user-visible features, so we ought to
bump the patch version of Humility to indicate this.

[1]: #453 (review)
@hawkw hawkw mentioned this pull request Mar 13, 2024
hawkw added a commit that referenced this pull request Mar 13, 2024
As suggested by @bcantrill in [this comment][1], recent changes from
PRs #452 and # 453 have added new user-visible features, so we ought to
bump the patch version of Humility to indicate this.

[1]: #453 (review)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants