Info

A set of tools to capture RAW frames from Pi Camera HQ (IMX477 sensor, crop 5.5) and send them over network to process on a machine more capable than the Raspberry Pi. RAW data provides a lot more precision and range, making processing easier and images more detailed (not corrupted by compression, denoising artifacts, suboptimal demosaicing and postprocessing). It is intended to work in realtime on long series of frames, to capture as much information as possible. Tested only with libcamera camera stack. Work in progress.

Initially started as a CDLL plugin for Python 3 written in C (+ Python tools using this lib), it was later rewritten in Rust as a standalone set of binaries and developed further.

It uses UNIX pipes to connect given processing steps to form a pipeline. Each processing step is a separate tool.

It would probably be faster when using dedicated libraries for serialization/deserialization (like serde), but it was fun exercise to implement things from scratch. I may try to optimize it a bit in the future. Throughput of the software can be checked with fake data input from /dev/zero, by using the pv command, like this:

cd pipecam_tools_rust/target/release
cat /dev/zero | pv | ./pipecam_mark_headers_rs | ./pipecam_unpack_bayer_u12_to_u16_rs | ./pipecam_u16_to_f32_rs | ./pipecam_average_f32_rs | ./pipecam_f32_to_u16_rs | ./pipecam_u16_to_ser_seq > /dev/null

Or like this, on RPi with Pi Camera HQ attached and configured, taking the hardware camera performance into account:

cat /dev/video0 | pv | ...

Useful resource: Siril preprocessing tutorial. Covers preprocessing basics, which can be applied to this software - you can prepare and apply dark, bias and flat images using these tools (with quite an elaborate bash oneliner) or export frames as SER file and do processing in other software supporting this file format.

Tools

Rust: ./pipecam_tools_rust

• pipecam_average_f32_rs.rs -- averages many frames into one to reduce noise and enable emulation of very long shutter times, works on 32bit floats.
• pipecam_crop_image_rs.rs -- crops the image, useful for decreasing necessary network troughput. Check arguments in help using command: pipecam_crop_image_rs -h.
• pipecam_dark_subtract_f32_rs.rs -- subtracts the previously prepared dark frame (f32) for cleaner image. Check pipecam_dark_subtract_f32_rs -h for more info.
• pipecam_f32_to_u16_rs.rs -- converts pixel/subpixel data type
• pipecam_flat_apply_f32_rs.rs -- applies previously prepared flat to the image (removes vingette and other linear nonuniformities). Check pipecam_flat_apply_f32_rs -h for more info.
• pipecam_leak_to_file_rs.rs -- writes the image to a file and passes it further down the pipeline. Check pipecam_leak_to_file_rs -h for more info.
• pipecam_mark_headers_rs.rs -- adds headers to the raw data from /dev/video0 to mark individual frames.
• pipecam_u16_to_f32_rs.rs -- converts pixel/subpixel data type
• pipecam_u16_to_ser_seq.rs -- converts frames to SER file format to be used as input for Siril - an astronomical image processing tool.
• pipecam_unpack_bayer_u12_to_u16_rs.rs -- converts Bayer data packing for a whole byte alignment.

Bash: ./scripts_*

Run pipecam_server scripts on Raspberry Pi, the rest (pipecam_receive) is for client machine with better processing power. Client scripts work also on RPi with enough RAM, but a bit slow. Might be ok for longer shutter speeds.

• ./scripts_server_raspberry

  • pipecam_server_send_crop8_packed_nobuffer.sh
  • pipecam_server_send_crop8_packed.sh
  • pipecam_server_send_full_packed_nobuffer.sh
  • pipecam_server_send_full_packed.sh

• ./scripts_client

  • pipecam_receive_preview_only.sh
  • pipecam_receive_preview_pass.sh
  • pipecam_receive.sh

Python 3: ./pipecam_tools_preview_python3

Deprecated, most Python 3 tools rewritten and removed. One script left to rewrite, but it works as it is (albeit a bit slow).

• pipecam_preview.py -- a tool to preview the image from the pipe without saving, passes data further down the pipeline. TODO: rewrite in Rust.

C: ./pipecam_tools_c_cdll_for_python3

Deprecated, left as an example how to wrap C libraries for use in Python 3. Might be useful in the future.

• picam_crop_image_pipe.c -- crops the image.
• picam_mark_headers.c -- adds headers to the raw data from /dev/video0.
• picam_remove_headers.c -- removes header.