Skip to content

jmpatri615/frc_auto_log_dl

Repository files navigation

logDownloader

Automatically download .wpilog files from an FRC roboRIO after each match. Optionally acts as a hub — proxying NetworkTables for multi-client AdvantageScope access, serving logs over HTTP, and recording cameras synced to match cycles.

How It Works

Monitor PC (this tool)              roboRIO (10.TE.AM.2)
┌───────────────────────┐           ┌──────────────┐
│  NT4 client           │◄─────────►│  NT4 server  │
│  watches /Robot/Enabled│          │              │
│                       │           │              │
│  SFTP client          │──────────►│  SSH server  │
│  downloads .wpilog    │           │  /home/lvuser│
├───────────────────────┤           └──────────────┘
│  Hub services         │
│  ├─ NT4 proxy  :5811  │◄─────── AdvantageScope clients
│  ├─ File server:5812  │◄─────── Browser / remote tools
│  └─ Camera recorder   │
└───────────────────────┘
  1. Connects to roboRIO via NetworkTables (NT4)
  2. Watches for enabled → disabled transitions
  3. Waits for log files to flush, then downloads new .wpilog files via SFTP
  4. Splits multi-session logs into per-enable-cycle files
  5. Repeats for the next match

Requirements

  • Python 3.10+
  • Robot code must publish an enabled state key to NetworkTables (see Robot Setup)
  • PC must be on the same network as the roboRIO

Installation

pip install -e .

Installs logdownloader and all dependencies (pyntcore, paramiko, pyyaml, rich).

For camera recording support:

pip install -e ".[camera]"

Configuration

Edit config.yaml:

team_number: 1756

roborio:
  address: null          # null = auto-derive from team number (10.17.56.2)
  username: lvuser
  password: ""

networktables:
  enabled_key: "/Robot/Enabled"
  download_delay: 3.0    # seconds to wait after disable before downloading

download:
  mode: sftp             # "sftp" (roboRIO) or "local" (simulator)
  local_dir: "./logs"
  remote_paths:          # scanned in sftp mode
    - "/home/lvuser/logs"
    - "/U/logs"
  # local_source_dir: "./sim-logs"  # source dir for local mode
  file_extensions:
    - ".wpilog"
  idle_suffix: "-disabled"   # skip idle-period logs (set to "" to disable)
  tracker_file: ".downloaded_logs.json"
  stable_wait_interval: 1.0  # seconds between file-size stability checks
  stable_wait_timeout: 15.0  # max seconds to wait for files to stop growing

hub:
  proxy_port: 5811       # NT4 proxy for AdvantageScope clients
  file_server_port: 5812 # HTTP server for remote log/video access
  enable_proxy: true
  enable_file_server: true

cameras:                 # requires opencv-python
  # - name: DriverCam
  #   source: 0            # USB camera index or RTSP URL
  #   resolution: [1920, 1080]
  #   fps: 30

The roboRIO address is derived from team_number as 10.TE.AM.2. Set address explicitly to override. In local mode it defaults to localhost.

Usage

Monitor mode (default)

Continuously watches the robot and downloads logs after each disable:

logdownloader
[logDownloader] Team 1756 | mode: sftp | roboRIO: 10.17.56.2
[14:32:01] Connecting to roboRIO...
[14:32:02] Robot state: Disabled
[14:32:15] Robot state: Enabled
[14:35:42] Robot state: Disabled
[14:35:45] Downloading logs...
           ├─ FRC_20260221_193215.wpilog  [2.3 MB] ████████████ 100%
           └─ FRC_20260221_193542.wpilog  [1.1 MB] ████████████ 100%
[14:35:48] Downloaded 2 files (3.4 MB) → ./logs/
[14:35:48] Monitoring for next enable/disable cycle...

All CLI flags

Flag Description
--config PATH Config file path (default: ./config.yaml)
--once Download new logs once and exit
--download-all Re-download all logs, ignoring tracker
--list List remote log files without downloading
--watch Continuously watch remote files for changes (no download)
--debug Monitor remote file changes for 10s after disable, then download
--clear-rio Delete all log files from roboRIO (with confirmation)

Examples

logdownloader                              # monitor mode
logdownloader --once                       # one-shot download
logdownloader --list                       # list files on roboRIO
logdownloader --download-all               # force re-download everything
logdownloader --watch                      # watch file changes (no download)
logdownloader --debug                      # debug file stability issues
logdownloader --clear-rio                  # clear roboRIO storage
logdownloader --config custom.yaml         # use a different config

Features

Hub Services

When enabled in config, the monitor also starts:

  • NT4 Proxy (port 5811) — Re-publishes all robot topics so multiple AdvantageScope clients can connect without overloading the roboRIO. Forwards /SuperLogger/ config topics back to the robot.
  • HTTP File Server (port 5812) — Serves downloaded logs and camera recordings. Endpoints: GET /files (JSON listing), GET /file/{name} (download), GET /status.

Camera Recording

Records USB or IP cameras during each enable cycle. Requires opencv-python.

Each recording produces:

  • {name}_{timestamp}.mp4 — video file
  • {name}_{timestamp}.mp4.sync.json — sync metadata (robot timestamp, wall clock, fps, resolution)

Session Splitting

Monolithic .wpilog files containing multiple enable/disable cycles are automatically split into {stem}_session{N}.wpilog files. Uses a Rust binary (wpilog-split/) when available, with a Python fallback.

Idle Log Filtering

Log files whose stem ends with -disabled (configurable via idle_suffix) are skipped during download. In SFTP mode, idle files are also deleted from the roboRIO.

Download Tracking

A JSON tracker (.downloaded_logs.json) prevents re-downloading files. Each entry records the remote path, filename, size, and timestamp. Use --download-all to bypass.

Robot Setup

Your robot code must publish a boolean NetworkTables key so this tool can detect enable/disable transitions:

# robotInit()
from ntcore import NetworkTableInstance
nt = NetworkTableInstance.getDefault()
self.enabled_pub = nt.getBooleanTopic("/Robot/Enabled").publish()

# robotPeriodic()
from wpilib import DriverStation
self.enabled_pub.set(DriverStation.isEnabled())

See robot_snippet/log_state_publisher.py for the complete example.

Project Structure

logDownloader/
├── pyproject.toml              # Project metadata & dependencies
├── config.yaml                 # User configuration
├── config-simulator.yaml       # Simulator configuration example
├── src/
│   ├── main.py                 # Entry point & orchestration
│   ├── config.py               # Config loader & CLI args
│   ├── nt_monitor.py           # NetworkTables state monitor
│   ├── sftp_downloader.py      # SFTP download client
│   ├── local_downloader.py     # Local file copy (simulator mode)
│   ├── download_tracker.py     # Duplicate-prevention tracker
│   ├── wpilog_splitter.py      # Multi-session log splitter
│   ├── nt4_proxy.py            # NT4 proxy server (hub)
│   ├── file_server.py          # HTTP file server (hub)
│   └── camera_recorder.py      # Camera recording (hub)
├── wpilog-split/               # Rust wpilog splitter binary
│   ├── Cargo.toml
│   └── src/
└── robot_snippet/
    └── log_state_publisher.py  # Example robot code

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors