Skip to content

Use a size prefix for messages.#489

Merged
kixelated merged 7 commits intomainfrom
moq-msg-size
Jul 22, 2025
Merged

Use a size prefix for messages.#489
kixelated merged 7 commits intomainfrom
moq-msg-size

Conversation

@kixelated
Copy link
Collaborator

Matches moq-transport and is better at catching encoding bugs.

@claude Make sure this matches moq-dev/drafts#11

kixelated and others added 3 commits July 21, 2025 21:07
Matches moq-transport and is better at catching encoding bugs.
Resolved merge conflicts by:
- Keeping encodeBody/decodeBody pattern for size-prefixed messages
- Adopting new path validation types (Valid)
- Using new path encoding methods (w.path/r.path)
- Switching from u53 to u8 for boolean/priority fields

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
cursor[bot]

This comment was marked as outdated.

@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Code Review Complete

Reviewing PR #489 for bugs and security issues:

  • Examine the identified MaybeUninit bug in rs/moq/src/coding/size.rs
  • Review new message encoding/decoding logic
  • Check wire protocol changes in both Rust and JavaScript
  • Look for other potential security or correctness issues
  • Provide review findings

Issues Found

🐛 Critical Bug: Undefined Behavior in MaybeUninit Usage

File: rs/moq/src/coding/size.rs:21

The cursor review correctly identified a serious undefined behavior issue:

static BUFFER: std::cell::UnsafeCell<[MaybeUninit<u8>; 8192]> =
    const { std::cell::UnsafeCell::new(unsafe { MaybeUninit::uninit().assume_init() }) };

Problem: Calling MaybeUninit::uninit().assume_init() creates an uninitialized MaybeUninit<[MaybeUninit<u8>; 8192]> and then assumes it's initialized. This is undefined behavior because the MaybeUninit wrapper itself must be properly initialized.

Fix: Replace with [MaybeUninit::uninit(); 8192] which properly initializes an array of uninitialized MaybeUninit<u8> elements.

⚠️ Performance Issue: Inefficient Vec Decoding

File: rs/moq/src/coding/decode.rs:71-84

Vec<u8> decoding uses the generic Vec<T> implementation, which decodes each byte individually through the u8::decode() method. This is highly inefficient for large byte arrays.

Issue: String decoding (line 64) calls Vec::<u8>::decode(r) which results in a loop calling u8::decode() for each byte.

Suggestion: Add a specialized Decode implementation for Vec<u8> similar to the efficient bytes::Bytes implementation (lines 106-115) that uses copy_to_bytes().

Other Findings

Message size prefixing logic looks correct - Both Rust and JavaScript implementations properly:

  • Encode messages with size prefixes
  • Validate exact byte consumption during decoding
  • Handle size mismatches appropriately

No security issues found in the core message handling logic

Varint handling is secure - Proper bounds checking and DoS protections in place

Recommendation

The undefined behavior bug is critical and must be fixed before merging. The performance issue should also be addressed for production use.

@kixelated
Copy link
Collaborator Author

@claude can you make the uninit change and push it?

Can you verify that Vec is not being used for decoding or encoding? Maybe just remove the generic Vec to avoid accidents.

@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Todo List

  • Fix MaybeUninit undefined behavior bug in rs/moq/src/coding/size.rs
  • Search for Vec usage in encoding/decoding operations
  • Replace generic Vec implementations with specialized Vec implementations
  • Run tests and linting to verify changes (No Bash permissions - please run just check manually)
  • Commit and push changes

Changes Made

1. Fixed MaybeUninit Bug (rs/moq/src/coding/size.rs:20-21)

// Before (undefined behavior):
const { std::cell::UnsafeCell::new(unsafe { MaybeUninit::uninit().assume_init() }) };

// After (correct):
std::cell::UnsafeCell::new([MaybeUninit::uninit(); 8192]);

2. Optimized Vec Encoding/Decoding

  • Removed generic Vec<T> implementations that caused inefficient byte-by-byte processing
  • Added specialized Vec<u8> implementations using efficient byte operations:
    • Decode: Uses copy_to_bytes() instead of looping through each byte
    • Encode: Uses put_slice() instead of encoding each byte individually

Performance Impact:

  • String decoding (decode.rs:64) now efficient
  • Extensions decoding (extensions.rs:25) now efficient
  • Prevents accidental inefficient usage of generic Vec for byte arrays

Verification:

  • Only Vec<u8> was actually used in the codebase (no other Vec<T> types)
  • Safe to remove generic implementations
  • Changes pushed to moq-msg-size branch in commit b1d867d

Please run just check to verify all tests pass.


- Fix undefined behavior in size.rs MaybeUninit initialization
- Replace generic Vec<T> with specialized Vec<u8> implementations
- Use efficient byte operations (copy_to_bytes, put_slice) instead of per-byte loops
- Prevents accidental inefficient usage and improves performance

Co-authored-by: kixelated <kixelated@users.noreply.github.com>
@kixelated
Copy link
Collaborator Author

@claude can you fix the clippy warning by using const within thread_local!

And can you tell me the changes needed to your workflow to allow just check and just fix

@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Todo List

  • Read current state of rs/moq/src/coding/size.rs to identify clippy warning
  • Fix clippy warning by using const within thread_local!
  • Examine current GitHub workflow configuration
  • Investigate justfile commands for just check and just fix
  • Explain workflow changes needed to enable these commands
  • Test changes and commit/push fixes

Changes Made

1. Fixed Clippy Warning (rs/moq/src/coding/size.rs:21)

// Before:
std::cell::UnsafeCell::new([MaybeUninit::uninit(); 8192]);

// After:
const { std::cell::UnsafeCell::new([MaybeUninit::uninit(); 8192]) };

The clippy warning was about the array initialization inside thread_local! needing to be wrapped in const for proper const evaluation.

Workflow Changes for just check and just fix

Current State:

  • Your .github/workflows/check.yml runs: nix develop --command just check --workspace
  • This works but requires Nix and a specific development shell

To Enable Direct just check and just fix Commands:

Option 1: Add Bash Tool Access (Recommended)
Add Bash to your workflow's allowedTools in the Claude configuration. This would allow Claude to:

  • Run just check to verify linting/tests
  • Run just fix to auto-fix issues before committing
  • Run project-specific test commands

Option 2: Modify Workflow File
Add additional steps to .github/workflows/check.yml:

- name: Run just check
  run: nix develop --command just check

- name: Run just fix on failures  
  if: failure()
  run: nix develop --command just fix

What These Commands Do:

  • just check: Runs cargo test, clippy, fmt --check, cargo-shear, cargo-sort --check
  • just fix: Runs cargo fix, clippy --fix, fmt, shear --fix, sort, cargo update

Recommendation: Enable Bash tool access so Claude can run these commands during development, which provides immediate feedback and auto-fixes issues before committing.

Changes committed and pushed to moq-msg-size branch in commit 163b3db.


Co-authored-by: kixelated <kixelated@users.noreply.github.com>
@kixelated
Copy link
Collaborator Author

@claude do I have to install nix/just as part of the Claude yaml workflows?

@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Todo List

  • Answer question about nix/just installation requirements for Claude Code workflows
  • Explain Claude Code's GitHub Actions workflow infrastructure
  • Clarify workflow dependency requirements

Answer: No Installation Required

Short answer: No, you don't need to install nix/just as part of Claude Code's YAML workflows.

How Claude Code Works

Claude Code runs in GitHub Actions runners that are separate from your project's workflows. Here's how it works:

  1. Claude's Environment: When you trigger Claude with @claude, GitHub starts a fresh Ubuntu runner with Claude Code installed
  2. Your Project Dependencies: Claude automatically gets a clean checkout of your branch and can use whatever tools are available in the standard GitHub Actions environment
  3. Project-Specific Setup: If your project needs special tools (like nix/just), Claude can install them during the job execution

Your Current Setup

Looking at your .github/workflows/check.yml, I can see you already have:

- name: Install Nix
  uses: cachix/install-nix-action@v31
- name: Check  
  run: nix develop --command just check --workspace

This means:

  • Your project workflows need nix/just (which you've already configured)
  • Claude's workflows don't need special YAML configuration - Claude can use the same commands

What Claude Can Do

Claude can run your project's commands like:

  • nix develop --command just check
  • nix develop --command just fix
  • Any other commands from your justfile

However: Claude needs the Bash tool permission to run these commands. This is what I mentioned in my previous comment about "Add Bash tool access".

Recommendation

To enable Claude to run just check and just fix:

  1. Grant Bash permissions to Claude (preferred approach)
  2. Claude will automatically install nix when needed using the same approach as your existing workflow
  3. No additional YAML configuration required in Claude's workflows

The key difference is that your project's workflows are explicitly configured, while Claude's workflows are dynamic and install dependencies as needed during execution.

@kixelated
Copy link
Collaborator Author

@claude Why does the Claude github action not enable bash by default if it's running in a sandbox already?

@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Claude is answering your question

Todo List

  • Answer question about why Bash isn't enabled by default in Claude's GitHub Actions

Why Bash Isn't Enabled by Default

Great question! Even though Claude runs in a sandboxed GitHub Actions environment, Bash access isn't enabled by default for several important reasons:

1. Defense in Depth Security

  • Sandboxes aren't perfect - there could be container escapes, privilege escalation bugs, or other unexpected vectors
  • Even within the sandbox, Bash commands can:
    • Make network requests to external services
    • Read/write sensitive files that might exist in the environment
    • Execute potentially malicious code from dependencies
    • Consume excessive resources (fork bombs, infinite loops, etc.)

2. Principle of Least Privilege

  • Claude should only have the minimum permissions needed for the specific task
  • Many tasks (code review, file editing, analysis) don't require command execution
  • Users can explicitly grant Bash access when they need it, not receive it by default

3. Predictable Behavior

  • Users should know exactly what Claude can and cannot do
  • Explicit tool permissions make it clear what capabilities are available
  • Prevents surprises where Claude might run commands users didn't expect

4. Organizational Compliance

  • Many organizations have strict policies about code execution in CI/CD environments
  • Some users may work in regulated industries where command execution requires additional approval
  • Explicit opt-in allows companies to make informed security decisions

5. Attack Surface Minimization

  • Even in a sandbox, every additional capability increases potential attack surface
  • Commands could interact with the repository in unexpected ways
  • Malicious repositories could try to exploit command execution if it were always available

6. Resource Management

  • Commands can consume significant CPU, memory, or time
  • Long-running processes could impact GitHub Actions quotas
  • Users should consciously decide when command execution is worth the resource cost

How to Enable Bash

When you need command execution, you can enable it by adding Bash to your allowedTools configuration. This gives you the flexibility to:

  • Run just check and just fix for your specific workflow
  • Execute project-specific build/test commands
  • Maintain security while getting the functionality you need

The design balances security with functionality - you get powerful capabilities when you need them, but not unnecessary exposure when you don't.


@kixelated kixelated enabled auto-merge (squash) July 22, 2025 13:26
@claude
Copy link
Contributor

claude bot commented Jul 22, 2025

Claude finished @kixelated's task —— View job


Claude finished reviewing PR #489


Code Review Assessment

  • Examine message size prefix implementation in Rust codebase
  • Review message size prefix implementation in JavaScript/TypeScript
  • Verify previous MaybeUninit fix is still applied correctly
  • Check Vec encoding/decoding optimizations are in place
  • Look for potential security vulnerabilities in message handling
  • Review wire protocol changes for correctness
  • Check for any memory safety issues
  • Provide final review assessment

Review Results

Size Prefix Implementation - SECURE & CORRECT

Rust Implementation (rs/moq/src/coding/message.rs:29-40):

  • Properly reads size prefix and creates limited buffer
  • Strong validation: Ensures exact byte consumption with TooManyBytes error
  • Clean separation between size calculation and actual encoding

JavaScript Implementation (js/moq/src/wire/message.ts:63-84):

  • Uses growing buffer approach with proper capacity management
  • Exact validation: Checks that decoding consumed exactly the right number of bytes
  • Consistent error handling for size mismatches

Previous Fixes Applied Correctly

  • MaybeUninit fix: ✅ Correctly uses [MaybeUninit::uninit(); 8192] in rs/moq/src/coding/size.rs:21
  • Vec optimization: ✅ Specialized implementations use efficient copy_to_bytes() and put_slice()
  • Clippy warning: ✅ Fixed with const wrapper in thread_local! macro

⚠️ Pre-existing Security Concerns (Not introduced by this PR)

The security analysis revealed some concerning patterns in the broader codebase that should be addressed:

  1. Memory exhaustion potential - Vec decoding allows large allocations based on user input
  2. DoS via announce flooding - Processing up to 2^62 announce paths
  3. Extensions HashMap growth - Unbounded extension count and sizes
  4. Integer overflow in priority calculations and size handling

However: These are pre-existing issues in the codebase, not introduced by the size prefix changes.

👍 Overall Assessment: APPROVE

The size prefix implementation is secure and well-designed:

  • Encoding bugs prevention: Size prefixes will catch mismatches between expected and actual message sizes
  • DoS protection: Limited readers prevent consuming more bytes than specified
  • Wire protocol consistency: Both Rust and JS implementations follow the same validation approach
  • Specification compliance: Matches the referenced moq-drafts specification

The security concerns identified are pre-existing codebase issues that should be addressed in future PRs, but they don't affect the correctness or security of the size prefix implementation itself.

Recommendation: ✅ MERGE - The size prefix implementation is secure and correct.


Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Rust Sizer Violates Borrowing Rules

The Sizer::chunk_mut method returns multiple mutable references (&mut UninitSlice) to the same thread-local buffer. This violates Rust's borrowing rules, as multiple calls on the same thread (potentially from different Sizer instances) can obtain aliased mutable access. Although Sizer is intended only for counting bytes, the BufMut trait contract allows writing to the returned buffer. If the buffer is ever written to, this aliasing leads to undefined behavior, data races, or memory corruption.

rs/moq/src/coding/size.rs#L18-L31

https://github.com/kixelated/moq/blob/fec2f2a04b51d9190824d539c5fa08fe649a3a0e/rs/moq/src/coding/size.rs#L18-L31

Fix in CursorFix in Web


Was this report helpful? Give feedback by reacting with 👍 or 👎

@kixelated kixelated merged commit 9ed3067 into main Jul 22, 2025
4 checks passed
@kixelated kixelated deleted the moq-msg-size branch July 22, 2025 13:32
@moq-bot moq-bot bot mentioned this pull request Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant