Dynamic parallel cpu DAG scheduler 🚀

[EricLBuehler]

Based on the idea of CUDA streams.

[Copilot]

Pull Request Overview

This PR implements a dynamic CPU DAG scheduler inspired by CUDA streams for improved parallel execution and concurrency. Key changes include:

• Replacing Rc/RefCell with thread-safe Arc, Mutex, and AtomicUsize in tensor ID management.
• Updating the buffer pool to use Arc<Mutex<…>> for thread safety.
• Revamping the DAG evaluation logic in the CPU storage module with concurrent scheduling using rayon and mpsc for synchronization.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
constensor-core/src/graph.rs	Replaces Rc/Cell with Arc/AtomicUsize for advanced thread-safe tensor ID handling.
constensor-core/src/cpu_storage/pool.rs	Updates shared pool type to Arc<Mutex<BufferPool>> for safe concurrent buffer management.
constensor-core/src/cpu_storage/mod.rs	Implements dynamic DAG scheduling with concurrency primitives and parallel task spawning.

Comments suppressed due to low confidence (2)

constensor-core/src/cpu_storage/mod.rs:408

• [nitpick] For consistency with the rest of the evaluation branches, consider using unreachable!("NoOp should not be evaluated") instead of panic!().

Op::NoOp => panic!("NoOp should not be evaluated"),

constensor-core/src/cpu_storage/mod.rs:420

• [nitpick] Variable names like 'pool2', 'ng2', 'res2', etc. are ambiguous; consider using more descriptive names to improve code clarity.

let pool2 = pool.clone();

[EricLBuehler]

Before:

cpu_graph_matmul_128x128
                        time:   [533.90 µs 535.85 µs 537.77 µs]

After

cpu_graph_matmul_128x128
                        time:   [388.75 µs 390.19 µs 391.65 µs]
                        change: [-27.543% -27.084% -26.626%] (p = 0.00 < 0.05)
                        Performance has improved.

This is ~30% faster!