docs: RFC by azchin · Pull Request #59 · sslab-gatech/oss-crs

azchin · 2026-02-09T18:09:15Z

RFC for the OSS-CRS interface, configurations, and CRS integration.

We appreciate any and all feedback! Leave a comment or review in this issue if you are interested in discussing.

HQ1995 · 2026-02-10T22:29:04Z

One major consideration is whether we should combine bug-finding and bug-fixing into a single RFC, as we are merging these two processes together.

david-a-wheeler · 2026-02-17T20:29:59Z

docs/RFC.md

+
+This RFC proposes a standardized set of interfaces and specifications for Cyber Reasoning Systems (CRSs) following the DARPA AIxCC competition.
+The aim is to unify bug finding and reporting workflows to make them portable across teams, scalable from laptops to clusters, and compatible with oss-fuzz.
+


I'm sure that's the short-term aim, but this overview text doesn't justify why anyone would care about unifying bug finding and reporting workflows. This overview should also give the "bigger picture" that I think you know but aren't expressing. I suggest adding something like this:

Suggested change

This unification enables CRSs to interoperate and accurately measure their capabilities, leading to steady improvements over time so that real-world software vulnerabilities can be automatically identified and fixed at scale, vastly reducing the risks to society from vulnerable software.

(If you want, you can accept this comment, but feel to write something else. My point is that you need to explain why this can help users and society, not just what it does, to put it in context.)

david-a-wheeler · 2026-02-17T20:33:18Z

docs/RFC.md

+
+**The implementation of this proposal: OSS-CRS ([https://github.com/sslab-gatech/oss-crs/tree/main](https://github.com/sslab-gatech/oss-crs/tree/main)) is an independent standard CRS framework that can test and run against OSS-Fuzz projects.**
+
+## Glossary and terms


I know what these are, but does the reader know? These are just acronym expansions, which is a good start, but you might want to follow each with a 1-sentence definition if you think some readers might not know what these are (as you already did with libCRS).

david-a-wheeler · 2026-02-17T20:35:21Z

docs/RFC.md

+This helps us understand where and how much LLMs can contribute without waiting for the entire campaign to finish.
+
+#### Pushing OSS-Fuzz Closer to developers via Diff Mode
+


Nowhere does this explain what "Diff Mode" is. I mean, I can guess, but if that's a key concept, it should be briefly defined in this section (since this section is all about Diff Mode).

david-a-wheeler

Here are several comments, I hope they help!

david-a-wheeler · 2026-02-17T20:36:59Z

docs/RFC.md

+
+## Background
+
+- **CRS (Cyber Reasoning System):** automation systems that find and patch vulnerabilities.


Shouldn't this be in the definitions/glossary earlier, so you don't need to repeat it here?

david-a-wheeler · 2026-02-17T20:38:25Z

docs/RFC.md

+## Standardization Proposals / CRS Integration Topology
+
+### Command Structure
+


Add some intro text, e.g.:

Suggested change

Here are examples of command-line commands that our work would enable:

david-a-wheeler · 2026-02-17T20:42:00Z

docs/RFC.md

+
+### LLM Configuration
+
+LLM configuration uses the native [LiteLLM Proxy Configuration](https://docs.litellm.ai/docs/proxy/configs) format. The configuration file path is specified in the CRS-Compose file via `llm_config.litellm_config`.


You suddenly introduce litellm here, without any fanfare, yet it's implicitly defining key parts of the interface.

I don't object to the use of litellm, but since it's embedded in the proposal, you really should introduce litellm earlier & why you're using it. E.g., "As part of this proposal we propose embedding the litellm Python library...." Explain briefly why you're using it, why not some other library. Then it won't be a surprise later on.

david-a-wheeler · 2026-02-17T20:43:35Z

docs/RFC.md

+### Diff Mode Support
+
+Diff Mode lets the CRS focus on code changes between commits instead of fuzzing the whole project.
+It's useful for validating patches, PRs, or a series of commits where you only want to explore the modified code paths.


That's a fine explanation of Diff Mode, except you already introduced Diff Mode earlier. You should explain what Diff Mode is/does the first time you use the term. Maybe that goes early in the document, to make sure readers are aware of it. Diff Mode seems like a key new capability that deserves early discussion.

david-a-wheeler · 2026-02-17T20:46:38Z

docs/RFC.md

+
+### CRS Ensemble Support
+
+**Motivation:** Combine pure agents with fuzzers for maximum coverage.


I'm not sure what a "pure" agent is. Is this only using AI? Only doing static analysis?

I'm all for combining and using fuzzers, but I think we should use all tools available for finding vulnerabilities - if a traditional static analysis tool, or an AI reading code, can help find vulnerabilities, great! Indeed, I'd love to see human inputs as a potential source (e.g., "I'm especially concerned that the interaction of X and Y might produce Z, can you look into that?").

david-a-wheeler · 2026-02-17T20:47:57Z

docs/RFC.md

+
+#### Environment Variables
+
+CRSs receive the following environment variables:


Suggested change

CRSs receive the following environment variables:

CRSs may receive the following environment variables:

Are all of these strictly required? I doubt it. I suggest that these be as optional as you can make them.

david-a-wheeler · 2026-02-17T20:48:49Z

docs/RFC.md

+- Secure handling of LLM API keys.
+- Resource isolation between CRS instances.
+
+## References


I would expect the AIxCC competition (at least some key docs) be included in the references. What about the oss-fuzz documentation?

Signed-off-by: Andrew Chin <achin34@gatech.edu>

david-a-wheeler reviewed Feb 17, 2026

View reviewed changes

azchin added 5 commits February 23, 2026 13:01

docs: RFC port and update

09d4a40

Signed-off-by: Andrew Chin <achin34@gatech.edu>

fix(rfc): incorporate feedback

eace983

Signed-off-by: Andrew Chin <achin34@gatech.edu>

docs(rfc): add patching related parts

40b999e

Signed-off-by: Andrew Chin <achin34@gatech.edu>

docs(rfc): uncomment cli interface

0641e09

Signed-off-by: Andrew Chin <achin34@gatech.edu>

docs(rfc): libCRS builder sidecar and snapshot

1609704

Signed-off-by: Andrew Chin <achin34@gatech.edu>

azchin force-pushed the docs/rfc branch from 36920cc to 1609704 Compare February 23, 2026 20:20

Merge branch 'main' into docs/rfc

255e542


		This RFC proposes a standardized set of interfaces and specifications for Cyber Reasoning Systems (CRSs) following the DARPA AIxCC competition.
		The aim is to unify bug finding and reporting workflows to make them portable across teams, scalable from laptops to clusters, and compatible with oss-fuzz.


	This unification enables CRSs to interoperate and accurately measure their capabilities, leading to steady improvements over time so that real-world software vulnerabilities can be automatically identified and fixed at scale, vastly reducing the risks to society from vulnerable software.


		The implementation of this proposal: OSS-CRS ([https://github.com/sslab-gatech/oss-crs/tree/main](https://github.com/sslab-gatech/oss-crs/tree/main)) is an independent standard CRS framework that can test and run against OSS-Fuzz projects.

		## Glossary and terms

		This helps us understand where and how much LLMs can contribute without waiting for the entire campaign to finish.

		#### Pushing OSS-Fuzz Closer to developers via Diff Mode


		## Background

		- CRS (Cyber Reasoning System): automation systems that find and patch vulnerabilities.

		## Standardization Proposals / CRS Integration Topology

		### Command Structure



	Here are examples of command-line commands that our work would enable:


		### LLM Configuration

		LLM configuration uses the native [LiteLLM Proxy Configuration](https://docs.litellm.ai/docs/proxy/configs) format. The configuration file path is specified in the CRS-Compose file via `llm_config.litellm_config`.


		### CRS Ensemble Support

		Motivation: Combine pure agents with fuzzers for maximum coverage.


		#### Environment Variables

		CRSs receive the following environment variables:

	CRSs receive the following environment variables:
	CRSs may receive the following environment variables:

Conversation

azchin commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HQ1995 commented Feb 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

david-a-wheeler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

azchin commented Feb 9, 2026 •

edited

Loading