From 58fecdfb29be10e7e953b410e7d047ef299cbe31 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 6 Feb 2024 15:49:01 -0800 Subject: [PATCH 1/9] adds initial draft of memory safety continuum Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 64 +++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 docs/memory-safety-continuum.md diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md new file mode 100644 index 0000000..82a71f3 --- /dev/null +++ b/docs/memory-safety-continuum.md @@ -0,0 +1,64 @@ +# Memory Safety Continuum + +At this SIG's January 25, 2024 meeting, we discussed the idea of a continuum when it comes to software memory safety. + +When discussing memory safety, it's easy to fall into the idea of a memory safety binary - either your software is "memory safe" or it is not. When considering reality - especially with re: to so much existing software (legacy or not) - it is more nuanced. + +This DRAFT document lays out the idea of the "Memory Safety Continuum". Eventually, this idea may be used as a foundation for other content which will help developers and organizations identify where they are on the continuum and how to get to where they want to be. + +## What is Memory Safety? + +Rather than using terms like "Memory Safe Language" and "Memory Unsafe Language", this SIG prefers the terms "memory safe by default" and "non-memory safe by default". Please see our [useful definitions](definitions.md) file for more information about memory safety and undefined behavior. + + +## The Continuum + +This is a very rough idea of what the continuum might look like - from "least safety" to "most safety" + +* Using a non-memory safe by default language (such as C or C++) without any developer best practices or automated tooling to check for memory safety +* Using a non-memory safe by default language with developer best practices, but no automated tooling to check for memory safety +* Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety +* Using a memory safe by default language (such as Rust, Go, Python, Java, JavaScript, C#) without developer best practices and automated tooling +* Using a memory safe by default language with developer best practices +* Using a memory safe by default language with developer best practices and automated tooling + +### Using a non-memory safe by default language without an developer best practies or automated tooling + +Using raw C or C++ (or old versions of C and C++ language and compiler) + +### Using a non-memory safe by default language with developer best practices, but no automated tooling + +Examples +* [Using attributes such as `cleanup` and classes when writing C](https://lwn.net/Articles/934679/) (and depending on developers to manually check this) +* Following the [C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines) when writing C++ +* Using the [C++ Compiler Hardening Guide](https://github.com/ossf/wg-best-practices-os-developers/tree/main/docs/Compiler-Hardening-Guides) when compiling C++ code + +### Using a non-memory safe by default language with developer best practices and automated tooling + +TO DO + +### Using a memory safe by default language without developer best practices or automated tooling + +Examples +* Using unsafe blocks in Rust [without assessing the entire module in which the unsafe code appears](https://github.com/ossf/Memory-Safety/issues/15#issuecomment-1847939439) +* Using the [no_mangle](https://github.com/rust-lang/rust/issues/28179) attribute in Rust + +### Using a memory safe by default language with developer best practices, but no automated tooling + +Examples +* Following the [Rustnomicon](https://doc.rust-lang.org/nomicon/intro.html) careful practices when using unsafe blocks in Rust +* Following best practices (LINK NEEDED) when using the Go [unsafe](https://pkg.go.dev/unsafe#pkg-overview) package +* Following [Javascript Memory Management](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management) practices + +### Using a memory safe by default language with developer best practices and automated tooling + +Examples +* Using the [Go Data Race Detector](https://go.dev/doc/articles/race_detector) +* Using other tools such as [govulncheck, fuzzing, and vet](https://go.dev/doc/security/best-practices) when writing Go code + +## FAQ + +### Why do you rank using automated tooling higher than just using developer best practices? + +The amount software that has already been produced is staggering - and it is only growing every day. Expecting every developer to manually check their code (and the code their code interacts with) is no longer practical or possible. Automated tooling must be used to catch (at the least) known, common vulnerabilities that a developer may unintentionally introduce in their code. + From 403991c6a8311c71d7d91227d88f0b67cc105de9 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 6 Feb 2024 15:53:41 -0800 Subject: [PATCH 2/9] fixes markdown errors Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index 82a71f3..844f997 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -4,13 +4,12 @@ At this SIG's January 25, 2024 meeting, we discussed the idea of a continuum whe When discussing memory safety, it's easy to fall into the idea of a memory safety binary - either your software is "memory safe" or it is not. When considering reality - especially with re: to so much existing software (legacy or not) - it is more nuanced. -This DRAFT document lays out the idea of the "Memory Safety Continuum". Eventually, this idea may be used as a foundation for other content which will help developers and organizations identify where they are on the continuum and how to get to where they want to be. +This DRAFT document lays out the idea of the "Memory Safety Continuum". Eventually, this idea may be used as a foundation for other content which will help developers and organizations identify where they are on the continuum and how to get to where they want to be. ## What is Memory Safety? Rather than using terms like "Memory Safe Language" and "Memory Unsafe Language", this SIG prefers the terms "memory safe by default" and "non-memory safe by default". Please see our [useful definitions](definitions.md) file for more information about memory safety and undefined behavior. - ## The Continuum This is a very rough idea of what the continuum might look like - from "least safety" to "most safety" @@ -28,7 +27,8 @@ Using raw C or C++ (or old versions of C and C++ language and compiler) ### Using a non-memory safe by default language with developer best practices, but no automated tooling -Examples +Examples: + * [Using attributes such as `cleanup` and classes when writing C](https://lwn.net/Articles/934679/) (and depending on developers to manually check this) * Following the [C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines) when writing C++ * Using the [C++ Compiler Hardening Guide](https://github.com/ossf/wg-best-practices-os-developers/tree/main/docs/Compiler-Hardening-Guides) when compiling C++ code @@ -39,20 +39,23 @@ TO DO ### Using a memory safe by default language without developer best practices or automated tooling -Examples +Examples: + * Using unsafe blocks in Rust [without assessing the entire module in which the unsafe code appears](https://github.com/ossf/Memory-Safety/issues/15#issuecomment-1847939439) * Using the [no_mangle](https://github.com/rust-lang/rust/issues/28179) attribute in Rust ### Using a memory safe by default language with developer best practices, but no automated tooling -Examples +Examples: + * Following the [Rustnomicon](https://doc.rust-lang.org/nomicon/intro.html) careful practices when using unsafe blocks in Rust * Following best practices (LINK NEEDED) when using the Go [unsafe](https://pkg.go.dev/unsafe#pkg-overview) package * Following [Javascript Memory Management](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management) practices ### Using a memory safe by default language with developer best practices and automated tooling -Examples +Examples: + * Using the [Go Data Race Detector](https://go.dev/doc/articles/race_detector) * Using other tools such as [govulncheck, fuzzing, and vet](https://go.dev/doc/security/best-practices) when writing Go code @@ -61,4 +64,3 @@ Examples ### Why do you rank using automated tooling higher than just using developer best practices? The amount software that has already been produced is staggering - and it is only growing every day. Expecting every developer to manually check their code (and the code their code interacts with) is no longer practical or possible. Automated tooling must be used to catch (at the least) known, common vulnerabilities that a developer may unintentionally introduce in their code. - From 157692096d70c55fe64daea2f1b0ad0bf26b6217 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 6 Feb 2024 15:57:24 -0800 Subject: [PATCH 3/9] more markdown fixing Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index 844f997..3fac679 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -29,7 +29,7 @@ Using raw C or C++ (or old versions of C and C++ language and compiler) Examples: -* [Using attributes such as `cleanup` and classes when writing C](https://lwn.net/Articles/934679/) (and depending on developers to manually check this) +* [Using attributes such as `cleanup` and classes when writing C](https://lwn.net/Articles/934679/) (and depending on developers to manually check this) * Following the [C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines) when writing C++ * Using the [C++ Compiler Hardening Guide](https://github.com/ossf/wg-best-practices-os-developers/tree/main/docs/Compiler-Hardening-Guides) when compiling C++ code From 85bc433d1b1e6b823bc76961acfc21ef3c545e6a Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 20 Feb 2024 15:12:59 -0800 Subject: [PATCH 4/9] expand categories to include checks for third party code Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index 3fac679..cd9671b 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -6,6 +6,10 @@ When discussing memory safety, it's easy to fall into the idea of a memory safet This DRAFT document lays out the idea of the "Memory Safety Continuum". Eventually, this idea may be used as a foundation for other content which will help developers and organizations identify where they are on the continuum and how to get to where they want to be. +## Audience + +The audience for this draft is developers - however, the audience may expand with future drafts/versions of this document. + ## What is Memory Safety? Rather than using terms like "Memory Safe Language" and "Memory Unsafe Language", this SIG prefers the terms "memory safe by default" and "non-memory safe by default". Please see our [useful definitions](definitions.md) file for more information about memory safety and undefined behavior. @@ -16,10 +20,12 @@ This is a very rough idea of what the continuum might look like - from "least sa * Using a non-memory safe by default language (such as C or C++) without any developer best practices or automated tooling to check for memory safety * Using a non-memory safe by default language with developer best practices, but no automated tooling to check for memory safety -* Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety +* Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code +* Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND automated tooling to check for memory safety in third party code (dependencies) * Using a memory safe by default language (such as Rust, Go, Python, Java, JavaScript, C#) without developer best practices and automated tooling -* Using a memory safe by default language with developer best practices -* Using a memory safe by default language with developer best practices and automated tooling +* Using a memory safe by default language with developer best practices, but no automated tooling to check for memory safety +* Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code +* Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND automated tooling to check for memory safety in third party code (dependencies) ### Using a non-memory safe by default language without an developer best practies or automated tooling @@ -33,7 +39,11 @@ Examples: * Following the [C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines) when writing C++ * Using the [C++ Compiler Hardening Guide](https://github.com/ossf/wg-best-practices-os-developers/tree/main/docs/Compiler-Hardening-Guides) when compiling C++ code -### Using a non-memory safe by default language with developer best practices and automated tooling +### Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code + +TO DO + +### Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND automated tooling to check for memory safety in third party code (dependencies) TO DO @@ -52,13 +62,17 @@ Examples: * Following best practices (LINK NEEDED) when using the Go [unsafe](https://pkg.go.dev/unsafe#pkg-overview) package * Following [Javascript Memory Management](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management) practices -### Using a memory safe by default language with developer best practices and automated tooling +### Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code Examples: * Using the [Go Data Race Detector](https://go.dev/doc/articles/race_detector) * Using other tools such as [govulncheck, fuzzing, and vet](https://go.dev/doc/security/best-practices) when writing Go code +### Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND third party code + +TO DO + ## FAQ ### Why do you rank using automated tooling higher than just using developer best practices? From a985d789974474b2ad678592e3dfd823289b4988 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 20 Feb 2024 15:41:55 -0800 Subject: [PATCH 5/9] expanding FAQs Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index cd9671b..75aaec5 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -53,6 +53,7 @@ Examples: * Using unsafe blocks in Rust [without assessing the entire module in which the unsafe code appears](https://github.com/ossf/Memory-Safety/issues/15#issuecomment-1847939439) * Using the [no_mangle](https://github.com/rust-lang/rust/issues/28179) attribute in Rust +* Using compiler options which turn off some or all of the compiler's memory safety checks ### Using a memory safe by default language with developer best practices, but no automated tooling @@ -75,6 +76,24 @@ TO DO ## FAQ +### Is there a way to be COMPLETELY memory safe? + +In theory, yes. In reality...it's very hard. It is theoretically possible to write C++ code in a way that is free from memory safety bugs. In practice, however, we still see a very large amount of memory safety related security vulnerabilities every year - known vulnerabilities that could be prevented if the code were written in a memory safe by default language. + +This is not to say that using a memory safe by default language will protect you from all vulnerabilities. Say something is "default" also implies that there are ways of using the language in a non default way. This is also true of your software's dependencies - for example, your Rust code may be free of unsafe blocks (or, at least, use them sparingly), but an Open Source package you depend on may not be. Evaluating the safety of your software includes evaluating anything your software depends on. + +It is also possible that your software written in a memory safe by default language will need to interface with software written in a non-memory safe by default language (for example, Rust code which must interface with a C++ driver). + +No matter where your software is on this memory safety continuum, you will still need to exercise some level of personal/professional judgement on what is an acceptable amount of risk and what is not. + +### Are you saying we should just re-write billions of lines of C and C++ code in Rust? + +No. + +This is not a real world possibility. Even if it were, it would not be practical nor would it be truly helpful. + +There are times a rewrite of a particular section of software (such as one where vulnerabilities are particularly prevalent) may be necessary. However, this SIG's focus is not on rewriting the world in Rust (or any other language). Instead, it is on helping developers (and others) understand the memory safety of their code now and how to improve it in meaningful and achievable ways. + ### Why do you rank using automated tooling higher than just using developer best practices? The amount software that has already been produced is staggering - and it is only growing every day. Expecting every developer to manually check their code (and the code their code interacts with) is no longer practical or possible. Automated tooling must be used to catch (at the least) known, common vulnerabilities that a developer may unintentionally introduce in their code. From 368f817f12e8f96357d0dabee439cc236d99abd8 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Wed, 21 Feb 2024 13:17:02 -0800 Subject: [PATCH 6/9] fixes a few grammar issues Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index 75aaec5..87f11df 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -78,9 +78,9 @@ TO DO ### Is there a way to be COMPLETELY memory safe? -In theory, yes. In reality...it's very hard. It is theoretically possible to write C++ code in a way that is free from memory safety bugs. In practice, however, we still see a very large amount of memory safety related security vulnerabilities every year - known vulnerabilities that could be prevented if the code were written in a memory safe by default language. +In theory, yes. In reality...it's very hard. It is theoretically possible to write C++ code in a way that is free from memory safety bugs. In practice, however, we still see a very large number of memory safety related security vulnerabilities every year - known vulnerabilities that could be prevented if the code were written in a memory safe by default language. -This is not to say that using a memory safe by default language will protect you from all vulnerabilities. Say something is "default" also implies that there are ways of using the language in a non default way. This is also true of your software's dependencies - for example, your Rust code may be free of unsafe blocks (or, at least, use them sparingly), but an Open Source package you depend on may not be. Evaluating the safety of your software includes evaluating anything your software depends on. +This is not to say that using a memory safe by default language will protect you from all vulnerabilities. Saying something is "default" also implies that there are ways of using the language in a non default way. This is also true of your software's dependencies - for example, your Rust code may be free of unsafe blocks (or, at least, use them sparingly), but an Open Source package you depend on may not be. Evaluating the safety of your software includes evaluating anything your software depends on. It is also possible that your software written in a memory safe by default language will need to interface with software written in a non-memory safe by default language (for example, Rust code which must interface with a C++ driver). From 4f1a5ed9078b75a43c91984675dad04d94c8287b Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Wed, 21 Feb 2024 13:18:44 -0800 Subject: [PATCH 7/9] makes wording more consistent Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index 87f11df..e077389 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -18,7 +18,7 @@ Rather than using terms like "Memory Safe Language" and "Memory Unsafe Language" This is a very rough idea of what the continuum might look like - from "least safety" to "most safety" -* Using a non-memory safe by default language (such as C or C++) without any developer best practices or automated tooling to check for memory safety +* Using a non-memory safe by default language (such as C or C++) without developer best practices or automated tooling to check for memory safety * Using a non-memory safe by default language with developer best practices, but no automated tooling to check for memory safety * Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code * Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND automated tooling to check for memory safety in third party code (dependencies) @@ -27,7 +27,7 @@ This is a very rough idea of what the continuum might look like - from "least sa * Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code * Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND automated tooling to check for memory safety in third party code (dependencies) -### Using a non-memory safe by default language without an developer best practies or automated tooling +### Using a non-memory safe by default language without developer best practies or automated tooling Using raw C or C++ (or old versions of C and C++ language and compiler) From 1e72bebcba0deb3bbaced6725141cdf2ed291984 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 2 Apr 2024 14:49:22 -0700 Subject: [PATCH 8/9] makes a few changes based on code review Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index e077389..d4afecf 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -14,7 +14,7 @@ The audience for this draft is developers - however, the audience may expand wit Rather than using terms like "Memory Safe Language" and "Memory Unsafe Language", this SIG prefers the terms "memory safe by default" and "non-memory safe by default". Please see our [useful definitions](definitions.md) file for more information about memory safety and undefined behavior. -## The Continuum +## The Continuum - Understanding Where Your Software Is Now This is a very rough idea of what the continuum might look like - from "least safety" to "most safety" @@ -38,6 +38,7 @@ Examples: * [Using attributes such as `cleanup` and classes when writing C](https://lwn.net/Articles/934679/) (and depending on developers to manually check this) * Following the [C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines) when writing C++ * Using the [C++ Compiler Hardening Guide](https://github.com/ossf/wg-best-practices-os-developers/tree/main/docs/Compiler-Hardening-Guides) when compiling C++ code +* Isolating code that processes un-trusted data from code that performs direct memory management operations or uses raw pointers (see [Language-theoretic Security](https://github.com/ossf/Memory-Safety/pull/20)) ### Using a non-memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code @@ -72,6 +73,11 @@ Examples: ### Using a memory safe by default language with developer best practices and automated tooling to check for memory safety in first party code AND third party code +* Using a fuzzer such as [AFL++](https://github.com/AFLplusplus/AFLplusplus) on both your own code and third party code +* TO DO - add more + +## The Continuum - Getting to a Better State with re: to memory safety + TO DO ## FAQ From 935898dccf6d0fe1a4a66fe6d0a005b09e706702 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 2 Apr 2024 14:54:48 -0700 Subject: [PATCH 9/9] appease markdown checker Signed-off-by: Nell Shamrell --- docs/memory-safety-continuum.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/memory-safety-continuum.md b/docs/memory-safety-continuum.md index d4afecf..be828e7 100644 --- a/docs/memory-safety-continuum.md +++ b/docs/memory-safety-continuum.md @@ -90,7 +90,7 @@ This is not to say that using a memory safe by default language will protect you It is also possible that your software written in a memory safe by default language will need to interface with software written in a non-memory safe by default language (for example, Rust code which must interface with a C++ driver). -No matter where your software is on this memory safety continuum, you will still need to exercise some level of personal/professional judgement on what is an acceptable amount of risk and what is not. +No matter where your software is on this memory safety continuum, you will still need to exercise some level of personal/professional judgement on what is an acceptable amount of risk and what is not. ### Are you saying we should just re-write billions of lines of C and C++ code in Rust?