How to make C++ API changes Backwards Compatible

I made this project to showcase some "tricks" on how to make backwards compatible API changes. (E.g. renaming files, types, changing parameter types, the return type, default parameters, an enum to enum class etc.)

I added some tests (in /tests) to ensure that these tricks are indeed backwards compatible.

There are also negative tests (see /neg-tests) to showcase some rare cases where the API change breaks code that previously compiled.

Contributing

This project just got created.

Please feel free to create Issues and Pull Requests to improve this list.

Warning

These tricks assume the users of the library API don't rely on ABI compatibility as well, or use forward declarations or function pointers (since, in general, they shouldn't do so with foreign types unless explicitly pointed out by the library).

Project internals

/some_unstable_lib contains a header file for each API change in the list further down. Each header exposes both the old API and the changed API based on the BC_API_CHANGED macro.

/tests represents the users of the library whose code must compile before and after the API change.

/neg-tests contains code that should not compile after the API change.

.github/workflows/deploy.yml updates Github Pages with the new changes from this README.

.github/workflows/cmake.yml checks if the tests in /tests compile with the macro BC_API_CHANGED OFF (before the API change) and ON (after). It also checks that the negative tests compile before the API change, and don't compile after.

Overview

• Rename a namespace
• Rename a type
• Rename a header
• Change default parameters
• Change the return type
• Change old-style enum to enum class
• Reasonably safe changes

How to:

Rename a namespace

Initial code:

namespace path::to::v1 { ... }

Scenario:

We maybe need to change the namespace name to fix a typo. We will change it from path::to::v1 to path::to::v2.

Solution:

Rename the old namespace to the new one and add a namespace alias for the old one.

+ namespace path::to::v2 {}
+ namespace path::to {
+   namespace v1 = path::to::v2;
+ }
+ 
- namespace path::to::v1 { ... }
+ namespace path::to::v2 { ... }

Remarks:

• [[deprecated]] attribute doesn't work on namespace aliases. You can try compiler specific directives (Eg. #pragma deprecated(keyword) for msvc)

• The empty namespace namespace path::to::v2 {} was added at the top of the file for visibility purposes

Relevant Files:

• API: NamespaceRename.hpp
• User: NamespaceRenameTest.cpp

Rename a type

Initial code:

struct OldName { ... };

Scenario:

We maybe need to update the struct name to fix a typo. We will change it to NewName.

Solution:

We can use a type alias.

- struct OldName { ... };
+ struct NewName { ... };
+ using OldName = NewName;

Remarks:

• You can deprecate the old OldName.
• The users might learn the hard way that they shouldn't forward declare foreign types.

Relevant Files:

• API: StructRename.hpp
• User: StructRenameTest.cpp

Rename a header

Initial code:

// v1/OldName.hpp:
...

Scenario:

We need to rename the header to v2/NewName.hpp.

Solution:

1. Rename the header:

- // v1/OldName.hpp:
+ // v2/NewName.hpp:
...

2. Create a compatibility header file in the old location that includes the renamed one.

// v1/OldName.hpp: <- created to only include the renamed header + deprecation notice
#include "v2/NewName.hpp"

// You can also deprecate it by inserting a compilation warning:
// #warning OldName.hpp is deprecated, include "v2/NewName.hpp".`
// Don't use #error since there is no way for users to silence it.

Remarks:

Rename using your versioning tool (Git/SVN) so you don't lose blame history. For Git, do the change 2 steps in 2 different commits.

Change default parameters

Initial code:

void SomeMethod(
    int mandatory,
    bool opt1 = false,
    float opt2 = 1e-6f,
    int opt3 = 42
) { ... }

Scenario:

This method receives too many default parameters, and it only becomes harder for users to call it with only 1 or 2 parameters changed. We need to change the method to receive a struct containing these parameters instead.

Solution:

If you would just overload SomeMethod with the default parameters changed, users calling SomeMethod with just the mandatory parameters will now have the compiler complain about ambiguity (that it doesn't know which of the 2 methods to call).

To tell the compiler to prefer the newer method we need to make the old one less specialized by making it a template.

+ template<int = 0>
void SomeMethod(
    int mandatory,
    bool opt1 = false,
    float opt2 = 1e-6f,
    int opt3 = 42
+ ) {
+  // Call the new implementation now
+  SomeMethod(mandatory, SomeMethodOpts{opt1, opt2, opt3});
+ }
+ 
+ struct SomeMethodOpts { bool opt1 = false; float opt2 = 1e-6; int opt3 = 42; };
+ void SomeMethod(
+     int mandatory,
+     SomeMethodOpts opts = {}
) { ... }

Remarks:

• You can deprecate the old SomeMethod (now a template)
• If the definition needs to be in the .cpp file, and the function is dll exported, you need to explicitly instantiate the templated SomeMethod in the .cpp file

template<> DLL_EXPORT void SomeMethod<0>(
    int mandatory, bool opt1 = false, float opt2 = 1e-6f, int opt3 = 42);

Relevant Files:

• API: MethodDefaultParams.hpp
• User: MethodDefaultParamsTest.cpp

Change the return type

Warning:

Prefer to just add a new method called slightly different instead. What's about to follow is over-engineered.

In short: we will overload the implicit cast operator of the new return type, and if the return type needs to be a primitive, we will create a new wrapper class.

Initial code:

// (1) change some primitive `T` to `NewUserDefT`
bool CheckPassword(std::string);

// (2) change some primitive `const T&` to primitive `T`
struct Strukt {
  const float& GetMemF() const { return m_memF; }
private:
  float m_memF;
};

Scenario:

(1) CheckPassword method returns true if it succeeds, otherwise false. Make this method return some meaningful error message so the user knows why it failed (why it returned false).

(2) Strukt::GetMemF returns a primitive type as const& which is bad for multiple reasons (performance, lifetime, complexity issues). We need to return by value.

Unfortunately, we cannot just overload a function by return type and then deprecate it.

Solution:

For situation (1): Add operator bool() so that the new type can be implicitly casted to bool.

// (1) change primitive `T` to `NewUserDefT`
+ struct CheckPasswordResult { // mimics std::expected<void, std::string>
+     operator bool() const { return !m_errMsg.has_value(); }
+     const std::string& error() const { return m_errMsg.value(); }
+ private:
+     std::optional<std::string> m_errMsg;
+ };
- bool CheckPassword(std::string);
+ CheckPasswordResult CheckPassword(std::string);

• (1.1): If you want to take it a step further, and allow implicit casts only to bool, since C++20 you can make the cast operator conditionally explicit (In the tests, int x = CheckPassword(""); doesn't compile after the API change, while bool x = CheckPassword(""); does. See neg-tests/ReturnTypeChangeTest.cpp)

For situation (2): Add a new class GetterRetT with 2 implicit cast operators to NewRetT and to OldRetT. "Mark" the implicit cast operator to OldRetT as deprecated and as "less specialized" (i.e. as template, so that the compiler will choose at "overload resolution" the NewRetT overload).

Additionally, inside the Strukt return GetterRetT by const& so that we avoid runtime exceptions from dangling references in user's code in case they have a StruktWrapper class that also has a const float& GetMemF() that called and returned the result of our GetMemF().

// (2) change primitive `const T&` to primitive `T`
+ struct GetterRetT {
+   template <int = 0> // (2.1)
+   operator OldRetT () const { ... }
+   operator NewRetT () const { ... }
+ };

struct Strukt {
-   const float& GetMemF() const { return m_memF; }
+   const GetterRetT& GetMemF() const { return m_memF; }
private:
-   float m_memF = 3.f;
+   GetterRetT m_memF = 3.f;
};

Relevant Files:

• API: include/ReturnTypeChange.hpp include/ReturnTypeChangeByValue.hpp
• User: tests/ReturnTypeChangeTest.cpp tests/ReturnTypeChangeByValueTest.hpp
• Neg: neg-tests/ReturnTypeChangeTest.cpp

Change old-style enum to enum class

Warning:

Changing the enum to enum class will inherently breaks implicit conversions to integers (e.g. when the enum is used as bit flags: STYLE_BOLD | STYLE_ITALLIC results in a int).

Initial code:

enum Style {
    STYLE_BOLD,
    STYLE_ITALLIC,
    STYLE_STRIKE_THROUGH,
};

Scenario:

We need to modernize the API to use enum class instead.

Solution:

In order to not break scoped uses of the enum (e.g. auto style = Style::STYLE_BOLD) we will duplicate the enum fields with the enum class's naming style, and make sure their value is assigned to the old enum fields.

In order to not break unscoped uses of the enum (e.g. auto style = STYLE_BOLD), we will define static variables for each enum entry.

- enum Style {
+ enum class Style {
+   Bold,
+   Itallic,
+   StrikeThrough,
-   STYLE_BOLD,
-   STYLE_ITALLIC,
-   STYLE_STRIKE_THROUGH,
+   STYLE_BOLD = Bold,
+   STYLE_ITALLIC = Itallic,
+   STYLE_STRIKE_THROUGH = StrikeThrough,
};

+ static inline Style STYLE_BOLD = Style::Bold;
+ static inline Style STYLE_ITALLIC = Style::Itallic;
+ static inline Style STYLE_STRIKE_THROUGH = Style::StrikeThrough;

Remarks:

• Inspired by the memory_order change in the standard

• If the enum was used as bit flags, define bitwise operators as well. And if there were methods that recieved the unscoped enum as int, overload them to receive the scoped enum now (since the bitwise operators return the scoped enum, if they were to return an int, users will not be able to chain more than 2 of them: e.g. Print(STYLE_BOLD | STYLE_ITALLIC | STYLE_STRIKE_THROUGH) // operator|(Style, int) is not overloaded).

// Add `friend` if the enum lies inside a `struct`
[friend] inline Style operator|(Style lhs, Style rhs) {
    return static_cast<Style>(static_cast<int>(lhs) | static_cast<int>(rhs));
}

Relevant Files:

• API: ChangeToEnumClass.hpp
• User: ChangeToEnumClassTest.cpp

Other reasonably safe changes:

• Adding const to a member function (T Get(); -> T Get() const;)
• Making a member function static (T Get() const; -> static T Get();)
  • the code obj.Get(..) will still compile.
• Adding [[nodiscard]] ([[nodiscard]] int Get() or class [[nodiscard]] Result)
  • should not be added to any random class or methods that have side effects (the user might have called the method for its side effectes)
• Adding explicit to a constructor with only 1 parameter.
  • except for classes that are expected to be implicitly constructed from that 1 parameter.
• Removing the const& when passing a copyable parameter (void Set(const T&); -> void Set(T);)
  • e.g. cannot remove const& from void Set(const std::unique_ptr<T>&); since unique_ptr is not copyable
• Removing the const when returning by value: const RetT Get()
  • except from non-private virtual methods, since the user's derived class might overide them.
  • this change can break some rare cases (see neg-tests/RemoveConstReturnByValueTest.cpp):
    • auto& val = Get(); was valid code until we removed the const
    • the return type was passed directly to a method with 2 overloads Foo(const RetT&) to Foo(RetT&), it now calls the non-const one.
    • if Get returns Base now by value and Derived has an implicit constructor from Base, you can get incompatible types in ternary operators: cond ? const Derived : Base (cond ? Derived{} : Get();)
• Converting a class that has only static methods to a namespace
  • make sure the constructors and operator= are private, otherwise deprecate them before changing the class to a namespace
• Changing the underlying type of an enum (e.g. from enum Flags to enum Flags: uint64_t, which happens when the enum is used as bit flags and we need to add another entry after 1<<31)
  • except if users depend on sizeof(Flags), e.g. if they serialize it to binary data
  • int x = Flags::X, where X=1<<31, will still compile and it will not overflow, even if Flags is now of type uint64_T, since the compiler sees that the value of X still fits inside int
  • users will get warnings if they compile with -Wconversion and assign a value of type Flags into a now narrower type like an int

TODOs

• Add tests to ensure no breakings for dynamic libraries. (So we check ODR violations as well)
• Change a base class by making the new one extend from the old one
• ...