Shared Libraries - Binary Components

C-Interoperability and Shared Libraries - Binary Components

Overview

Shared libraries are compiled object-code exporting functions, data structures and classes which can be used by many different programs. They provide many benefits such as: faster compilation time; less disk space usage since each program using the library do not need a copy of the library source code or its compiled-code; ability to update the library and provide security fixes to client applications without recompilation, just by replacing the library file.

On Windows, shared libraries are called DLLs which stands for Dynamic Linked Libraries, on Unix-like operating systems such as Linux, BSD and MacOSX, they are called SO - shared objects or DSO - dynamic shared objects.

Shared libraries are similar to executables, however they do not have the main() function entry point.

System			Extension
Operating	Long Name	Short name	File	Binary Format
Windows	Dynamic Linked Library	DLL	.dll	PE32/PE64 - Portable Executable
Linux	(Dynamic) Shared Object	DSO or SO	.so	ELF or ELF64 (for 64 bits processors)
BSD	(Dynamic) Shared Object	DSO or SO	.so	ELF or ELF64
MacOSX	-	dylib	.dylib or .so	MachO

In addition to C and C++, shared libraries, containing functions with C-linkage, can also be used as binary components (libraries) and be consumed from higher level programming languages such as Python, Ruby, Java or C# through some foreign-function interface. In Python C-functions can be imported from shared libraries using the cytpes library; in C# C-functions can be consumed using the P-invoke API and in Java it is possible to use C-functions from shared libraries using the JNI (Java Native Interface) or JNA, which is easier to use.

Note:

Shared libraries are specific to a particular operating system and are not part of C++ standard.

Use Cases and advantages:

Multiple programs can reuse the same library reducing disk space.
Faster compilation once the shared library is compiled.
Changes in the library such as security fixes, updates and new features may not require client programs recompilation if the ABI is not broken.
Better for security updates. For instance, if a program is statically compiled with open-ssl library, a program recompilation would be required to update the library to a new version with security fixes for some vulnerability. It could even be a greater security problem and waste of space, if all application depending on open-ssl were statically liked with this library. If a program uses the open-sll shared library, the library can be updated just by replacing a file and the program it will not need to be recompiled as long as there are no ABI breaking changes.
Plugin system => The main program can contain an interface class called IInterface and load its implementations from shared libraries plugins. The implementation classes provided by the plugins or shared library files allow the program to be updated and extended at runtime without recompilation.
Cross language interoperability => Functions using C-linkage can be called from any high level programming language with foreign function interface such as Python with ctypes library. C++ functions and classes cannot be called due to name mangling and compiler ABI issues. The workaround to this problem is to create a C-interface or functions with C-linkage for classes, functions and namespaces. Instances of a class can be passed around as opaque pointers (void*).

Problems:

Due to C++ lack of standard ABI Application Binary Interface, classes, STL containers, functions without C-linkage and namespaces may not be reusable with a client program compiled with different compiler than used to build the library. However, even in this case, there is still the benefit of faster compilation time if the shared library and the client application are part of the same project.
- TL;DR => Classes, STL and functions without C-linkage of shared library cannot be used with a different compiler than the one used to build the DLL.
Cross-compiling interoperability - The only way to ensure that a shared library can work with all compilers is by using a functions with C-linkage (functions with extern “C”) annotation and creating C-linkage functions wrappers to classes, STL containers and classes.
Classes can be made compatible among different compilers by using an interface class, a class with only pure virtual member functions, using only compatible types in the methods signature or declaration, therefore it cannot use STL container types such as std::string or std::vector. However, each implementation of an interface class can use STL containers internally.

MSDN Documentation

Exporting C++ Functions for Use in C-Language Executables
Exporting C Functions for Use in C or C++ Language Executables

Further Reading

Creating the DLL functions
HowTo: Export C++ classes from a DLL - CodeProject
Creating and using shared libraries with different compilers on different operating systems - Gernot.Klingler
The Big Bang on Reading Busy » Blog Archive » Using Python ctypes to link C/C++ library

Design Guidelines for C-Interfaces

Macro for C-linkage and exporting symbols

Instead of:

// Does not work on Unix-like OSes. 
// Only works on compilers with supporting this MSVC (Visual VC++) compiler-extension. 
extern "C" __declspec(dllexport)  
double SomeFunctionWithCLinkage(const char* name) {
  ... function code ... 
}

Better export the symbol using a macro for ensuring cross-platform portability:

Note: Names of macros should be unique in order to avoid name collisions. This is why the macros are prefixed with library name.
Note: __declspec(dllexport) => MSVC (Visual C++) compiler extension only supported on MSVC and Mingw. (No in C++ ISO standard)
Note: __attribute__ ((visibility ("default"))) => GCC/Clang compiler extension used on Unix-like Oses for making symbols hidden by default.

#if defined(__cplusplus)
  // Code being compiled with C++ compiler 
  #define MATHTOOLS_EXTERN_C extern "C"
#else 
  // Code being compiled with C compiler (Not C++ compiler)
  #define MATHTOOLS_EXTERN_C 
#endif 

#if defined(_WIN32)
  // MS Windows DLLs (*.dll)
  #define MATHTOOLS_EXPORT_C MATHTOOLS_EXTERN_C __declspec(dllexport)
#else 
  // Unix-like Shared Object (.so) operating systems and GCC.
  #define MATHTOOLS_EXPORT_C MATHTOOLS_EXTERN_C __attribute__ ((visibility ("default")))
#endif

So, the function code becomes:

MATHTOOLS_EXPORT_C
double SomeFunctionWithCLinkage(const char* name) 
{
  ... function code ... 
}

Name Prefix

Functions with C-linkage are always in the global namespace, therefore every function needs an unique name in order to avoid name clashing which can result in undefined behavior. A good practice to avoid those name collisions is to use a common prefix before in the name of every function, for instance:

Instead of:

MATHTOOLS_EXPORT_C const char* GetVersion();
MATHTOOLS_EXPORT_C double RootSolver(double x0, double (* function) double);

// Or in multi-line style:
MATHTOOLS_EXPORT_C 
const char* GetVersion();

MATHTOOLS_EXPORT_C 
double RootSolver(double x0, double (* function) double);

Better:

// All functions contains the library name as a prefix in order to avoid 
// name clashes.
MATHTOOLS_EXPORT_C const char* mathtools_GetVersion();

MATHTOOLS_EXPORT_C 
double mathtools_RootSolver(double x0, double (* function) double);

Naming Schema for C-wrapers or C-interfaces for C++ classes

Sample class:

class Plotter{
  struct impl; 
  std::unique_ptr<impl> m_pimpl;    
public: 
  Plotter();
  Plotter(double width, double height);
  Plotter(Plotter const& rhs) = default;
 ~Plotter() = default;

 void addPoint(double x, double y);
 void clear();
};

C-wrappers schema:

C-interface for default constructor

MATHTOOLS_EXPORT_C
Plotter* mathtools_Plotter_new() 
{
   return new (std::nothrow) Plotter();
}

C-interface or C-Wrapepr for second constructor:

MATHTOOLS_EXPORT_C
Plotter* mathtools_Plotter_new2(double width, double height) 
{
   return new (std::nothrow) Plotter(width, height);
}

C-interface for copy constructor:

// Wrapper for copy constructor 
MATHTOOLS_EXPORT_C
Plotter* mathtools_Plotter_new_copy(Plotter* hPlotter) 
{
   return new (std::nothrow) Plotter(*hPlotter);
}

C-interface for destructor:

MATHTOOLS_EXPORT_C
void mathtools_Plotter_delete(Plotter* hPlotter) 
{
  delete hPlotter;
}

C-interface for class method.
- As C-interfaces cannot throw exceptions, the last parameter is used for returning an error code.

// Schema: <Library Name>_<Class Name>_<method>
MATHTOOLS_EXPORT_C
void mathtools_Plotter_addPoint(Plotter* hPlotter, double x, double y)
{
   hPlotter->addPoint(x, y);
}

Exceptions and Error Handling

DO NOT: Throw exceptions in functions with C-linkage (extern “C”) as exceptions are incompatibles among different compilers. Moreover, the C-language cannot handle exceptions.
DO: Instead of exceptions, return an error code enumeration as a function return value or as function parameter.

Example: If a function throws exceptions such as the function mathtools_Plotter_addPoint, the exceptions should be caught and the function should return an error code eihter as return value or function parameter.

MATHTOOLS_EXPORT_C
void mathtools_Plotter_addPoint(Plotter* hPlotter, double x, double y, int* ErrorCode)
{
    *ErrorCode = ErrorEnum::OK;
    try { 
       // Method addPoint throws exceptions 
       hPlotter->addPoint(x, y);
    } catch(Exception1 const& ex){
       *ErrorCode = ErrorEnum::NO_SPACE_ERROR; // Set with value of some enum 
    } catch(Exception2 const& ex){
       *ErrorCode = ErrorEnum::SOME_ERROR_ENUM;
   }
}

Passing std::string STL Object across a DLL boundary

Type synonym:

// h prefix stands for handle 
using hString = std::string*;

C-interface for std::string default constructor.

LIBRARY_EXPORT_C  
hString libprefix_string_new()
{
   // std::nothrow => Do not throw std::bad_alloc, just return 
   // a null pointer on failure.
   return new (std::nothrow) std::string();   
}

C-interface for std::string copy-constructor.

LIBRARY_EXPORT_C  
hString libprefix_string_new_copy(hString hstr)
{
   // std::nothrow => Do not throw std::bad_alloc, just return 
   // a null pointer on failure.
   return new (std::nothrow) std::string(*hstr);   
}

C-interface for std::string overloaded constructor.

LIBRARY_EXPORT_C  
hString libprefix_string_new2(const char* text)
{
   return new (std::nothrow) std::string(text);   
}

C-interface for std::string destructor.

// Style 1: 
LIBRARY_EXPORT_C 
hString libprefix_string_delete(hString hstr)
{
   delete hstr;
}

// Style 2: 
LIBRARY_EXPORT_C hString 
libprefix_string_delete(hString hstr)
{
   delete hstr;
}

// Style 3: 
LIBRARY_EXPORT_C 
hString 
libprefix_string_delete(hString hstr)
{
   delete hstr;
}

// Style 4: Trailing return-type.
LIBRARY_EXPORT_C 
auto libprefix_string_delete(hString hstr) -> hString 
{
   delete hstr;
}

C-interface for std::string getter, returns const char*.

LIBRARY_EXPORT_C 
const char*
libprefix_string_get(hString hstr)
{
   return hstr->data();
}

C-interface for std::string setter.

LIBRARY_EXPORT_C 
void 
libprefix_string_set(hString hstr, const char* text)
{
   *hstr = text; 
}

C-interface for std::string concat.

LIBRARY_EXPORT_C 
void 
libprefix_string_conat(hString hstr, const char* text)
{
   *hstr = *hstr + text; 
}

C-function to print std::string object

LIBRARY_EXPORT_C 
void 
libprefix_string_print(hString hstr)
{
   std::cout << *hstr << "\n";
}

C-wrapper to method std::string::clear()

LIBRARY_EXPORT_C 
void 
libprefix_string_clear(hString hstr)
{
   hstr->clear();
}

Passing std::vector<double> across a DLL boundary

Handle type:

using hVectorD = std::vector<double>*;

Constructors:

LIBRARY_EXPORT_C 
hVectorD libprefix_vectorD_new()
{
  return new (std::nothrow) std::vector<double>();
}

LIBRARY_EXPORT_C 
hVectorD libprefix_vectorD_new2(size_t n, double x)
{
  return new (std::nothrow) std::vector<double>(n, x);
}

LIBRARY_EXPORT_C 
hVectorD libprefix_vectorD_new3(size_t n, double [] array)
{
  // Iterator-range constructor 
  return new (std::nothrow) std::vector<double>(array, array + n);
}

Destructor:

LIBRARY_EXPORT_C 
void libprefix_vectorD_delete(hVectorD hvec)
{
  delete hvec;
}

Get size:

LIBRARY_EXPORT_C 
size_t libprefix_vectorD_size(hVectorD hvec)
{
  hvec->size();
}

Getter:

LIBRARY_EXPORT_C 
double libprefix_vectorD_get(hVectorD hvec, size_t n)
{
  return hvec->operator[](n);
}

Setter:

LIBRARY_EXPORT_C 
void libprefix_vectorD_set(hVectorD hvec, size_t n, double x)
{
  hvec->operator[](n) = x;
}

Element-wise fundamental math function ‘sin()’:

// Returns a new vector Out[i],  Out[i] = sin(Input[i]), i = 0, 1, 2... Input.size() 
LIBRARY_EXPORT_C 
hVector libprefix_vectorD_sin(hVectorD hvec)
{
   auto result = new (std::nothrow) std::vector<double>(hvec->size());
   for(size_t i = 0; i <= hvec->size(); i++)
     (*result)[i] = std::sin((*hvec)[i]);
   return result;
}

// Returns a new vector Out[i],  Out[i] = exp(Input[i]), i = 0, 1, 2... Input.size() 
LIBRARY_EXPORT_C 
hVector libprefix_vectorD_exp(hVectorD hvec)
{
   auto result = new (std::nothrow) std::vector<double> (hvec->size());
   for(size_t i = 0; i <= hvec->size(); i++)
     (*result)[i] = std::exp((*hvec)[i]);
   return result;
}

Higher order C-function to apply a function to each element:

// Returns a new vector Out[i],  Out[i] = exp(Input[i]), i = 0, 1, 2... Input.size() 
LIBRARY_EXPORT_C 
hVector 
libprefix_vectorD_map( 
             hVectorD hvec              /* Vector handle */
            ,double Function (double))  /* Function pointer to math function */                                      
{
   auto result = new (std::nothrow) std::vector<double>(hvec->size());
   for(size_t i = 0; i <= hvec->size(); i++)
     (*result)[i] = Function((*hvec)[i]);
   return result;
}

See also:

c++ - How do I safely pass objects, especially STL objects, to and from a DLL? - Stack Overflow

Cross platform Shared Library, C-Interfaces and Language Interoperability

Overview

Source:

Shared library Source Code

File: src/dlls/example-windows1/testlib.cpp
File: src/dlls/example-windows1/testlib.hpp

Sample Client C++ Program:

File: src/dlls/example-windows1/client1.cpp

Sample Python 3 Client code: (Python3 module wrapper library)

File: src/dlls/example-windows1/testlib.py

GIST - Better for online view:

Gisthub

Compile the DLL libtest.dll and client program with MSVC

Shared Library

$ cl.exe testlib.cpp /EHsc /LD /nologo user32.lib

C++ Client program (client code)

$ cl.exe /EHsc client1.cpp /Fe:client1.exe testlib.lib && out.exe

Compile the DLL libtest.dll and client program with Mingw/GCC

Shared Library

$ g++ testlib.cpp -o testlib.dll -g -fvisibility=hidden -shared -std=c++14 -Wall

C++ Client program

$ g++ client.cpp -o client.exe -g -std=c++14 testlib.dll -Wall

Compile and run from GIST

Clone the GIST (Gisthub).

# Clone GIST: 
$ git clone https://gist.github.com/caiorss/2bba4c50866d9467aaa8c7792b337f71 dllex1
Cloning into 'dllex1'...
remote: Enumerating objects: 66, done.
remote: Counting objects: 100% (66/66), done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 66 (delta 36), reused 45 (delta 19), pack-reused 0
Unpacking objects: 100% (66/66), done.

# Enter directory containing sources
$ cd dllex1/

$ tree .
.
├── build.bat
├── client1.cpp
├── CMakeLists.txt
├── Makefile
├── pywrapper.py
├── testlib.cpp
└── testlib.hpp

0 directories, 7 files

Build shared library on Linux.

For Linux, build DLL and client code with make linux.

$ make linux
echo "Build shared library"
Build shared library
g++ testlib.cpp -o libtestlib.so  -std=c++14 -fPIC -shared -Wall
echo "Build client code 1"
Build client code 1
g++ client1.cpp -o client1.bin -std=c++14 libtestlib.so 

$ file libtestlib.so 
libtestlib.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), 
dynamically linked, BuildID[sha1]=52a16d16991d7aa71763140e97a1422c1387711d, not stripped

$ file client1.bin 
client1.bin: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, 
interpreter /lib64/ld-linux-x86-64.so.2, 
for GNU/Linux 3.2.0, BuildID[sha1]=1a4885e112f72372cfd66816a564f49a24110c50, not stripped

Build shared library on Windows:

Just run the batch script build-windows.bat or double click at it.

$ build-windows.bat
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.5.6
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
testlib.cpp
   Creating library testlib.lib and object testlib.exp
Microsoft (R) C/C++ Optimizing Compiler Version 19.12.25835 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

client1.cpp
Microsoft (R) Incremental Linker Version 14.12.25835.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:client1.exe
client1.obj
testlib.lib
   Creating library client1.lib and object client1.exp
 [StaticObject] => Initialize DLL
testlib.cpp:48: <DllMain> DLL Loaded into the process => PID = 1240
testlib.cpp:54: <DllMain> DLL attached to process.

=== EXPERIMENT 1 ===> Import C++ functions from DLL
client1.cpp:38: <main> Main process starts here.
 => Linalg::norm(xs) 7.4162
=> xs = [5]( 1, 2, 3, 4, 5,  )
=== EXPERIMENT 2 ===> Import class from DLL
 Instance created with name = Dummy

 .. ... ...  .. ... ...  .. ... ...  .. ... ...

Run C++ client code:

$ ./client1.bin 
 [StaticObject] => Initialize DLL

=== EXPERIMENT 1 ===> Import C++ functions from DLL
client1.cpp:38: <main> Main process starts here.
 => Linalg::norm(xs) 7.4162
=> xs = [5]( 1, 2, 3, 4, 5,  )
=== EXPERIMENT 2 ===> Import class from DLL
 Instance created with name = Dummy
 Counter set to value = 100
cls.getName() = Dummy
    cls.get() = 100

... ... ...

Run Python client code:

$ make py
python3 pywrapper.py
 [INFO] libpath = libtestlib.so
 [StaticObject] => Initialize DLL
Intializing library
Library initialized OK.

 .. ... ... ...

Header macros

Sources:

File: src/dlls/example-windows1/testlib.cpp
File: src/dlls/example-windows1/testlib.hpp

teslib.hpp / Heade File - Macros

The header files uses the following macros to reduce the boilerplate necessary to export the library functions and classes.

file: testlib.hpp => Macros for exporting functions and classes.

Macro: EXPORT_CPP

/** Macro EXPORT_CPP makes a symbol visible. */
#if defined(_WIN32)
  #define EXPORT_CPP __declspec(dllexport)
#else
  #define EXPORT_CPP __attribute__ ((visibility ("default")))
  // If not compiled on Windows, remove declspec compiler extension.
  #ifndef __declspec
    // For GCC only - make exported symbol visible symbol 
    #define __declspec(param) __attribute__ ((visibility ("default")))
  #endif
#endif

The macro EXPORT_CPP annotates the symbol as visible by expanding to:

// On Windows EXPORT_CPP becomes 
__declspec(dllexport)

// On Unix-like OSes, Linux, BSD, OSX ... 
 __attribute__ ((visibility ("default")))

So a function int SomeFunction(double x, double y) becomes:

 // Makes this symbol visible (exported) in the library (DLL)
 EXPORT_CPP int SomeFunction(double , double);

// On Windows: 
__declspec(dllexport) int SomeFunction(double , double);

// On Unix-like operating systems or Linux: 
__attribute__ ((visibility ("default"))) int SomeFunction(double , double);

Macro EXPORT_C:

/* Macro EXPORT_C is used for exporting symbol with C-linkage, it
 * means, without name mangling */
#ifdef __cplusplus
  // extern "C" - Indicates that a given symbol/function has C-linkage and
  // does not have name mangling.
  // 
  // On Linux EXPORT_C becomes
  //  => extern "C" __attribute__ ((visibility ("default")))
  // 
  // On Windows EXPORT_C becomes 
  //  => extern "C" __declspec(dllexport)
  #define EXPORT_C extern "C" EXPORT_CPP
#else
  // If a C-compiler uses this header, remove 'extern "C"'
  #define EXPORT_C  EXPORT_CPP
#endif

Export symbols with C-linkage, it means that the function is exported without name decoration or name mangling. The function symbol matches exactly its name. Functions with C-linkage can be called by C-code; C++ code compiled with other compilers without ABI issues and foreign function interfaces of scripting languages such as Python Ctypes or C# Pinvoke. Note: If a function is defined with C-linkage, it can only use C-compatible types. So, it is no possible to define functions with C-linkage with overload, default parameter or non C-compatible arguments.

On Windows, this macro is expanded to:

EXPORT_C int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

// On Windows it becomes: 
extern "C" __declspec(dllexport)  int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

// This function could also be declared as:
extern "C" __declspec(dllexport) 
int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

On Unix-like OSes or linux, the macro EXPORT_C expands to:

EXPORT_C int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

// On Linux, it becomes:
extern "C" __attribute__ ((visibility ("default"))) int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

// This function could also be declared as:
extern "C" __attribute__ ((visibility ("default"))) 
int functionWith_C_linkage(MyClass* pointerToClass, int x, double z);

On Unix-like operating system (GCC or Clang compilers), all functions in an object-code (aka compiled code) becomes visible by default what can lead to undefined behavior when there are name clashes due to repeated symbols. With appropriate compiler options such as -fvisibility=hidden, the symbols can be made private by default as happens in Windows where is necessary to annotate the symbol as visible explicitly with __declspec(dllexport). On Unix-like OSes for GCC and Clang compiler the annotation for making the symbol visible is __attribute__ ((visibility ("default"))). This statement has no effect if the appropriate compiler flags are not set.

Namespace Linalg

Functions of Namespace Linalg

Namespace containing sample linear algebra functions, the function norm computes an Euclidean norm of a N-dimension vector, the function linTransform computes a linear transformation performing the computation v[i] * a + b for each vector element.

On Windows, all DLL symbols, such as functions, variables and clases, are private by default, they are not exported by default as happens in Unix-like OSes shared libraries. Therefore, in order to a function be exported in a Windows shared library, it is necessary to use the MSVC or visual C++ compiler extension __declspec(dllexport) before the function declaration.

Interface => File: testlib.hpp

#ifdef __cplusplus

namespace Linalg {
        EXPORT_CPP
        double norm(const std::vector<double>& xs);

        EXPORT_CPP std::vector<double>
        linTransform(
                double a,
                double b,
                std::vector<double>& xs
                );

        EXPORT_CPP
        std::ostream&
        printVector(std::ostream& os, std::vector<double>& xs);	
}
#endif

The compiler directives #ifdef __cplusplus … #endif are used for disabling this code block when the header is used by a C-compiler as the C-language does not support classes, namespaces and many other C++ features.

file: testlib.cpp - Implementation

// Linear algebra tools 
 namespace Linalg{
         EXPORT_CPP 
          double norm(const std::vector<double>& xs){
                double sum = 0.0;
                for(const auto& x : xs) sum += x * x;
                return std::sqrt(sum);
        }

        EXPORT_CPP
        std::vector<double>
        linTransform(double a, double b, std::vector<double>& xs){
                std::vector<double> out(xs.size());
                for(size_t i = 0; i < xs.size(); i++){		
                        out[i] = a * xs[i] + b;
                }
                return out;
        }

        EXPORT_CPP
        std::ostream&
        printVector(std::ostream& os, std::vector<double>& xs){
                os << "[" << xs.size() << "]( ";
                for(const auto&  x: xs)
                        os << x << ", ";
                return os << " )";
        }	
}

C-interface of Namespace Linalg

C-interface of Namespace Linalg

A C-interface is necessary for ensuring that the DLL can be called from a C-program or a foreign function interface of a higher level language.

File: testlib.hpp => Function declaration.

// ======= C-interface for Linalg namespace =========//

/** Handle or opaque pointer for std::vector<double> */
typedef void* hVectorD;

/* ----- C-Wrappers for Linalg namespace ---- */

EXPORT_C 
double testlib_vectorD_Linalg_norm(hVectorD hv);

EXPORT_C 
void testlib_vectorD_Linalg_printVector(const char* name, hVectorD hv);

File: testlib.cpp => C-interface function implementations.

The type synonym hVectorD is handle or a opaque poiter for the type std::vector<double>. It is a workaround, to pass std::vector<double> accross a DLL functions with C-linkage and use std::vector<double> from C programs or foreign function interfaces.

//=========== C-wrappers ---------------///
// Handler for double vector 
using hVectorD = void*;
using pVectorD = std::vector<double>*;

/** C-wrapper for vector<double> constructor */
EXPORT_C  
hVectorD testlib_vectorD_make0(size_t n, double x){
    return new std::vector<double>(n, x);
}

/** C-wrapper for range constructor */
EXPORT_C
hVectorD testlib_vectorD_make1(size_t n, double array []){
    return new std::vector<double>(array, array + n);
}

/** C-wrapper for setting elements of vector<double> */
EXPORT_C 
void testlib_vectorD_set(hVectorD hv, size_t n, double x){
   reinterpret_cast<pVectorD>(hv)->operator[](n) = x;
}

/** C-wrapper for vector<double> destructor */
EXPORT_C void testlib_vectorD_delete(hVectorD hv){
    delete reinterpret_cast<pVectorD>(hv);
}

/** C-wrapepr for Linalg::norm function */
EXPORT_C 
double testlib_vectorD_Linalg_norm(hVectorD hv){
    return Linalg::norm(*reinterpret_cast<pVectorD>(hv));
}

EXPORT_C 
void testlib_vectorD_Linalg_printVector(const char* name, hVectorD hv){
    std::cout << name << " = ";
    Linalg::printVector(std::cout, *reinterpret_cast<pVectorD>(hv));
    std::cout << std::endl;
}

Non-polymorphic Class - SampleClass

File: testlib.hpp => Class declaration.

In order to a class be exported in a Windows’ shared library, it requires the annotation __declspec(dllexport).

Note: It will not be possible to compiled client program using this class with a compiler different from the used to build the shared library libtest.dll. It happens because, the C++ ABI - Application Binary Interface is not the same accross different compilers. If the DLL is compiled with MSVC, the only way to use this class from a program compiled using MingW or even a different version of MSVC is by using a C-interface (functions with C-linkage) and opaque pointers (void* pointers).

// ======= Non-polymorphic class exported by DLL =========//

#ifdef __cplusplus
// Non-polymorphic class 
class EXPORT_CPP SampleClass{
public:
        SampleClass();
        SampleClass(const std::string& name);	
        ~SampleClass();
        std::string getName() const;
        int get();
        void set(int n);	
private:
        std::string m_name;
        int m_counter;	
};
#endif

File: testlib.cpp => Class implementation.

SampleClass::SampleClass(const std::string& name)
   : m_name(name), m_counter(0)
{
    std::cout << " Instance created with name = " << m_name << std::endl;
}
/** Delegated constructor on right-hand-side */
SampleClass::SampleClass(): SampleClass("unnamed"){}   
	
SampleClass::~SampleClass(){
    std::string text = std::string("SampleClass => name = ") + m_name +  " deleted";
    DbgTrace(text);
}

std::string SampleClass::getName() const {
    return m_name;
}
int SampleClass::SampleClass::get(){		
    return m_counter;
}
void SampleClass::set(int n){
    std::cout << " Counter set to value = " << n << std::endl;
    m_counter = n;
}

C-interface for SampleClass

File: testlib.hpp => Functions declarations.

/* ----- C-Wrappers for SampleClass namespace ---- */
using hSampleClass = void*;

/** Nullable constructor zero-arg constructor */
extern "C" __declspec(dllexport)
hSampleClass
testlib_SampleClass_make0();

/** Other constructor */
EXPORT_C hSampleClass testlib_SampleClass_make1(const char* name);

/** Destructor */
EXPORT_C
void
testlib_SampleClass_delete(hSampleClass hnd);

/** Wrapper for get method */
EXPORT_C
int
testlib_SampleClass_get(hSampleClass hnd);

/** Wrapper for set method */
EXPORT_C
void
testlib_SampleClass_set(hSampleClass hnd, int n);

EXPORT_C
const char*
testlib_SampleClass_getName(hSampleClass hnd);

File: testlib.cpp => Functions implementation.

/* ----- C-Wrappers for SampleClass namespace ---- */

using hSampleClass = void*;

/** Nullable constructor zero-arg constructor */
EXPORT_C 
hSampleClass
testlib_SampleClass_make0();

/** Other constructor */
EXPORT_C hSampleClass testlib_SampleClass_make1(const char* name);

/** Destructor */
EXPORT_C
void
testlib_SampleClass_delete(hSampleClass hnd);

/** Wrapper for get method */
EXPORT_C
int
testlib_SampleClass_get(hSampleClass hnd);

/** Wrapper for set method */
EXPORT_C
void
testlib_SampleClass_set(hSampleClass hnd, int n);

EXPORT_C
const char*
testlib_SampleClass_getName(hSampleClass hnd);

Polymorphic class InterfaceClass

File: testlib.hpp => Class declaration.

As the name implies, this class is an interface class, which is a class containing only pure virtual functions (abstract methods). As a result, it is declared only in the header file and cannot be instatiated directly.

This class uses only C-types to be binary compatible with different compilers, if it used STL containers such as string or any other non-compatible C-type, it would not be possible to compile a C++ program using Mingw/GCC against this DLL built with MSVC.

// Polymorphic Interface class binary compatible across different
// compilers as it does not use any STL container on the interface.
#ifdef __cplusplus
struct InterfaceClass{
        /* Returns class unique ID */
        virtual const char* getID()	const = 0;
        /** Set class internal state */
        virtual void setName(const char* name) = 0;
        virtual const char* getName() = 0;

        /** Virtual constructor */
        virtual ~InterfaceClass() = default;
        // virtual ~InterfaceClass();
};
#else
  #define InterfaceClass void 
#endif

File: testlib.cpp => class ImplementationA of interface InterfaceClass.

class ImplementationA: public InterfaceClass
{
private:
    std::string m_name;
public:
    static constexpr const char* class_id = "ImplementationA";

    ImplementationA(): m_name("Unammed-A"){ }
    ImplementationA(const std::string& name)
       : m_name(name){}	
    ~ImplementationA(){
         std::cout << " [INFO] ImplementationA deleted => name = "
                   << m_name
                   << " ; type = " << class_id
                   << std::endl;
    }
    const char* getID() const {		
        return class_id;
    }
    void setName(const char* name) {
        m_name = name;
    }
    const char* getName() {
        return m_name.c_str();
    }
};

File: testlib.cpp => class ImplementationB of interface InterfaceClass.

class ImplementationB: public InterfaceClass
{
private:
    std::string m_name;
public:
    static constexpr const char* class_id = "ImplementationB";

    ImplementationB(): m_name("Unammed-B"){ }
    ImplementationB(const std::string& name)
       : m_name(name){}	
    ~ImplementationB(){
        std::cout << " [INFO] ImplementationB deleted => name = "
                  << m_name
                  << " ; type = " << class_id
                  << std::endl;
    }
    const char* getID() const {		
            return class_id;
    }
    void setName(const char* name) {
            m_name = name;
    }
    const char* getName() {
            return m_name.c_str();
    }
};

File: testlib.hpp => C-interface and factory function to load implementations from the DLL.

/** Factory function */
EXPORT_C InterfaceClass* teslib_InterfaceClass_factory(const char* class_id);

/** C-wrapper for destructor */
EXPORT_C void testlib_InterfaceClass_delete(InterfaceClass* hinst);

/** C-wrapper for getID method */
EXPORT_C const char* testlib_InterfaceClass_getID(InterfaceClass* hinst);

EXPORT_C void testlib_InterfaceClass_setName(InterfaceClass* hinst, const char* name);

EXPORT_C const char* testlib_InterfaceClass_getName(InterfaceClass* hinst);

File: testlib.cpp => C-interfaces and factory functions definitions.

EXPORT_C InterfaceClass*
teslib_InterfaceClass_factory(const char* class_id)
{
    auto s = std::string(class_id);
    if(s == "ImplementationA")
            return new ImplementationA();
    if(s == "ImplementationB")
            return new ImplementationB();
    return nullptr;
}

EXPORT_C void testlib_InterfaceClass_delete(InterfaceClass* hinst)
{
    delete hinst;
}
EXPORT_C
const char* testlib_InterfaceClass_getID(InterfaceClass* hinst)
{
    return hinst->getID();
}
EXPORT_C
void testlib_InterfaceClass_setName(InterfaceClass* hinst, const char* name)
{
    hinst->setName(name);
}
EXPORT_C
const char* testlib_InterfaceClass_getName(InterfaceClass* hinst){
    return hinst->getName();
}

DLL startup function DLLMain

When a Windows shared library is loaded at compile-time or at runtime by some process, the function DLLMain is always invoked, it is similar to the function main() from an executable.

Note: logging statements with printf, std::cout, std::cerr will not printed if the DLL is loaded by some non-console program, a program compiled to the window subsystem. Therefore, the easiest way to log is to use the OutputDebugString function which output can viewed by the DebugView sysinternal tool.

File: testlib.cpp => DLL main function.

/** - DLL Entry point - main function of DLL which is executed when 
    the DLL is loaded by some process. 
 */
#if defined(_WIN32)
extern "C" __declspec(dllexport)
BOOL APIENTRY DllMain (HINSTANCE hInst, DWORD reason, LPVOID lpReserved)
{
    std::string text =
            std::string("DLL Loaded into the process => PID = ")
            + std::to_string(::GetCurrentProcessId());
    WindbgTrace(text);
    DbgTrace(text);

    switch (reason)
    {
    case DLL_PROCESS_ATTACH:
        WindbgTrace("DLL attached to process");
        DbgTrace("DLL attached to process.");			
      break;
    case DLL_PROCESS_DETACH:
         WindbgTrace("=> DLL detached.");
         DbgTrace("=> DLL attached");
      break;
    case DLL_THREAD_ATTACH:
        WindbgTrace("DLL attached to thread");
        DbgTrace("DLL detached to thread.");
      break;
    case DLL_THREAD_DETACH:
        WindbgTrace("DLL detached from thread");
        DbgTrace("DLL detached from thread.");
      break;
    }

  return TRUE;
}
#endif

Another more portable way to peform some way action when the DLL is loaded by a process or unloaded is to use a static object. The startup task is executed int the object’s constructor and the finalization task, which happens when the process ends, is executed at object’s destructor. This approach may a better replacement for DLLMain function, which is specific for Windows, as this technique works both on Windows and any other Unix-like operating system.

File: testlib.cpp => Static Object for replacing DLLMain

The class _StaticObject and a static instance of it is defined within an anonymous namespace to make them private to the compilation unit testlib.cpp and not allow them to be used from anywhere else.

// Class private to this compilation unit - cannot be accessed from
// any other file
namespace {
   class _StaticObject{		
   public:
       using Action = std::function<void ()>;
       Action m_end;

       _StaticObject(Action init, Action end)
               :  m_end(end)
       {
               init();
       }
       ~_StaticObject(){ m_end(); }
   };

   // Static object for replacing DLLMain 
   auto initDLL = _StaticObject(
      []{
            std::cout << " [StaticObject] => Initialize DLL"
                       << std::endl;						
      },
      []{	
           std::cout << " [StaticObject] => Shutdown DLL"
                     << std::endl;								
      });
}

DLL entry point for run32dll.exe

This function is a DLL entry point for the Windows program rundll32.exe which can execute this function with:

$ rundll32.exe testlib.dll,entryPoint1
$ rundll32.exe testlib.dll,entryPoint1 arg0 arg1 arg2 ... argn

When the DLL testlib.dll is run with run32dll calling the function entryPoint1, it will display a message box.

Note: It is an optional function which could be used for installing the DLL in the system or creating some entry in Windows’ registry.

File: testlib.cpp

#if defined(_WIN32)
extern "C" __declspec(dllexport)
void entryPoint1(HWND hwn, HINSTANCE hinst, LPSTR cmdLine, int nCmdShow){  
     DbgDisp(cmdLine);
     OutputDebugString("Rudll32 called entryPoint1()");
     MessageBoxA(NULL, "DLL ENTRY POINT", "Entry point 1", 0);
}
#endif

C++ Client program - client1.cpp

File: src/dlls/example-windows1/client1.cpp

File client1.cpp

Includes:

Note: (#include “testlib.hpp”) a header file between quotes, indicates to the compiler that the header is in the same directory as the source file client1.cpp.

#include <iostream>
#include <ostream>
#include <vector>
#include <string>

#include "testlib.hpp"

Declaration of functions with C-linkages not declarated in the header testlib.hpp

extern "C" hVectorD testlib_vectorD_make0(size_t n, double);
extern "C" hVectorD testlib_vectorD_make1(size_t n, double array []);
extern "C" void testlib_vectorD_delete(hVectorD hv);

Main function - Experiment 1 and Experiment 2:

#ifndef DISABLE
std::cout << "\n=== EXPERIMENT 1 ===> Import C++ functions from DLL" << std::endl;

DbgTrace("Main process starts here.");
std::vector<double> xs{1.0, 2.0, 3.0, 4.0, 5.0};
std::cout << " => Linalg::norm(xs) " << Linalg::norm(xs) << std::endl;
std::cout << "=> xs = "; Linalg::printVector(std::cout, xs); std::cout << std::endl;

std::cout << "=== EXPERIMENT 2 ===> Import class from DLL" << std::endl;
auto cls = SampleClass("Dummy");
cls.set(100);
std::cout << "cls.getName() = " << cls.getName() << std::endl;
std::cout << "    cls.get() = " << cls.get() << std::endl;
#endif // -- eof DISABLE flag

Main Function - Experiment 3

//=========>> Load functions and classes using C-interface ==============//

std::cout << "\n== EXPERIMENT 3 ===> Import C-functions from DLL - C-interface" << std::endl;

double arr [] = {1, 2, 3, 4, 5};
hVectorD v1 = testlib_vectorD_make1(5, arr);
testlib_vectorD_Linalg_printVector("v1", v1);
std::cout << "norm(v1) = " << testlib_vectorD_Linalg_norm(v1) << std::endl;

testlib_vectorD_delete(v1);

Main Functions - Experiment 4

std::cout << "\n== EXPERIMENT 4 ===> Non-polymorphic class with C-interface " << std::endl;

hSampleClass hcls = testlib_SampleClass_make1("[EXPERIMENT4]ClassHandle-OOP-C-API");
std::cout << "[EXPERIMENT 4] hcls.getName() = " << testlib_SampleClass_getName(hcls) << std::endl;
testlib_SampleClass_set(hcls, 100);
std::cout << "[EXPERIMENT 4] hcls.get()     = " << testlib_SampleClass_get(hcls) << std::endl;
testlib_SampleClass_set(hcls, 200);
std::cout << "[EXPERIMENT 4] hcls.get()     = " << testlib_SampleClass_get(hcls) << std::endl;

testlib_SampleClass_delete(hcls);

Main Functions - Experiment 5

std::cout << "\n== EXPERIMENT 5 ===> Load polymorphic classes from DLL " << std::endl;

InterfaceClass* hinstA = teslib_InterfaceClass_factory("ImplementationA");
InterfaceClass* hinstB = teslib_InterfaceClass_factory("ImplementationB");

std::cout << " => hinstA->getID() = " << hinstA->getID() << std::endl;
std::cout << " => hinstA->getID() = " << hinstB->getID() << std::endl;
hinstA->setName("ClassA-implA");
hinstB->setName("ClassB-implB");
std::cout << " => hinstA->getName() = " << hinstA->getID() << std::endl;
std::cout << " => hinstB->getName() = " << hinstB->getID() << std::endl;

// Note: If delete is used directly to delete hinstA and hinstB,
// a segmentatin fault will happen whenc compiling with Mingw/GCC
testlib_InterfaceClass_delete(hinstA);
testlib_InterfaceClass_delete(hinstB);

std::cout << " [INFO] After deleting instances" << std::endl;

DbgTrace("Program ended OK.");

Compile and run client code with MSVC

Note: Both the DLL and the client code are compiled with MSVC-2017 64 bits target.

$ cl.exe /EHsc client.cpp /nologo /Fe:client1.exe testlib.lib && client1.exe

# Output: 
client.cpp
   Creating library client1.lib and object client1.exp
 [StaticObject] => Initialize DLL
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 1112
testlib.cpp:45: <DllMain> DLL attached to process.

=== EXPERIMENT 1 ===> Import C++ functions from DLL
client.cpp:33: <main> Main process starts here.
 => Linalg::norm(xs) 7.4162
=> xs = [5]( 1, 2, 3, 4, 5,  )
=== EXPERIMENT 2 ===> Import class from DLL
 Instance created with name = Dummy
 Counter set to value = 100
cls.getName() = Dummy
    cls.get() = 100

== EXPERIMENT 3 ===> Import C-functions from DLL - C-interface
v1 = [5]( 1, 2, 3, 4, 5,  )
norm(v1) = 7.4162

== EXPERIMENT 4 ===> Non-polymorphic class with C-interface 
 Instance created with name = [EXPERIMENT4]ClassHandle-OOP-C-API
[EXPERIMENT 4] hcls.getName() = [EXPERIMENT4]ClassHandle-OOP-C-API
 Counter set to value = 100
[EXPERIMENT 4] hcls.get()     = 100
 Counter set to value = 200
[EXPERIMENT 4] hcls.get()     = 200
testlib.cpp:159: <SampleClass::~SampleClass> SampleClass => name = [EXPERIMENT4]ClassHandle-OOP-C-API deleted

== EXPERIMENT 5 ===> Load polymorphic classes from DLL 
 => hinstA->getID() = ImplementationA
 => hinstA->getID() = ImplementationB
 => hinstA->getName() = ImplementationA
 => hinstB->getName() = ImplementationB
 [INFO] ImplementationA deleted => name = ClassA-implA ; type = ImplementationA
 [INFO] ImplementationB deleted => name = ClassB-implB ; type = ImplementationB
 [INFO] After deleting instances
client.cpp:87: <main> Program ended OK.
testlib.cpp:159: <SampleClass::~SampleClass> SampleClass => name = Dummy deleted
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 1112
testlib.cpp:49: <DllMain> => DLL attached
 [StaticObject] => Shutdown DLL

[FAILURE] ABI - Issue - Compile with Mingw client1.cpp against testlib.dll built with MSVC

The client code is compiled with Mingw and the library was built with MSVC-2017 for 64 bits target.

Build and run client [FAILURE]:

$ g++ client1.cpp -g -o client1-gcc.exe -std=c++14 testlib.dll  && client1-gcc.exe
C:\Users\archbox\AppData\Local\Temp\cckNg9eZ.o: In function `main':
client1.cpp:35: undefined reference to `Linalg::norm(std::vector<double, std::allocator<double> > const&)'
client1.cpp:36: undefined reference to `Linalg::printVector(std::ostream&, std::vector<double, std::allocator<double> >&)'
client1.cpp:39: undefined reference to `SampleClass::SampleClass(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
client1.cpp:40: undefined reference to `SampleClass::set(int)'
client1.cpp:41: undefined reference to `SampleClass::getName[abi:cxx11]() const'
client1.cpp:42: undefined reference to `SampleClass::get()'
client1.cpp:39: undefined reference to `SampleClass::~SampleClass()'
client1.cpp:39: undefined reference to `SampleClass::~SampleClass()'
collect2.exe: error: ld returned 1 exit status
Compilation exited abnormally with code 1 at Sun Dec 16 17:41:47

The compilation fails because C++ does not have a standard and stable ABI - Application Binary Interface, as a result, it is not possible to use an object-code (here - any type of compiled code) built by a different compiler than the current one. The ABI comprises the name mangling schema or name decoration schema which is compiler-dependent, padding, class memory layout and so on. In addition to ABI issues, the STL - Standard Template Library implementations may not be compatible and also not be the same across different compilers.

The only way to ensure that a DLL can work with all possible compilers is to use a functions with C-linkage (extern “C”) with opaque pointers (void*) for passing around classes and wrapped STL containers. Classes can be used by different compilers, only if they are interface classes (classes with only pure virtual functions) containing only C-compatible types in the declaration.

Compile with Mingw client1.cpp against testlib.dll built with MSVC

The compilation works when compiling with the custom flag -DDISABLE. The directive #ifndef DISABLE … #endif removes all usages of the namespace Linalg and the class SampleClass. This flag makes main the function use only functions with C-linkage and the interface class InterfaceClass.

$ g++ client.cpp -g -o client-gcc.exe -std=c++14 testlib.dll -DDISABLE   && client-gcc.exe

   [StaticObject] => Initialize DLL
  testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 5724
  testlib.cpp:45: <DllMain> DLL attached to process.

  == EXPERIMENT 3 ===> Import C-functions from DLL - C-interface
  v1 = [5]( 1, 2, 3, 4, 5,  )
  norm(v1) = 7.4162

  == EXPERIMENT 4 ===> Non-polymorphic class with C-interface 
   Instance created with name = [EXPERIMENT4]ClassHandle-OOP-C-API
  [EXPERIMENT 4] hcls.getName() = [EXPERIMENT4]ClassHandle-OOP-C-API
   Counter set to value = 100
  [EXPERIMENT 4] hcls.get()     = 100
   Counter set to value = 200
  [EXPERIMENT 4] hcls.get()     = 200
  testlib.cpp:159: <SampleClass::~SampleClass> SampleClass => name = [EXPERIMENT4]ClassHandle-OOP-C-API deleted

  == EXPERIMENT 5 ===> Load polymorphic classes from DLL 
   => hinstA->getID() = ImplementationA
   => hinstA->getID() = ImplementationB
   => hinstA->getName() = ImplementationA
   => hinstB->getName() = ImplementationB
   [INFO] ImplementationA deleted => name = ClassA-implA ; type = ImplementationA
   [INFO] ImplementationB deleted => name = ClassB-implB ; type = ImplementationB
   [INFO] After deleting instances
  client.cpp:87: <main> Program ended OK.
  testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 5724
  testlib.cpp:49: <DllMain> => DLL attached
   [StaticObject] => Shutdown DLL

Load DLL in Python3 REPL

Functions with C-linkage can be called by Python using the ctypes library. The following code presents how it can be done:

Table for C types / Python ctypes conversion

Get a Python string from a const char* ptr or char* ptr, use ctypes.string_at(ptr).
Pass a python string to an API with const char* (ctypes.POINTER(ctypes.c_char)) as argument, use Function(“string-argument”.encode(‘utf-8’))

C-type	Python C-types	C-type
char	ctypes.c_char	char*	ctypes.POINTER(ctypes.c_char)
int	ctypes.c_int	int*	ctypes.POINTER(ctypes.c_int)
size_t	ctypes.c_int	-
double	ctypes.c_double	double*	ctypes.POINTER(ctypes.c_double)
void	None	void*	ctypes.c_void_p

Python C-types documentation and further reading:

ctypes — A foreign function library for Python — Python 3.7.2rc1 documentation
Extending Python With C Libraries and the “ctypes” Module – dbader.org
ctypes.CDLL Python Example
Calling C functions from Python – part 1 – using ctypes – Yi Zhang’s MSDN Blog
Python Programming/Extending with ctypes - Wikibooks, open books for an open world
Using C from Python: How to create a ctypes wrapper - Scientific IT-Systems

Load the library:

C:\Users\archbox\Desktop\experiments
λ "C:\Users\archbox\Miniconda3\pkgs\python-3.6.5-h0c2934d_0\python"
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> import ctypes

# This messages is print by the DLLMain function 
>>> lib = ctypes.cdll.LoadLibrary("testlib.dll")
 [StaticObject] => Initialize DLL
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 4280
testlib.cpp:45: <DllMain> DLL attached to process.
>>> testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 4280
testlib.cpp:53: <DllMain> DLL detached to thread.

Check whether exported functions with C-linkage exist:

>>> lib.testlib_vectorD_make0
<_FuncPtr object at 0x0000021BFCAFF388>

>>> lib.testlib_vectorD_Linalg_printVector
<_FuncPtr object at 0x0000021BFCAFF458>
>>>

>>> lib.testlib_vectorD_delete
<_FuncPtr object at 0x0000021BFCAFF528>

# Function which does not exist.
>>> lib.testlib_InterfaceClass_do_not_exist
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\archbox\Miniconda3\pkgs\python-3.6.5-h0c2934d_0\lib\ctypes\__init__.py", line 361, in __getattr__
    func = self.__getitem__(name)
  File "C:\Users\archbox\Miniconda3\pkgs\python-3.6.5-h0c2934d_0\lib\ctypes\__init__.py", line 366, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'testlib_InterfaceClass_do_not_exist' not found

Load std::vector<double> and Linalg namespace wrapper functions

Set up the std::vector<double> functions to be imported.

Note: hVectorD => Handle for std::vector<double> is the same as void* or an opaque pointer.

# Funciton: hVectorD testlib_vectorD_make0(size_t n, double x)
# Set function arguments
lib.testlib_vectorD_make0.argtypes = [ctypes.c_int, ctypes.c_double]
# Set return type 
lib.testlib_vectorD_make0.restype  = ctypes.c_void_p
 
# void testlib_vectorD_Linalg_printVector(const char* name, hVectorD hv)
lib.testlib_vectorD_Linalg_printVector.argtypes = [ctypes.POINTER(ctypes.c_char), ctypes.c_void_p ]
lib.testlib_vectorD_Linalg_printVector.restype = None 

# Set vector elements hv[n] = x
# void testlib_vectorD_set(hVectorD hv, size_t n, double x)
lib.testlib_vectorD_set.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_double]
lib.testlib_vectorD_set.restype  = None

Testing C-interface functions for std::vector<double>:

# Creating a std::vector<double> with 4 elements equal to 3.0 
>>> vec1 = lib.testlib_vectorD_make0(4, 3.0)

# This number is the memory address held by the opaque pointer,
# it cannot be used directly. 
>>> vec1
2319184200912
>>>
# ========== Modify vector elements ======== #
# Printarray2 the wrapped std::vector<double>
>>> lib.testlib_vectorD_Linalg_printVector("vec1".encode('utf-8'), vec1)
vec1 = [4]( 3, 3, 3, 3,  )
>>>

>>> lib.testlib_vectorD_set(vec1, 0, 5.0)
>>> lib.testlib_vectorD_set(vec1, 1, 10.5)
>>> lib.testlib_vectorD_set(vec1, 2, 4.78)

>>> lib.testlib_vectorD_Linalg_printVector("vec1".encode('utf-8'), vec1)
vec1 = [4]( 5, 10.5, 4.78, 3,  )

Compute vector Euclidian norm:

# Wrapper for function 
# double Linalg::norm(const std::vector<double>& xs)
# double testlib_vectorD_Linalg_norm(hVectorD hv)
lib.testlib_vectorD_Linalg_norm.argtypes = [ ctypes.c_void_p ]
lib.testlib_vectorD_Linalg_norm.restype  = ctypes.c_double

>>> lib.testlib_vectorD_Linalg_printVector("vec1".encode('utf-8'), vec1)
vec1 = [4]( 5, 10.5, 4.78, 3,  )

>>> lib.testlib_vectorD_Linalg_norm(vec1)
12.926654632966722

# Check if calculations are right 
>>> math.sqrt(5 * 5 + 10.5 * 10.5 + 4.78 * 4.78 + 3 * 3)
12.926654632966722
>>>

Delete vector vec1 using its destructor function:

# Function: void testlib_vectorD_delete(hVectorD hv)
lib.testlib_vectorD_delete.argtypes = [ ctypes.c_void_p ]
lib.testlib_vectorD_delete.restype = None

>>> lib.testlib_vectorD_delete(vec1)

Load wrapper function for interface class InterfaceClass

Load factory function:

void* teslib_InterfaceClass_factory(const char* class_id)

# InterfaceClass* teslib_InterfaceClass_factory(const char* class_id)
lib.teslib_InterfaceClass_factory.argtypes = [ ctypes.POINTER(ctypes.c_char) ]
lib.teslib_InterfaceClass_factory.restype  = ctypes.c_void_p

# const char* testlib_InterfaceClass_getID(InterfaceClass* hinst)
lib.testlib_InterfaceClass_getID.argtypes = [ ctypes.c_void_p ]
lib.testlib_InterfaceClass_getID.restype  = ctypes.POINTER(ctypes.c_char)

Creating class instances and testing member function C-wrappers:

>>> hinstA = lib.teslib_InterfaceClass_factory("ImplementationA".encode('utf-8'))
>>> hinstA
2319184196864
>>>

>>> hinstB = lib.teslib_InterfaceClass_factory("ImplementationB".encode('utf-8'))
>>> hinstB
2319184198016

>>> s1 = lib.testlib_InterfaceClass_getID(hinstA)
>>> s1
<ctypes.LP_c_char object at 0x0000021BFBBF14C8>
>>> ctypes.string_at(s1)
b'ImplementationA'
>>>

>>> s2 = lib.testlib_InterfaceClass_getID(hinstB)
>>> ctypes.string_at(s2)
b'ImplementationB'

>>> ctypes.string_at(lib.testlib_InterfaceClass_getID(hinstB))
b'ImplementationB'
>>>

Load more C-wrappers for member function of class InterfaceClass:

# const char* testlib_InterfaceClass_getName(InterfaceClass* hinst) 
lib.testlib_InterfaceClass_getName.argtypes = [ ctypes.c_void_p ]
lib.testlib_InterfaceClass_getName.restype  = ctypes.POINTER(ctypes.c_char)

# void testlib_InterfaceClass_setName(InterfaceClass* hinst, const char* name)
>>> lib.testlib_InterfaceClass_setName.argtypes = [ ctypes.c_void_p, ctypes.POINTER(ctypes.c_char) ]
>>> lib.testlib_InterfaceClass_setName.restype = None

>>> lib.testlib_InterfaceClass_getName(hinstA)
<ctypes.LP_c_char object at 0x0000021BFBBF1548>

>>> ctypes.string_at(lib.testlib_InterfaceClass_getName(hinstA))
b'Unammed-A'

>>> ctypes.string_at(lib.testlib_InterfaceClass_getName(hinstB))
b'Unammed-B'

>>> lib.testlib_InterfaceClass_setName(hinstA, "Instance-ClassA1".encode('utf-8'))
>>> ctypes.string_at(lib.testlib_InterfaceClass_getName(hinstA))
b'Instance-ClassA1'

>>> lib.testlib_InterfaceClass_setName(hinstB, "Instance-B1".encode('utf-8'))
>>> ctypes.string_at(lib.testlib_InterfaceClass_getName(hinstB))
b'Instance-B1'

Load C-functions wrappers for the InterfaceClass destructor and dispose both instances hinstA and hinstB:

# Destructor function:
# void testlib_InterfaceClass_delete(InterfaceClass* hinst)
>>> lib.testlib_InterfaceClass_delete.argtypes = [ ctypes.c_void_p ]
>>> lib.testlib_InterfaceClass_delete.restype  = None

# Dispose class hinstA, delete this object 
>>> lib.testlib_InterfaceClass_delete(hinstA)
 [INFO] ImplementationA deleted => name = Instance-ClassA1 ; type = ImplementationA

>>> lib.testlib_InterfaceClass_delete(hinstB)
 [INFO] ImplementationB deleted => name = Instance-B1 ; type = ImplementationB

Exit python3 REPL:

>>> exit()
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 4280
testlib.cpp:49: <DllMain> => DLL attached
 [StaticObject] => Shutdown DLL

Python 3 Client Code - Wrapper Module

All the boilerplate code required to load the shared library testlib.dll can be eliminated by crafting a python module file and wrapper classes.

Source:

File: src/dlls/example-windows1/pywrapper.py
Gist: Gisthub

Module Initialization:

File: pywrapper.py

The function _config initializes the wrapper module, loading the DLL - share library testlib.dll into the current Python3 interpreter process and setting up the C-functions (functions exported with C-linkage, without name mangling) exported by the library.

import ctypes

def _getSharedLibrary(libname):
    import sys
    import os
    libfile = libname 
    if sys.platform == "linux" or sys.platform == "linux2":
        libfile = "lib" + libname + ".so"
    elif sys.platform == "darwin":
        libfile =  libname + ".dylyb"
    elif  sys.platform == "win32":
        libfile = libname + ".dll"
    libpath = os.path.join(os.path.dirname(__file__), libfile)
    print(" [INFO] libpath = " + libpath)
    return libpath

# _lib = ctypes.cdll.LoadLibrary("testlib")
_lib = ctypes.cdll.LoadLibrary(_getSharedLibrary("testlib"))

# Startup ctypes FFI - Foreign Function Interface 
def _config():
    print("Intializing library")
    # ======= std::vector<double> and Linalg:: namespace ==========##
  
    # hVectorD testlib_vectorD_make0(size_t n, double x)
    _lib.testlib_vectorD_make0.argtypes = [ctypes.c_int, ctypes.c_double]
    _lib.testlib_vectorD_make0.restype  = ctypes.c_void_p

    # hVectorD testlib_vectorD_make1(size_t n, double array [])
    _lib.testlib_vectorD_make1.argtypes = [ctypes.c_int, ctypes.POINTER(ctypes.c_double)]
    _lib.testlib_vectorD_make1.restype  = ctypes.c_void_p    

_config()

The class VectorD is a wrapper class for the STL class std::vector<double> and the namespace Linalg containing linear algebra functions.

class VectorD:
    def __init__(self, handle):
        self.hnd = ctypes.c_void_p(handle)
        self.name = "std::vector<double> vx"

    @classmethod
    def fromValue(cls, size, x):
        return VectorD(_lib.testlib_vectorD_make0(size, x))

    @classmethod
    def fromArray(cls, array):
        carray_size_n = ctypes.c_double * len(array)
        return VectorD(_lib.testlib_vectorD_make1(len(array), carray_size_n(*array)))

    # Destructor 
    def __del__(self):
        print(" [TRACE] - Vector disposed - C++ Destructor invoked Ok.")
        _lib.testlib_vectorD_delete(self.hnd)

    def setName(self, name):
        self.name = name

    # Display vector 
    def disp(self):
        _lib.testlib_vectorD_Linalg_printVector(self.name.encode('utf-8'), self.hnd)

    # Set element at nth position 
    def set(self, idx, x):
        _lib.testlib_vectorD_set(self.hnd, idx, x)

    def norm(self):
        return _lib.testlib_vectorD_Linalg_norm(self.hnd)

The class CPPInterfaceClass is a wrapper for the C++ interface class InterfaceClass exported by the DLL. The python wrapper class contains factory methods .factory, .makeA for loading the implementation class ImplementationA and the method .makeB for loading the implementation ImplementationB.

# Proxy for C++ Interface class in the shared library 
class CPPInterfaceClass:
    # Constructor 
    def __init__(self, handle):
        self.hnd = ctypes.c_void_p(handle)

    # Destructor 
    def __del__(self):
        # Call C++ destructor
        # print(" [__del__] => self.hnd = " + str(self.hnd))
        _lib.testlib_InterfaceClass_delete(self.hnd)

    @classmethod
    def factory(cls, classID):
        return CPPInterfaceClass(_lib.teslib_InterfaceClass_factory(classID.encode('utf-8')))

    @classmethod
    def makeA(cls):              
        "Instantiate the class ImplementationA from the DLL."
        return CPPInterfaceClass.factory("ImplementationA")

    @classmethod
    def makeB(cls):              
        "Instantiate the class ImplementationB from the DLL."
        return CPPInterfaceClass.factory("ImplementationB")

    def getType(self):
        return ctypes.string_at(_lib.testlib_InterfaceClass_getID(self.hnd)).decode('utf-8')

    def getName(self):
        return ctypes.string_at(_lib.testlib_InterfaceClass_getName(self.hnd)).decode('utf-8')

    def setName(self, name):
        _lib.testlib_InterfaceClass_setName(self.hnd, name.encode('utf-8'))

    # String representation 
    def __str__(self):
        s   = "CInterfaceClass ; type = " + self.getType()
        s +=  " - name =  " + self.getName() + "\n"
        return s

    # Make class printable in the REPL
    def __repr__(self):
        return self.__str__()

Function test1() runs an example code:

def test1():
    print("\n ========  Test 1 - std::vector<double> wrapper and Linalg module ======")
    v1 = VectorD.fromValue(4, 3.5)
    print(" [*]=> Before changing std::vector<double> object")
    v1.disp()
    print("v1.norm() = " + str(v1.norm()))
    print()
    print(" [*]=> After changing std::vector<double> object")
    v1.set(0, 5); v1.set(1, 2.6); v1.set(2, 9.81); v1.set(3, 3.76)
    v1.disp()
    print("v1.norm() " + str(v1.norm()))
    print()

    print("\n ========  Interface class 'InterfaceClass' ======")
    clsA = CPPInterfaceClass.makeA()
    print("clsA = " + str(clsA))
    print("clsA.getType() = " + clsA.getType())

    clsB = CPPInterfaceClass.makeB()
    print("clsB = " + str(clsB))
    print("clsB.getType() = " + clsB.getType())    

if __name__ == "__main__":    
    test1()

Usage in Python3 REPL:

This step assumes that the shared library file (testlib.dll on Windows or libtestlib.so on Unix-like OS or Linux) is in the same directory as the python script pywrapper.py.

Import module

$ C:/Users/archbox/Miniconda3/pkgs/python-3.6.5-h0c2934d_0/python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

>>> from testlib import VectorD
 [StaticObject] => Initialize DLL
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 5776
testlib.cpp:45: <DllMain> DLL attached to process.
Intializing library
Library initialized OK.

>>> from testlib import CPPInterfaceClass

Testing class VectorD - wrapper for std::vector<double>

>>> v1 = VectorD.fromValue(5, 3.0)
>>> v1.disp()
std::vector<double> vx = [5]( 3, 3, 3, 3, 3,  )

>>> v1.setName("v1")
>>> v1.disp()
v1 = [5]( 3, 3, 3, 3, 3,  )

>>> v1.norm()
6.708203932499369

>>> v1.set(0, 5)
>>> v1.set(1, 4.5)
>>> v1.set(2, 9.8)
>>> v1.set(3, 18.4)

>>> v1.disp()
v1 = [5]( 5, 4.5, 9.8, 18.4, 3,  )
>>>
>>> v1.norm()
22.10995251012539

>>> v1 = 100
 [TRACE] - Vector disposed - C++ Destructor invoked Ok.

>>> v2 = VectorD.fromArray([4.0, 10.25, 9.6, 3, 10, 6, 15])
>>> v2.disp()
std::vector<doubl> vx = [7]( 4, 10.25, 9.6, 3, 10, 6, 15,  )

>>> v2.norm()
24.15

>>> v2.set(0, 100)
>>> v2.set(1, 200)

>>> v2.disp()
std::vector<double> vx = [7]( 100, 200, 9.6, 3, 10, 6, 15,  )

>>> v2.norm()
224.63784186997523
>>>

Testing class CPPInterfaceClass, proxy for C++ classes InterfaceClass, ImplementationA and ImplementationB.

>>> from testlib import CPPInterfaceClass

# ======== Load ImplementationA  ============== # 
#
>>> a = CPPInterfaceClass.makeA()
>>> a
CInterfaceClass ; type = ImplementationA - name =  Unammed-A

>>> a.getType()
'ImplementationA'
>>> a.getName()
'Unammed-A'
>>> a.setName("Instance-of-ImplA")
>>> a
CInterfaceClass ; type = ImplementationA - name =  Instance-of-ImplA

# ======== Load ImplementationB  ============== # 
#
>>> b = CPPInterfaceClass.makeB()
>>> b
CInterfaceClass ; type = ImplementationB - name =  Unammed-B

>>> b.setName("Instance-of-class-B")
>>> b.getName()
'Instance-of-class-B'
>>>
>>> b.getType()
'ImplementationB'

Exit REPL.

>>> exit()
 [TRACE] - Vector disposed - C++ Destructor invoked Ok.
 [INFO] ImplementationB deleted => name = Instance-of-class-B ; type = ImplementationB
testlib.cpp:39: <DllMain> DLL Loaded into the process => PID = 5776
testlib.cpp:49: <DllMain> => DLL attached
 [StaticObject] => Shutdown DLL

CSharp Client Code - PInvoke

The services provided by the DLL can be consumed from C# or any other .NET language via Pinvoke-API which is the Foreign-Function Interface of .NET platform. The Pinvoke API can import C-function or C++ demangled functions exported with extern “C”.

C++ classes, namespaces and functions without C-linkage (extern “C”) cannot be loaded directly by P-Invoke or any other Foreign Function-Interface as C++ does not define a standard and ABI - Application Binary Interface, every compiler has its own name mangling schema, padding bytes and class memory layout for non-POD (Plain-Old data) types. A workaround to call C++ from an FFI is to defined C-interface comprised of functions with C-linkage and opaque pointers (void*) for passing class instances around or invoking C++ functions.

File: client-sharp.cs

Compiling and running:

$ csc.exe client-csharp.cs && client-csharp.exe

Wrapper for class std::vector<double> and namespace Linalg.

class VectorD{
        // Constructor Function:
        // hVectorD testlib_vectorD_make0(size_t n, double x)
        // void* testlib_vectorD_make0(size_t n, double x)
        [DllImport("testlib.dll")]
        public static extern
        IntPtr testlib_vectorD_make0(int size, double x);

        [DllImport("testlib.dll")]
        public static extern
        IntPtr testlib_vectorD_make1(int n, double[] x);

        // Destructor function:
        // void testlib_vectorD_delete(hVectorD hv)
        // void testlib_vectorD_delete(void* hv)
        [DllImport("testlib.dll")]
        public static extern
        IntPtr testlib_vectorD_delete(IntPtr hv);

        // void testlib_vectorD_Linalg_printVector(const char* name, hVectorD hv)
        // void testlib_vectorD_Linalg_printVector(const char* name, vector* hv)
        [DllImport("testlib.dll")]
        public static extern
        void testlib_vectorD_Linalg_printVector(string name, IntPtr hv);

        // double testlib_vectorD_Linalg_norm(hVectorD hv)
        // double testlib_vectorD_Linalg_norm(void* hv)
        [DllImport("testlib.dll")]
        public static extern
        double testlib_vectorD_Linalg_norm(IntPtr hv);

        // void testlib_vectorD_set(hVectorD hv, size_t n, double x)
        // void testlib_vectorD_set(void* hv, size_t n, double x)
        [DllImport("testlib.dll")]
        public static extern
        void testlib_vectorD_set(IntPtr hv, int n, double x);

        // ------------ Objet Oriented Wrapper --------//

        // Hadle 
        private IntPtr m_handle;

        public VectorD(int size, double x){
                m_handle = testlib_vectorD_make0(size, x);
        }
        public VectorD(double[] array){
                m_handle = testlib_vectorD_make1(array.Length, array);
        }        
        // Finalized destructor 
        ~VectorD(){
                testlib_vectorD_delete(m_handle);
        }
        public void print(string name = "std::vector<double>"){
                testlib_vectorD_Linalg_printVector(name, m_handle);
        }
        public void set(int n, double x){
                testlib_vectorD_set(m_handle, n, x);
        }
        public double norm(){
                return testlib_vectorD_Linalg_norm(m_handle);
        }
}

Wrapper for C++ interface class ‘InterfaceClass’ and its implementations, class ImplementationA and ImplementationB:

class CPPInterfaceClass{
        // Handle type => Opaque pointer for InterfaceClass instances.
        // using HInterf = IntPtr;

        // Factory function for loading functions from this interface.
        //-----------------------------------------------------------
        // InterfaceClass* teslib_InterfaceClass_factory(const char* class_id)
        // void** teslib_InterfaceClass_factory(const char* class_id)
        [DllImport("testlib.dll")]
        private static extern
        IntPtr teslib_InterfaceClass_factory(string class_id);

        // void testlib_InterfaceClass_delete(InterfaceClass* hinst)
        // void testlib_InterfaceClass_delete(void*)
        [DllImport("testlib.dll")]
        private static extern
        void testlib_InterfaceClass_delete(IntPtr hinst);

        // char* testlib_InterfaceClass_getID(InterfaceClass* hinst)
        [DllImport("testlib.dll")]        
        private static extern IntPtr testlib_InterfaceClass_getID(IntPtr hinst);

        // void testlib_InterfaceClass_setName(InterfaceClass* hinst, const char* name)
        // Note: Charset.UNICODE does not work as it is UTF16 (Wide Unicode), not UTF-8
        // which is the C++ default unicode type.
        [DllImport("testlib.dll", CharSet = CharSet.Ansi)]
        private static extern
        void testlib_InterfaceClass_setName(IntPtr hinst, string name);

        // char* testlib_InterfaceClass_getName(InterfaceClass* hinst)
        [DllImport("testlib.dll")]
        private static extern
        IntPtr testlib_InterfaceClass_getName(IntPtr hinst);

        private IntPtr m_handle;

        private CPPInterfaceClass(IntPtr handle){
                m_handle = handle;
        }

        ~CPPInterfaceClass(){
                testlib_InterfaceClass_delete(m_handle);
        }

        /** Creates an instance of the the C++ implementation ImplementationA (class) from the 
         * C++ interface: 'InterfaceClass' 
         */
        public static CPPInterfaceClass ImplementationA(){
                return new CPPInterfaceClass(teslib_InterfaceClass_factory("ImplementationA"));
        }
        /** Creates an instance of the C++ implementationa ImplementationB (class) */
        public static CPPInterfaceClass ImplementationB(){
                return new CPPInterfaceClass(teslib_InterfaceClass_factory("ImplementationB"));
        }

        // Get type of the wrapped C++ class 
        public string GetID(){
                IntPtr p = testlib_InterfaceClass_getID(m_handle);
                return Marshal.PtrToStringAnsi(p);
        }

        // Get/Set name of current instance 
        public string Name
        {
                get{
                        IntPtr p = testlib_InterfaceClass_getName(m_handle);
                        return Marshal.PtrToStringAnsi(p);
                }
                set{
                        testlib_InterfaceClass_setName(m_handle, value);
                }
        }
}

Main function - Experiment 1:

Console.WriteLine(" ===== EXPERIMENT 1 = Using C-functions directly ==");
Console.WriteLine(" [CSharp] Loading Native DLL C++ Shared Library");                
IntPtr vectorObject = VectorD.testlib_vectorD_make0(5, 3.0);
Console.WriteLine(" [CSharp] Opaque pointer = ", vectorObject.ToString());
VectorD.testlib_vectorD_Linalg_printVector("vectorX", vectorObject);


double x = VectorD.testlib_vectorD_Linalg_norm(vectorObject);
Console.WriteLine(" [CSharp] Vector norm = " + x.ToString());

Console.WriteLine(" [CSharp] End application");
VectorD.testlib_vectorD_delete(vectorObject);

Output:

===== EXPERIMENT 1 = Using C-functions directly ==
 [CSharp] Loading Native DLL C++ Shared Library
 [StaticObject] => Initialize DLL
testlib.cpp:48: <DllMain> DLL Loaded into the process => PID = 1320
testlib.cpp:54: <DllMain> DLL attached to process.
 [CSharp] Opaque pointer = 
vectorX = [5]( 3, 3, 3, 3, 3,  )
 [CSharp] Vector norm = 6.70820393249937
 [CSharp] End application

Main function - Experiment 2:

Console.WriteLine(" ===== EXPERIMENT 2 = Using OOP wrapper for std::vector ===");
Console.WriteLine(" ==> Before changing vector");
VectorD v1 = new VectorD(10, 3.5);
v1.print();
v1.print("vector_v1");
Console.WriteLine("v1.norm() = " + v1.norm().ToString());

Console.WriteLine(" ==> After changing vector");
v1.set(1, 10.0); v1.set(2, 5.53); v1.set(3, 8.96);
v1.set(4, -10.34); v1.set(8, 80.54);
v1.print("v1_changed");

Console.WriteLine(" ==> Creating Vector from Array");

VectorD v2 = new VectorD(new double[] {4.5, -8.84, 78.23, 652.3, 34.56, 45.12});
v2.print("v2");
Console.WriteLine("v2.norm() = " + v2.norm().ToString());

Output:

 ===== EXPERIMENT 2 = Using OOP wrapper for std::vector ===
 ==> Before changing vector
std::vector<double> = [10]( 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5,  )
vector_v1 = [10]( 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5,  )
v1.norm() = 11.0679718105893
 ==> After changing vector
v1_changed = [10]( 3.5, 10, 5.53, 8.96, -10.34, 3.5, 3.5, 3.5, 80.54, 3.5,  )
 ==> Creating Vector from Array
v2 = [6]( 4.5, -8.84, 78.23, 652.3, 34.56, 45.12,  )
v2.norm() = 659.502711518307

Main function - Experiment 3:

Console.WriteLine(" ===== EXPERIMENT 3 = OOP wrapper for InterfaceClass  ===");
CPPInterfaceClass instA = CPPInterfaceClass.ImplementationA();
CPPInterfaceClass instB = CPPInterfaceClass.ImplementationB();

// Console.WriteLine("instA.Name = " + instA.Name);
Console.WriteLine("instA.GetID() = " + instA.GetID());
Console.WriteLine("instB.GetID() = " + instB.GetID());

Console.WriteLine(" **=> Before changing");
Console.WriteLine("instA.Name = " + instA.Name);
Console.WriteLine("instB.Name = " + instB.Name);
Console.WriteLine("\n **=> Before changing");
instA.Name = "Instance-of-ImplA";
instB.Name = "Instance-of-ImplB";
Console.WriteLine("instA.Name = " + instA.Name);
Console.WriteLine("instB.Name = " + instB.Name);

Console.WriteLine(" ===== END  ===========================");

Output:

 ===== EXPERIMENT 3 = OOP wrapper for InterfaceClass  ===
instA.GetID() = ImplementationA
instB.GetID() = ImplementationB
 **=> Before changing
instA.Name = Unammed-A
instB.Name = Unammed-B

 **=> Before changing
instA.Name = Instance-of-ImplA
instB.Name = Instance-of-ImplB
 ===== END  ===========================
testlib.cpp:48: <DllMain> DLL Loaded into the process => PID = 1320
testlib.cpp:62: <DllMain> DLL detached to thread.
 [INFO] ImplementationB deleted => name = Instance-of-ImplB ; type = ImplementationB
 [INFO] ImplementationA deleted => name = Instance-of-ImplA ; type = ImplementationA
testlib.cpp:48: <DllMain> DLL Loaded into the process => PID = 1320
testlib.cpp:58: <DllMain> => DLL attached
 [StaticObject] => Shutdown DLL

View symbols exported by the shared library

View exported symbols on Windows (dumpbin)

Location of dumpbin tool

The symbols exported by the DLL can viewed with the dumpbin tool available from the MSVC developer console.

$ where dumpbin.exe
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\Hostx64\x64\dumpbin.exe

Show DLL exported symbols

$ dumpbin.exe /exports testlib.dll

Output:

Microsoft (R) COFF/PE Dumper Version 14.12.25835.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file testlib.dll

File Type: DLL

  Section contains the following exports for testlib.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
          31 number of functions
          31 number of names

    ordinal hint RVA      name

          1    0 00005C50 ??0SampleClass@@QEAA@AEBV0@@Z
          2    1 00001650 ??0SampleClass@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z
          3    2 000015E0 ??0SampleClass@@QEAA@XZ
          4    3 000016E0 ??1SampleClass@@QEAA@XZ
          5    4 00007280 ??4SampleClass@@QEAAAEAV0@AEBV0@@Z
          6    5 00001880 ?get@SampleClass@@QEAAHXZ
          7    6 00001830 ?getName@SampleClass@@QEBA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@XZ
          8    7 00001370 ?linTransform@Linalg@@YA?AV?$vector@NV?$allocator@N@std@@@std@@NNAEAV23@@Z
          9    8 000012D0 ?norm@Linalg@@YANAEBV?$vector@NV?$allocator@N@std@@@std@@@Z
         10    9 00001480 ?printVector@Linalg@@YAAEAV?$basic_ostream@DU?$char_traits@D@std@@@std@@AEAV23@AEAV?$vector@NV?$allocator@N@std@@@3@@Z
         11    A 000018A0 ?set@SampleClass@@QEAAXH@Z
         12    B 00001D50 DllMain
         13    C 00002720 entryPoint1
         14    D 00002800 main
         15    E 00001B10 teslib_InterfaceClass_factory
         16    F 00001C60 testlib_InterfaceClass_delete
         17   10 00001CC0 testlib_InterfaceClass_getID
         18   11 00001D20 testlib_InterfaceClass_getName
         19   12 00001CF0 testlib_InterfaceClass_setName
         20   13 00001A10 testlib_SampleClass_delete
         21   14 00001A60 testlib_SampleClass_get
         22   15 00001AB0 testlib_SampleClass_getName
         23   16 00001900 testlib_SampleClass_make0
         24   17 00001950 testlib_SampleClass_make1
         25   18 00001A80 testlib_SampleClass_set
         26   19 00001560 testlib_vectorD_Linalg_norm
         27   1A 00001580 testlib_vectorD_Linalg_printVector
         28   1B 000026D0 testlib_vectorD_delete
         29   1C 00002590 testlib_vectorD_make0
         30   1D 00002610 testlib_vectorD_make1
         31   1E 00002690 testlib_vectorD_set

  Summary

        3000 .data
        4000 .pdata
       18000 .rdata
        1000 .reloc
       2F000 .text

Note that:

Functions with C-linkage are not mangled, their symbols match their names, for instance:

 15    E 00001B10 teslib_InterfaceClass_factory
16    F 00001C60 testlib_InterfaceClass_delete
17   10 00001CC0 testlib_InterfaceClass_getID
18   11 00001D20 testlib_InterfaceClass_getName
19   12 00001CF0 testlib_InterfaceClass_setName
20   13 00001A10 testlib_SampleClass_delete
21   14 00001A60 testlib_SampleClass_get
22   15 00001AB0 testlib_SampleClass_getName

Functions, member functions, namespaces and everything without C-linkage are mangled, the compiler generate an unique name for the generated symbols:

 1    0 00005C50 ??0SampleClass@@QEAA@AEBV0@@Z
 2    1 00001650 ??0SampleClass@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z
 3    2 000015E0 ??0SampleClass@@QEAA@XZ
 4    3 000016E0 ??1SampleClass@@QEAA@XZ
 5    4 00007280 ??4SampleClass@@QEAAAEAV0@AEBV0@@Z
 6    5 00001880 ?get@SampleClass@@QEAAHXZ
 7    6 00001830 ?getName@SampleClass@@QEBA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@XZ
 8    7 00001370 ?linTransform@Linalg@@YA?AV?$vector@NV?$allocator@N@std@@@std@@NNAEAV23@@Z
 9    8 000012D0 ?norm@Linalg@@YANAEBV?$vector@NV?$allocator@N@std@@@std@@@Z
10    9 00001480 ?printVector@Linalg@@YAAEAV?$basic_ostream@DU?$char_traits@D@std@@@std@@AEAV23@AEAV?$vector@NV?$allocator@N@std@@@3@@Z
11    A 000018A0 ?set@SampleClass@@QEAAXH@Z

It is possible to demangle those symbols with the tool - https://demangler.com/

Symbol: ??0SampleClass@@QEAA@AEBV0@@Z demangled:

public: __cdecl SampleClass::SampleClass(class SampleClass const & __ptr64) __ptr64

Symbol: ?get@SampleClass@@QEAAHXZ demangled:

public: int __cdecl SampleClass::get(void) __ptr64

Symbol: ?linTransform@Linalg@@YA?AV?$vector@NV?$allocator@N@std@@@std@@NNAEAV23@@Z demangled:

class std::vector<double,class std::allocator<double> > __cdecl Linalg::linTransform(double,double,class std::vector<double,class std::allocator<double> > & __ptr64)

View exports symbols on Unix-like OSes (nm)

View exported symbols with nm:

Note that only functions annotated with extern “C” such as teslib_InterfaceClass_factory are not mangled.

$ nm -D libtestlib.so 

000000000020a3d8 B __bss_start
                 U __cxa_atexit
                 w __cxa_finalize
                 U __cxa_pure_virtual
000000000020a3d8 D _edata
000000000020a440 B _end
000000000000710c T _fini
                 w __gmon_start__
... ...   ... ...   ... ...   ... ...   ... ... 
0000000000005512 T teslib_InterfaceClass_factory
0000000000005629 T testlib_InterfaceClass_delete
0000000000005656 T testlib_InterfaceClass_getID
00000000000056a8 T testlib_InterfaceClass_getName
0000000000005677 T testlib_InterfaceClass_setName
000000000000545c T testlib_SampleClass_delete
000000000000548f T testlib_SampleClass_get
00000000000054cc T testlib_SampleClass_getName
 
... ... ... ...   ... ... ... ... 

0000000000005076 T _ZN11SampleClassC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
0000000000005106 T _ZN11SampleClassC2Ev
0000000000005196 T _ZN11SampleClassD1Ev

View exported symbols demangled (without name mangling):

 $ nm -CD libtestlib.so 
 000000000020a3d8 B __bss_start
                  U __cxa_atexit
                  w __cxa_finalize

  ... ... ...  ... ... ...  ... ... ...  ... ... ... 

 0000000000004fe7 T testlib_vectorD_Linalg_norm
 000000000000500f T testlib_vectorD_Linalg_printVector
 0000000000004e52 T testlib_vectorD_make0
 0000000000004ee2 T testlib_vectorD_make1
 0000000000004f80 T testlib_vectorD_set
                  U _Unwind_Resume
                  U operator delete(void*)
                  U operator delete(void*, unsigned long)
 00000000000052fe T SampleClass::get()
 0000000000005310 T SampleClass::set(int)

  ...  ...  ...  ...  ...  ...  ... 

 00000000000070ac W double* std::__copy_move<false, true, std::random_access_iterator_tag>::__copy_m<double>(double const*, double const*, double*)
 0000000000006cfe W void std::_Destroy_aux<true>::__destroy<double*>(double*, double*)
 0000000000006c74 W std::_Vector_base<double, std::allocator<double> >::_M_allocate(unsigned long)
 0000000000006a44 W std::_Vector_base<double, std::allocator<double> >::_Vector_impl::_Vector_impl(std::allocator<double> const&)
 0000000000006a44 W std::_Vector_base<double, std::allocator<double> >::_Vector_impl::_Vector_impl(std::allocator<double> const&)
 000000000000681a W std::_Vector_base<double, std::allocator<double> >::_Vector_impl::~_Vector_impl()
 000000000000681a W std::_Vector_base<double, std::allocator<double> >::_Vector_impl::~_Vector_impl()
 0000000000006ae8 W std::_Vector_base<double, std::allocator<double> >::_M_deallocate(double*, unsigned long)
 0000000000006a8e W std::_Vector_base<double, std::allocator<double> >::_M_create_storage(unsigned long)
 0000000000006926 W std::_Vector_base<double, std::allocator<double> >::_M_get_Tp_allocator()
 0000000000006836 W std::_Vector_base<double, std::allocator<double> >::_Vector_base(unsigned long, std::allocator<double> const&)
 00000000000069d2 W std::_Vector_base<double, std::allocator<double> >::_Vector_base(std::allocator<double> const&)
 0000000000006836 W std::_Vector_base<double, std::allocator<double> >::_Vector_base(unsigned long, std::allocator<double> const&)
 00000000000069d2 W std::_Vector_base<double, std::allocator<double> >::_Vector_base(std::allocator<double> const&)

... ... ... ... ... 

0000000000006706 W std::function<void ()>::function(std::function<void ()> const&)
0000000000006706 W std::function<void ()>::function(std::function<void ()> const&)
00000000000062b2 W std::function<void ()>::~function()
00000000000062b2 W std::function<void ()>::~function()
                 U std::ios_base::Init::Init()
                 U std::ios_base::Init::~Init()

Plugins and Dynamically Loading Classes

Overview

A Plugin system allows loading software components, classes and functions at runtime without from shared libraries without needing to compile or rebuild the main application.

A plugin architecture provides the following benefits:

Dynamically load class at runtime reducing compilation time.
Allow third-parties to create new plugins, addons, extensions and so on.
Hot-swap or replace plugins at runtime.
Faster compile-time

References:

Plugins in CPP: Dynamically Linking Shared Objects · christopherpoole/cppplugin Wiki · GitHub
How to Use the Dynamic Link Library in C++ Linux (gcc compiler)? | Technology of Computing
C++ Dynamic Loading of Shared Objects at Runtime - Programming - 0x00sec - The Home of the Hacker
SOFA – Create your plugin
The Geochemist’s Workbench® Plug-in: Wrapper
A Proposal to add Classes and Functions Required for Dynamic Library Load
Plugin Architecture – Nuclex Games Blog
Andy Nicholas » Blog Archive » A Simple Plugin Architecture For C++

Other Implementations:

GitHub - sourcey/pluga: Simple C++ cross-platform plugin system
DynObj - C++ Cross Platform Plugin Objects - CodeProject
Building a Simple C++ Cross-platform Plugin System | Sourcey
Making a Plugin System - C++ Articles
Using Dynamic Link Libraries (DLL) to Create Plug-Ins - 3D Game Engine Programming3D Game Engine Programming
https://github.com/jbandela/cppcomponents (Similar to Windows’ COM componets)
C++ patterns: static registration

API used for loading shared libraries dynamically

Unix C-API for loading shared libraries (shared objects)

// Returns handle to shared library 
void* dlopen(const char* fileName, int flags);

// Close handle 
int   dclose(void* handle)

// Get symbol (C-function or function with C-linkage)
void* dlsym(void* handle, const char* FunctionName)

Windows APIs: (Reference MSDN)

// Ansi version 
HMODULE LoadLibraryA( LPCSTR lpLibFileName);  
// Wide Unicode UTF-16 
HMODULE LoadLibraryW(LPCWSTR lpLibFileName);

// FAPROC => Returns Pointer to function casted as void*
FARPROC GetProcAddress(HMODULE hModule, LPCSTR  lpProcName);
 
void FreeLibrary(HMODULE hModule);

Example

Overview

This section presents a simple plugin system which allows loading classes at runtime from dynamically-loaded shared libraries. This approach allows adding new functionality to a software without recompilation or modification of the source code. The plugin system can load any class matching a user supplied interface without knowing anything about a particular class implementation.

In this example: The plugin PluginA, which the file name is PluginA.dll on Windows or PluginA.so on Linux, exports the classes ‘Log’ and ‘Exp’ that implements the interface class IMathFunction. The application loads those classes only knowing the interface IMathFunction. This design allows to add more third-party plugins and more classes matching any known interface class.

General Consideration for an Interface Class

MUST NOT have member variables.
MUST have only pure virtual member functions, in other words, only abstract methods.

ABI - Application Binary Interface Considerations for an Interface Class

SHOULD use only C-compatible types in the method signatures in order to ensure that the interface to be compatible with multiple compilers. Note: It is very likely that the application and the plugins (aka addons) may be developed with different compilers without a common ABI - Application Binary Interface.
SHOULD NOT use any use any STL type or container in the class method signatures, although the implementation classes provided by the plugins can use them internally. The reason to avoid the use of the STL is that the STL implementation is compiler-dependent what can lead to liking error when linking object-codes generated by different compilers.
The STL and non-C compatible can be used in interface classes’ methods signatures only if the main application and the plugins are developed by the same team with the same compiler version.

Fragile-Base-class Problem and class ABI

Avoid changing interfaces - Any interface class already being used by third parties plugins should not be changed, otherwise it will break the interface class ABI and disrupt third parties’ plugins as they will no longer be binary compatible with the interface. As a result, all plugins would have to be recompiled.
Fragile Base Class Problem: - Some changes that can break a base class ABI requiring recompilation of all derived classes are:
- Add new virtual member functions, aka virtual methods
- Change the order of virtual member functions
- Add new member variables
- Change the order or type of member variables
Solution: The solution to fragile base class problem in this case is: don’t modify interface classes relied by third-party plugins and add new interface classes or extend the existing ones instead of modifying them.

Source Code:

Repository: github/sample-cpp-plugin

File: psdk/interfaces.hpp

This interface class provides plugin metadata and a factory function (Factory) which allows to instantiate any class. The method Factory takes as argument the name of a class provided by the plugin and returns an instance of it casted as a void pointer (opaque pointer).

#ifndef _INTERFACES_HPP_
#define _INTERFACES_HPP_

struct IPluginFactory{
     /** Get Plugin Name */
     virtual const char* Name() const = 0 ;
     /** Get Plugin Version */
     virtual const char* Version() const = 0;

     virtual size_t NumberOfClasses() const = 0;
     virtual const char* GetClassName(size_t index) const = 0;

     /** Instantiate a class from its name */
     virtual void* Factory(const char* className) const = 0;
};

#endif

File: psdk/factory.hpp

Headers:

ifndef _FACTORY_HPP_
#define _FACTORY_HPP_

#include <string> 
#include <functional>
#include <map>
#include <vector>
#include <algorithm>

#include "interfaces.hpp"

/** Macro EXPORT_CPP makes a symbol visible. */
#if defined(_WIN32)
  // MS-Windows NT 
  #define PSDK_PLUGIN_EXPORT_C extern "C" __declspec(dllexport) 
#else
  // Unix-like OSes
  #define PSDK_PLUGIN_EXPORT_C extern "C" __attribute__ ((visibility ("default")))
#endif

Class PluginFactory: Implementation of the interface IPluginFactory.

class PluginFactory: public IPluginFactory
{
       // Constructor database
       using CtorItem = std::pair<std::string, std::function<void* ()>>;
       using CtorDB = std::vector<CtorItem>;
       std::string m_name;
       std::string m_version;	
       CtorDB m_ctordb;	
public:	
        PluginFactory(const char* name, const char* version):
            m_name(name),
            m_version(version)
        {
        }
	
        /** Get Plugin Name */
        const char* Name() const
        {
            return m_name.data();
        }

         /** Get Plugin Version */
        const char* Version() const
        {
            return m_version.data();
        }

        virtual size_t NumberOfClasses() const
        {
             return m_ctordb.size();
        }
        virtual const char* GetClassName(size_t index) const
        {
            return m_ctordb[index].first.data();
        }
	
        /** Instantiate a class from its name */
        void* Factory(const char* className) const
        {
            auto it = std::find_if(m_ctordb.begin(), m_ctordb.end(),
                                   [&className](const auto& p){
                                           return p.first == className;
                                   });
            if(it == m_ctordb.end())
                    return nullptr;
            return it->second();
        }

        template<typename AClass>
        PluginFactory& registerClass(std::string const& name)
        {
             auto constructor = []{ return new (std::nothrow) AClass; };
             m_ctordb.push_back(std::make_pair(name, constructor));
             return *this;
        }		
};
#endif

File: psdk/loader.hpp

Headers:

ifndef _LOADER_HPP_
#define _LOADER_HPP_

#include <cassert>
#include <map>
#include <memory>

#include "interfaces.hpp"

// Unix
#if defined(_WIN32)
   #include <windows.h>
#elif defined(__unix__)
  // APIs: dlopen, dlclose, dlopen 
  #include <dlfcn.h>
#else
  #error "Not supported operating system"
#endif 

#ifdef GetClassName 
  #undef GetClassName 
#endif

Class Plugin:

This class loads a shared library dynamically with dlopen on Unix or LoadLibrary on Windows and gets a function pointer to the symbol GetPluginFactory which returns a pointer to a statically allocated IPluginFactory object that allows instantiating at runtime any registered class in the plugin.

class Plugin
{	
public:
  // Function pointer to DLL entry-point
  using GetPluginInfo_fp = IPluginFactory* (*) ();

  void*        m_hnd		= nullptr;
  std::string  m_file		= "";
  bool         m_isLoaded = false;
  IPluginFactory* m_info     = nullptr;

  Plugin()
  {
  }

  explicit Plugin(std::string file)
  {
        m_file = std::move(file);
       #if !defined(_WIN32)
          m_hnd = ::dlopen(m_file.c_str(), RTLD_LAZY);
       #else
          m_hnd  = (void*) ::LoadLibraryA(m_file.c_str());
       #endif 
          m_isLoaded = true;
          assert(m_hnd != nullptr);
       #if !defined(_WIN32)
         auto dllEntryPoint =
             reinterpret_cast<GetPluginInfo_fp>(dlsym(m_hnd, "GetPluginFactory"));
       #else
         auto dllEntryPoint =
             reinterpret_cast<GetPluginInfo_fp>(GetProcAddress((HMODULE) m_hnd, "GetPluginFactory"));
       #endif 
       assert(dllEntryPoint != nullptr);
       // Retrieve plugin metadata from DLL entry-point function 
       m_info = dllEntryPoint();
  }

  ~Plugin()
  {
          this->Unload();
  }
  Plugin(Plugin const&) = delete;
  Plugin& operator=(const Plugin&) = delete;

  Plugin(Plugin&& rhs) 
  {
       m_isLoaded	= std::move(rhs.m_isLoaded);
       m_hnd		= std::move(rhs.m_hnd);
       m_file		= std::move(rhs.m_file);
       m_info      = std::move(rhs.m_info);
  } 
  Plugin& operator=(Plugin&& rhs)
  {
       std::swap(rhs.m_isLoaded, m_isLoaded);
       std::swap(rhs.m_hnd,  m_hnd);
       std::swap(rhs.m_file, m_file);
       std::swap(rhs.m_info, m_info);
       return *this;
  }

  IPluginFactory* GetInfo() const
  {
     return m_info;
  }

  void* CreateInstance(std::string const& className)
  {
      return m_info->Factory(className.c_str());
  }

  bool isLoaded() const
  {
      return m_isLoaded;
  }

  void Unload()
  {
      if(m_hnd != nullptr) {
         #if !defined(_WIN32)
            ::dlclose(m_hnd);
         #else
            ::FreeLibrary((HMODULE) m_hnd);
          #endif 
          m_hnd = nullptr;
          m_isLoaded = false;
      }
  }
};

Class PluginManager:

class PluginManager
{
public:
      using PluginMap = std::map<std::string, Plugin>;
      // Plugins database 
      PluginMap m_plugindb;

      PluginManager()
      {								
      }
      std::string GetExtension() const 
      {
         std::string ext;
         #if defined (_WIN32) 	     
               ext = ".dll"; // Windows 
         #elif defined(__unix__) && !defined(__apple__)
               ext = ".so";  // Linux, BDS, Solaris and so on. 
         #elif defined(__apple__)
               ext = ".dylib"; // MacOSX 
         #else 	 
           #error "Not implemented for this platform"
         #endif 
              return ext;
      }

      IPluginFactory* addPlugin(const std::string& name)
      {	  		
              std::string fileName = name + GetExtension();
              m_plugindb[name] = Plugin(fileName);
              return m_plugindb[name].GetInfo();
      }

      IPluginFactory* GetPluginFactory(const char* pluginName)
      {
              auto it = m_plugindb.find(pluginName);
              if(it == m_plugindb.end())
                      return nullptr;
              return it->second.GetInfo();
      }

      void* CreateInstance(std::string pluginName, std::string className)
      {
              auto it = m_plugindb.find(pluginName);
              if(it == m_plugindb.end())
                      return nullptr;		
              return it->second.CreateInstance(className);
      }

      template<typename T>
      std::shared_ptr<T>
      CreateInstanceAs(std::string pluginName, std::string className)
      {
              void* pObj = CreateInstance(std::move(pluginName), std::move(className));
              return std::shared_ptr<T>(reinterpret_cast<T*>(pObj));
      }
	
}; /* --- End of class PluginManager --- */


#endif

File: PluginA.cpp

Sample plugin file exporting classes Exp and Log which implement the interface IMathFunction. When compiled on Windows, the file name becomes PluginA.dll, when compiled on Linux, it becomes PluginA.so.

Header:

#include <iostream>
#include <cmath>
#include <psdk/factory.hpp>

Interface class IMathFunction:

class IMathFunction
{
public:
    virtual const char* Name() const = 0;
    virtual double  Eval(double x) const = 0;	
    virtual ~IMathFunction() = default;
};

Sample classes exported by the Plugin: implementations

class Exp: public IMathFunction
{
public:
    Exp() = default;
    const char* Name() const {
       return "Exp";
    }
    double Eval(double x) const {
       return std::exp(x);
    }
};

class Log: public IMathFunction
{
public:
    Log() = default;
    const char* Name() const {
       return "Log";
    }
    double Eval(double x) const {
       return std::log(x);
    }
};

Plugin Entry Point:

This function returns a PluginFactory object which contains the shared library metadata and a factory function for instantiating the exported classes Exp and Log. On Unix-like operating system, this global function is loaded by through the API dlopen and dlsym. On Windows, this entry point is loaded through the Win32 APIs LoadLibrary and GetProcAddress.

PSDK_PLUGIN_EXPORT_C
auto GetPluginFactory() -> IPluginFactory*
{
   static PluginFactory pinfo = []{
             auto p = PluginFactory("PluginA", "0.1-alpha");
             p.registerClass<Exp>("Exp");
             p.registerClass<Log>("Log");
             return p;
        }();
        return &pinfo;
}

File: main.cpp

Main application that loads the sample plugin PluginA, file name PluginA.so (Linux) or PluginA.dll (Windows).

Headers and interface:

#include <iostream>
#include <psdk/loader.hpp> 

class IMathFunction
{
public:
     virtual const char* Name() const = 0;
     virtual double  Eval(double x) const = 0;	
     virtual ~IMathFunction() = default;
};

Main Function:

Create a plugin manager object (ma), a repository of plugins:

PluginManager ma;

Load the plugin file at current directory, on Windows it loads the shared library file PluginA.dll and on Linux, it loads the file PluginA.so (aka shared object)

IPluginFactory* infoA = ma.addPlugin("PluginA");
assert(infoA != nullptr);

Print plugin metadata:

std::cout << " ---- Plugin Information --------- " << "\n"
          << "  =>              Name = " << infoA->Name() << "\n"
          << "  =>           Version = " << infoA->Version() << "\n"
          << "  => Number of classes = " << infoA->NumberOfClasses()
          << "\n\n";

std::cout << "Classes exported by the Plugin: (PluginA) " << "\n";
  
for(size_t n = 0; n < infoA->NumberOfClasses(); n++)
{
   std::cout << " -> Exported Class: " << infoA->GetClassName(n) << "\n";
}

Load class ‘Exp’ from the plugin or module ‘PluginA’ and casting it to the interface IMathFunction.

// Type of pExp: std::shared_ptr<IMathFunction>
auto pExp = ma.CreateInstanceAs<IMathFunction>("PluginA", "Exp");
assert(pExp != nullptr);
   
std::cout << "pExp->Name()    = " << pExp->Name() << "\n";
std::cout << "pExp->Eval(3.0) = " << pExp->Eval(3.0) << "\n";

Load class ‘Log’ from plugin ‘PluginA’:

auto pLog = ma.CreateInstanceAs<IMathFunction>("PluginA", "Log");
std::cout << "pLog->Name()    = " << pLog->Name() << "\n";
std::cout << "pLog->Eval(3.0) = " << pLog->Eval(3.0) << "\n";

Compilation on Linux:

# Build application 
$ g++ main.cpp -o main.bin -I. -Wall -std=c++1z -ldl

# Build the Plugin  DLL - Shared library 
$ g++ PluginA.cpp -o PluginA.so -I. -Wall -std=c++1z -shared -fPIC

# Test the application 
$ ./main.bin

Compile on Linux with CMake

$ git clone https://github.com/caiorss/sample-cpp-plugin
$ cd sample-cpp-plugin
$ cmake -Bbuild -H.
$ cmake --build build --target install
 
$ ls bin/
main.bin*  PluginA.so*

$ ./main.bin

Compile On Windows with CMake

$ git clone https://github.com/caiorss/sample-cpp-plugin
$ cd sample-cpp-plugin
$ cmake -H. -Bbuild2 -DCMAKE_BUILD_TYPE=DEBUG -G "Visual Studio 15 2017 Win64"
$ cmake --build build2 --target install
$ cd bin 

$ ls
PluginA.dll*  PluginA.lib  PluginA.so  main.bin  main.exe*
  
$ main.exe # Load PluginA.dll

Program Output:

$ ./main.bin 
 [TRACE] Shared library PluginA loaded OK.
 ---- Plugin Information --------- 
  =>              Name = PluginA
  =>           Version = 0.1-alpha
  => Number of classes = 2

Classes exported by the Plugin: (PluginA) 
 -> Exported Class: Exp
 -> Exported Class: Log
pExp->Name()    = Exp
pExp->Eval(3.0) = 20.0855
pLog->Name()    = Log
pLog->Eval(3.0) = 1.09861
 [TRACE] Shared library PluginA unloaded OK.
(base)

Return string from a C++ function to Python Ctypes

Example: Create a function with C linkage or C-API which returns a dynamically allocated string to the caller.

Source: Gist with all files

Necessary C-string functions:

strdup
strcpy, strcpy_s
strlen

Build shared library - DLL (cstr.so):

$ clang++ cstr.cpp -o cstr.so -std=c++1z -O0 -g -shared -fPIC -std=c++1z

File cstr.cpp

Headers:

#include <iostream>
#include <string>
#include <sstream>
#include <cstring>     // std::strncpy
#include <cmath>

#if defined(_WIN32)
  //  MS-Windows 
  #define EXPORT_C  extern "C" __declexpec(dllexport)
#else
  // Unix
  #define EXPORT_C  extern "C" 
#endif

Function - version A:

/** Design 1: Buffer allocated by the caller.
 * @param[in]     x            Float point number 
 * @param[in,out] buffer       Buffer allocated by the caller. 
 * @param[in]     buffer_size  Size of buffer allocated by the caller in bytes.
 * @return                     Size of copied array.
 */
EXPORT_C size_t 
square_root_as_stringA(double x, char* buffer, size_t buffer_size)
{
     std::string ss;
     ss = ss + "The square root of x = " + std::to_string(x)
             + " is equal to " + std::to_string(sqrt(x));
     std::strncpy(buffer, ss.data(), buffer_size);
     return ss.size();
}

Function - version B:

/** Design 2: Buffer allocated by the caller. If the buffer is null, the function 
 *            returns the required buffer size.
 * @param[in]     x            Float point number 
 * @param[in,out] buffer       Buffer allocated by the caller. 
 * @param[in]     buffer_size  Size of buffer allocated by the caller in bytes.
 * @return                     Size of copied array or required buffer size.                                
 */
EXPORT_C size_t 
square_root_as_stringB(double x, char* buffer, size_t buffer_size)
{
     std::string ss;
     ss = ss + "The square root of x = " + std::to_string(x)
             + " is equal to " + std::to_string(sqrt(x));
     if(buffer == nullptr)
             return ss.size();
     std::strncpy(buffer, ss.data() /* const char* */, buffer_size);
     return ss.size();
}

Function - version C:

/** Design3: Buffer allocated by callee (this code) and released by the
 * caller. */
EXPORT_C char* 
square_root_as_stringC(double x){
     std::string ss;
     ss = ss + "The square root of x = " + std::to_string(x)
             + " is equal to " + std::to_string(sqrt(x));
     return strdup(ss.data());
}

EXPORT_C void 
cstring_delete(char* pCstring)
{
    std::cout << " [INFO] C-string deleted, memory released. OK." << "\n";
    std::free(pCstring);
}

file: cstrp.py (Python Wrapepr)

Initial configuration:

import ctypes 

CString   = ctypes.POINTER(ctypes.c_char)
STDString = ctypes.c_void_p 
CVoid     = None 
dll = ctypes.cdll.LoadLibrary("cstr.so")

Wrapper for function A:

print("\n EXPERIMENT1 => Call square_root_as_stringA")
print("=============================================")
dll.square_root_as_stringA.argtypes = [ ctypes.c_double,
                                        CString,
                                        ctypes.c_size_t ]
dll.square_root_as_stringA.restype  = ctypes.c_size_t

buf_size = 1024 # 1 Kbyte = 1024 bytes
buf      = ctypes.create_string_buffer(buf_size)
n_read   = dll.square_root_as_stringA(100.45, buf, buf_size)

print("Size = ", n_read, " ; RESULT = ", buf.value)

Wrapper for function B:

print("\n EXPERIMENT 2 => Call square_root_as_stringB")
print("=============================================")
dll.square_root_as_stringB.argtypes = [ ctypes.c_double,
                                        CString,
                                        ctypes.c_size_t ]
dll.square_root_as_stringB.restype  = ctypes.c_size_t

x = 200.45
required_size = dll.square_root_as_stringB(x, ctypes.c_char_p(0), 0)
buf           = ctypes.create_string_buffer(required_size)
dll.square_root_as_stringB(x, buf, required_size)
print(" RESULT = ", buf.value)

Wrapper for function C:

print("\n EXPERIMENT 3 => Call square_root_as_stringC")
print("=============================================")

dll.square_root_as_stringC.argtypes = [ ctypes.c_double ]
dll.square_root_as_stringC.restype  = ctypes.c_char_p

dll.cstring_delete.argtypes = [ ctypes.c_char_p ]
dll.cstring_delete.restype  = CVoid

hCString = dll.square_root_as_stringC(167.42)
print(" RESULT = ", hCString)

Program output

$ python3 cstr.py 

 EXPERIMENT1 => Call square_root_as_stringA
=============================================
Size =  55  ; RESULT =  b'The square root of x = 100.450000 is equal to 10.022475'

 EXPERIMENT 2 => Call square_root_as_stringB
=============================================
 RESULT =  b'The square root of x = 200.450000 is equal to 14.158037'

 EXPERIMENT 3 => Call square_root_as_stringC
=============================================
 RESULT =  b'The square root of x = 167.420000 is equal to 12.939088'

Python Native Extension API

Overview

Many programming languages provide a C-native interfaces such as Python C-API or Java Native Interface where it is possible to create native-code libraries in C without overhead of virtual machine code interpretation. The advantage of a code for a C native interface (native API) of some programming language is the easier usage and better integration with the target language. The drawback for developing some native library with this approach is that the library will be tight coupled with the language native C API, as a result, it will be hard to port or call the code from other programming language. In addition, it may require deeper knowledge of the native C API which often is not well documented.

Definition:

A programming language native interface API is a C-API which allows creating libraries almost entirely in C compiled as shared libraries and load them in the same way as ordinary libraries.

Native Interface API X Foreign Function Interface based on Libffi

Advantages of Native Interface over FFI
- Better performance
- Enhanced integration
- Easier to use
Drawbacks of Native Interface over FFI:
- Most native APIs are not well documented and they are specific for a given programming language.
- Requires in-deep knowledge of the native C-API and the target programming language.
- Hard to reuse the code with other languages and even with C++.
- Code tightly coupled with the Native Interface API.
Advantages of FFI over Native Interface:
- The code does not need to conform to any native interface C-API and can be called from any programming language. The shortcoming of this approach is the need of C-interfaces (extern “C”) for the C++ code and a FFI wrapper.
- Most of FFIs are based on the library LibFFI which is used by Python (ctypes), Ruby, Java JNA, Haskell and so on. Therefore, the concept of creating FFI wrappers for C or C++ libraries with C-interfaces can be reused on all languages with LibFFI bindings.

Some Programming language specific Native Interface APIs (C-APIs):

Java Native Interface JNI
- JNI Specification
Python C-API
Ruby C-API
- https://silverhammermba.github.io/emberb/c/
Lua C-API
OCaml
- Chapter 20 Interfacing C with OCaml
GNU OCtave (Open source implementation of Matlab - Mathworks)
- GNU Octave: External Code Interface

Other approaches for developing shared libraries for Python native API:

Pybind11 header-only library
- => Provides a C++-friendly syntax and interface to the Python native C API. It allows building Python native modules and port C++ code in a easier and faster way than dealing directly with Python C-API.
SWIG wrapper generator
- => SWIG can parse C++ headers and interface files and generate C++ binding code to the native API of many programming languages such as Python, Ruby, Java and so on.
Cython
- => Python-like language that compile to C code (generate C-code). The disadvantage is the tight coupling to the Python C-API.

Further documentation and reading:

Documentation:

Porting Extension Modules to Python 3 — Python 3.7.2 documentation
Parsing Python Arguments — Python Extension Patterns 0.1.0 documentation
ActivePython 3.6.6 Documentation
Extending and Embedding the Python Interpreter — Python 3.7.2 documentation

General:

Common Object Structures — Python 3.7.2 documentation
Python Types and C-Structures — NumPy v1.15 Manual
7.3.1 String Objects
Code Objects — Python 2.7.15 documentation
The Very High Level Layer — Python 2.7.15 documentation
Create and call python function from string via C API - Stack Overflow
Embed Python scripting in C applications
Python C API: PyEval_CallFunction? - Stack Overflow
How do I call an object’s method from C?

Creating a native Python module:

Write C++ extensions for Python - Visual Studio | Microsoft Docs
Calling C functions from Python - part 2 - writing CPython extensions using Python/C API | yizhang82’s blog

Python (CPython) debugging with GDB:

22. gdb Support — Python Developer’s Guide
Mixed-mode debugging for Python - Visual Studio | Microsoft Docs

Reverse engineering:

Reverse Engineering Dynamic Languages - A Focus on Python

Python Raw C API

Module Code

Sources

Module Source:

File: src/dlls/mymodule.cpp
Gist: mymodule.cpp

Compilation:

# GCC - GNU C/C++ compiler 
$ g++ mymodule.cpp -o mymodule.so -g -std=c++1z -fPIC -shared -I/usr/include/python3.6m  
  
# Clang LLVM 
$ clang++ mymodule.cpp -o mymodule.so -g -std=c++1z -fPIC -shared -I/usr/include/python3.6m

File: setup.py

Python script used for building and installing the Python module.

from distutils.core import setup, Extension

mymodule = Extension('mymodule', sources = ['mymodule.cpp'])

setup(
    name        = 'mymodule',
    version     = '1.0',
    description = 'Sample python C-API exploration',
    ext_modules = [mymodule]
)

File: mymodule.cpp

Heades:

#include <iostream>
#include <string>
#include <functional>
#include <iomanip>

// Solve Mingw error: '::hyport' has not been declared 
#include <math.h>

//#define hypot _hypot

/// C-API header to Python Native Interface API, similar to Java JNI
/// (Java Native Interface)
#include <Python.h>

#ifdef _WIN32
  #include <process.h> // Exports _getpid()
#else
  #include <unistd.h>  // Exports getpid()
#endif

Global object for DLL initialization (optional):

The global object dllinit_hook is used for logging to stdout when the DLL is loaded or unloaded by the python process.

struct DLLInitialization
{
     DLLInitialization(){
         int pid;
         #ifdef _WIN32
            pid = _getpid();
         #else
            pid = getpid();       
         #endif 
         std::cerr << " [TRACE] Python module DLL loaded by process PID = <"
                           << pid << "> "
                           << std::endl;
         std::cerr << " [TRACE] Attach the debugger with: $ gdb -pid=" << pid << "\n";
     }
     ~DLLInitialization(){
         std::cerr << " [TRACE] DLL native DLL unloaded OK." << std::endl;
     }	
};

DLLInitialization dllinit_hook;

Exposed functions declarations:

/** ======== List of functions exposed to Python =========== */

PyObject*
exposedFunction(PyObject* self, PyObject* args);

PyObject* testArguments(PyObject* self, PyObject* args);

auto printRange(PyObject* self, PyObject* args) -> PyObject*;
auto taylorSeriesExp(PyObject* self, PyObject* args) -> PyObject*;
PyObject* returnTuple(PyObject* self, PyObject* args);
PyObject* returnDictionary(PyObject* self, PyObject* args);
PyObject* tabulateFunction(PyObject* self, PyObject* args);
PyObject* computeStatistics(PyObject* self, PyObject* args);

Table of exported functions to the Python module:

static PyMethodDef ModuleFunctions [] =
{
        { "exposedFunction", exposedFunction, METH_VARARGS,
          "Documentation or docstring of function exposedFunction1." }

        ,{"testArguments", testArguments, METH_VARARGS,
          "Test python arguments: Signature testArguments(int, float, double, const char*)"}
	
        // Function without docstring: printRange(int i, double x)
        ,{ "printRange", printRange, METH_VARARGS, nullptr }

        ,{ "taylorSeriesExp", &taylorSeriesExp, METH_VARARGS,
           "taylorSeriesExp(double x, size_t maxiter, double tol) -> double"
           "\n Computes exponential of a given value with taylor serie approximation."
           "\n Formula reference:  https://www.mathsisfun.com/algebra/taylor-series.html"
        }
        ,{"returnTuple", &returnTuple, METH_VARARGS, nullptr}
        ,{"returnDictionary", &returnDictionary, METH_VARARGS, nullptr}
        ,{"computeStatistics", &computeStatistics, METH_VARARGS, nullptr}
        ,{"tabulateFunction", tabulateFunction, METH_VARARGS,
          "Tabulate some mathematical function or callable object"}
        // Sentinel value used to indicate the end of function listing.
        // All function listing must end with this value.
        ,{nullptr, nullptr, 0, nullptr}									
};

Module definition:

/* Module definition */
static struct PyModuleDef ModuleDefinitions {
    PyModuleDef_HEAD_INIT,
   // Module name as string 
   "mymodule",
   // Module documentation (docstring)
   "A sample C++ native-code module for python3.",
   -1,
   // Functions exposed to the module 
   ModuleFunctions
};

Module initialization function:

Note: This function must contain the name in the format PyInit_<MODULE NAME> where module name is the same name as the module file name <MODULE NAME>.so (Linux) or <MODULE NAME>.pyd on Windows.

/** Module Initialization function: must have this name schema
 *  PyInit_<ModuleName> where ModuleName is the same base name of the 
 *  shared library ModuleName.so (on Linux) or ModuleName.pyd (on Windows)
 */
PyMODINIT_FUNC PyInit_mymodule(void)
{
   Py_Initialize();
   PyObject* pModule = PyModule_Create(&ModuleDefinitions);
   PyModule_AddObject(pModule, "version", Py_BuildValue("s", "version 0.1-Alpha"));
   return pModule;
}

Module Functions Implementations

All functions exposed to the Python C-API must have the same type signature of the function pointer PyExposedFunction or the same type signature of the function MyExposedFunction.

 using PyExposedFunction = PyObject* (*) (PyObject* self, PyObject* args)

 // Example: 
 PyObject* MyExposedFunction(PyObject* self, PyObject* args )
 {
    // On Failure return null 
    return nullptr;  
 
    return /* Some instance of PyObject */;
}

Notes:

The function takes always two arguments self, and args (tuple object containing arguments passed to the function)
The exposed function always return an object PyObject.
All Python objects are heap-allocated.
In Python everything is an object: integers, double, classes, class instances, modules and functions and so on. All those types have the same type PyObject in the C-API.
Functions exposed to Python runtime should not throw a C++ exceptions, instead they should return null (nullptr) that causes a runtime error or exception on Python side.

Function: exposedFunction

Function that takes a void argument and returns nothing.

PyObject*
exposedFunction(PyObject* self, PyObject* args)
{
     std::cout << " =>> Hello word Python from C++ side." << "\n";
     // All python functions that returns anything
     // should end with this macro 
     Py_RETURN_NONE;
}

Function: testArguments

Function that takes many arguments used to demonstrate how to extract tuple arguments.

PyObject* testArguments(PyObject* self, PyObject* args)
{
      int         abool;
      int         aint;
      float       afloat;
      double      adouble;
      const char* aword;
      // Parse function arguments 
      if(!PyArg_ParseTuple(args, "pifds", &abool, &aint, &afloat, &adouble, &aword))
          return nullptr;

      std::cerr << " Function: [" << __FUNCTION__ << "]" 
                << " ==> Number of arguments = " << PyTuple_GET_SIZE(args) << "\n";

      std::string str = aword;
      str = "'Received string = " + str + "'";

      std::cout << " Received Python Arguments " << "\n";
      std::cout << std::boolalpha;
      std::cout << "abool[bool] = " << (bool) abool << "\n"
                        << " abool[int] = " << abool << "\n"
                        << "      aint  = " << aint << "\n"
                        << "     afloat = " << afloat << "\n"
                        << "    adouble = " << adouble << "\n"
                        << "    aword   = " << str << "\n"
                        << "\n";		
      return Py_BuildValue("s", str.data());
}

Function PyArg_ParseTuple:

The function PyArg_ParseTuple is used for extracting function arguments from the tuple object (args).
The letters in the string parameter “pifds” means:
- p => Extract first argument from tuple object as boolean (integer 0 or 1) storing the value in the variable abool.
- i => Extract second argument from tuple object as integer storing the value in the variable ‘aint’.
- f => Extract third argument as float (float point 32 bits). The value is then stored in the variable ‘afloat’.
- d => Extract fourth argument as double (float point 64 bits)
- s => Extract fith argument as string.
If the function fails, it must return null which causes a runtime error on Python side.

int         abool;
int         aint;
float       afloat;
double      adouble;
const char* aword;
// Parse function arguments 
if(!PyArg_ParseTuple(args, "pifds", &abool, &aint, &afloat, &adouble, &aword))
    return nullptr;

Function Py_BuildValue(”s”, str.data());

This function is used for building return values to the python side. It can return any python object, such as string, number, class instance or multiple values as a tuple.

// Return None in the python side 
return Py_BuildValue("");

// Return a single value as string string 
return Py_BuildValue("s", str.data());
return Py_BuildValue("s", "A string to python side");

// Return integer 
return Py_BuildValue("i", 1000);

// Return float  
return Py_BuildValue("f", 3.465);

// Return double   
return Py_BuildValue("d", 140.346);

// Return a tuple with a single double value: (3.156)
return Py_BuildValue("(d)", 3.156);

// Return a tuple with a single double and integer: (3.156, 100)
return Py_BuildValue("(di)", 3.156, 100);

// Return a tuple with a single double, integer and string
// tuple object: (3.156, 100, "a string")
return Py_BuildValue("(dis)", 3.156, 100, "a string");

Function: printRange

Pseudo-Python signature: printRange(int n, float x) => void

auto printRange(PyObject* self, PyObject* args) -> PyObject*
{
    std::cerr << " =>> Printing numeric range " << "\n";	
    int    n;
    float  x;
    // Extract function argument from tuple argument object
    // "if" => i - Extract integer
    //      => f - Extract float (IEEE754 32 bits float point)
    if(!PyArg_ParseTuple(args, "if", &n, &x)){
        PyErr_SetString(PyExc_RuntimeError, "Invalid argument");
        // Always return null on failure 
        return nullptr; 
    }

    if(n <= 0){
        PyErr_SetString( PyExc_RuntimeError
                        ,"Invalid argument: n supposed to be greater than zero.");
        return nullptr;
    }	

    for(int k = 0; k < n; k++)
       std::cout << " k[" << k << "] = " << 3 * k + x << "\n";	
    Py_RETURN_NONE;
}

Function: returnTuple

Return multiple values as tuple.

/** Function that returns multiple values as tuple object */
PyObject* returnTuple(PyObject* self, PyObject* args)
{
     double x;
     double y;
     int    n;
     if(!PyArg_ParseTuple(args, "ddi", &x, &y, &n))
             return nullptr;
     // Build tuple object containing
     // (d: double, d: double, i: integer, s: string)
     return Py_BuildValue("(ddis)", 2 * x + y * n, x + 2 * y - n, n, "Result"); 
}

Function: returnDictionary

Return a dictionary (hash-table)

/** Return a dictionary object */
PyObject* returnDictionary(PyObject* self, PyObject* args)
{
     double x;
     double y;
     int    n;
     if(!PyArg_ParseTuple(args, "ddi", &x, &y, &n))
             return nullptr;

     // Create new dictionary object 
     PyObject* pDict = PyDict_New();

     PyObject* p = nullptr;
     // Fill dictionary
     p = Py_BuildValue("d", 2 * x + y * n);
     PyDict_SetItemString(pDict, "alpha", p);
     Py_DECREF(p);

     p = Py_BuildValue("i", 3 * n) ;
     PyDict_SetItemString(pDict, "size", p);
     Py_DECREF(p);

     p = Py_BuildValue("s", "Some string") ;
     PyDict_SetItemString(pDict, "beta", p);
     Py_DECREF(p);	

     return pDict;
}

Function: computeStatistics

Computes statistics sum and mean for any iterable object containing double values such as list.

PyObject* computeStatistics(PyObject* self, PyObject* args)
{
     std::cerr << " [TRACE] " << __FILE__ << "(" << __LINE__ << ") - "
                       << "Calling function: " << __FUNCTION__ << "\n";

     std::cerr << " [TRACE] Arguments = "; PyObject_Print(args, stdout, 0); std::cerr << "\n";

     PyObject* pSeq;
     // Parse function argument as sequence
     if(!PyArg_ParseTuple(args, "O", &pSeq))
             return nullptr;

     pSeq = PySequence_Fast(pSeq, "Expected iterable arguments");
     if(pSeq == nullptr) return nullptr;

     int numberOfElements = PySequence_Fast_GET_SIZE(pSeq);
     std::cerr << " [TRACE] numberOfElements = " << numberOfElements << "\n";

     PyObject* pItem = nullptr;
     double x;
     double sum;

     for(int n = 0; n < numberOfElements; n++)
     {
         pItem = PySequence_Fast_GET_ITEM(pSeq, n);
         if(!pItem) {
             Py_DECREF(pSeq);
             return nullptr;
         }
         // Print item 
         std::cout << "item[" << n << "] = ";
         PyObject_Print(pItem, stdout, 0);
         std::cout << "\n";

         x = PyFloat_AsDouble(pItem);
         if(PyErr_Occurred() != nullptr){
                 PyErr_SetString(PyExc_TypeError, "Error: expected float point.");
                 return nullptr;
         }
         sum += x;
     }

     double mean = sum / numberOfElements;
     std::cout << "\n (C++) Statics Result " << "\n";
     std::cout << "---------------" << "\n";
     std::cout << " Sum  = " << sum << "\n";
     std::cout << " Mean = " << mean << "\n";

     Py_DECREF(pSeq);
     return Py_BuildValue(""); // Return None 
}

Function: taylorSeriesExp

This function computes the exponential function approximation using its taylor series expansion. Formula at: taylor series

auto taylorSeriesExp(PyObject* self, PyObject* args) -> PyObject*
{
     double x;
     size_t maxiter;  // Maximum number of iterations 
     double tol;      // Tolerance

     // Parse function arguments 
     if(!PyArg_ParseTuple(args, "did", &x, &maxiter, &tol))
             return nullptr;

     // Validate function arguments

     if(tol <= 0 || tol > 1.0){
             PyErr_SetString( PyExc_RuntimeError
                            ,"Invalid tolerance, expected in range (0, 1]");
             return nullptr;
     }

     // Compute exponential taylor series available at
     // https://www.mathsisfun.com/algebra/taylor-series.html
     unsigned long factorial = 1;
     double        xpower    = 1.0;
     double        sum       = 0.0;
     double        term      = 0.0;
     size_t        idx       = 1;
     do{
          term      = xpower / factorial;
          sum       = sum + term;
          xpower    = xpower * x ;
          factorial = factorial * idx;		
          idx++;
     } while(idx <= maxiter && std::abs(term) > std::abs(sum) * tol );
     // Return float point constnat NAN (Not a Number)
     if(idx >= maxiter){
             std::cerr << " [ERROR] Series does not converge." << "\n";
             return Py_BuildValue("d", NAN);
     }
     return Py_BuildValue("d", sum);
}

Function: tabulateFunction

This function takes a callable object (callback) which can be a function, lambda function, callable object (object with method __call__) and so on.

Pseudo Python method signature using C++ notation:

using Callback = std::function<double (double)>;
void tabulateFunction(Callback callback, double xmin, double xmax, double step)

Function code:

PyObject* tabulateFunction(PyObject* self, PyObject* args)
{
     PyObject* pObj = nullptr;
     double xmin, xmax, xstep;

     if(!PyArg_ParseTuple(args, "Oddd", &pObj, &xmin, &xmax, &xstep))
          return nullptr;
     if(pObj == nullptr) {
          PyErr_SetString(PyExc_RuntimeError, "Error: invalid None object.");
          return nullptr;
     }
     PyObject* pArgs  = nullptr;
     PyObject* pResult = nullptr;
     double y = 0.0;

     std::cout << std::fixed << std::setprecision(4);

     std::cout << "Tabulating range: "
               << " ; xmin = " << xmin
               << " ; xmax = " << xmax
               << " ; step = " << xstep
               << "\n";

     for(double x = xmin; x <= xmax; x += xstep )
     {
          pArgs  = Py_BuildValue("(d)", x);
          pResult = PyEval_CallObject(pObj, pArgs);
          y = PyFloat_AsDouble(pResult);
          if(PyErr_Occurred() != nullptr){
                  PyErr_SetString(PyExc_RuntimeError, "Error: Invalid float point.");
                  return nullptr;
          }
          std::cout << std::setw(8) << x << std::setw(10) << y  << "\n";
     }
     Py_RETURN_NONE;
}

Extract python callable object (callback) pObj and the parameters xmin, xmax and xstep:

PyObject* pObj = nullptr;
double xmin, xmax, xstep;
// Extract parameters 
// O => pObj 
// d => xmin 
// d => xmax 
if(!PyArg_ParseTuple(args, "Oddd", &pObj, &xmin, &xmax, &xstep))
     return nullptr;

Annotated code block in the for-loop:

// Argument (tuple object) of callable object 
pArgs  = Py_BuildValue("(d)", x);

// Call callable object (method __call__)
pResult = PyEval_CallObject(pObj, pArgs);

// Try extracting result as double 
y = PyFloat_AsDouble(pResult);

// Check whether any error has happened and abort the operation in this case. 
if(PyErr_Occurred() != nullptr){
     PyErr_SetString(PyExc_RuntimeError, "Error: Invalid float point.");
     return nullptr;
}
// Display table row 
std::cout << std::setw(8) << x << std::setw(10) << y  << "\n";

Compiling and Running Native Module

Clone repository:

$ git clone https://gist.github.com/caiorss/bc7dc373d87b828029eb0e0ec048d91e python-module 
$ cd python-module

Manual Compilation

Compile the native module mymodule.so on Linux or any other Unix-like OS:

$ clang++ mymodule.cpp -o mymodule.so -g -std=c++1z -fPIC -shared -I/usr/include/python3.6m

Compile the native module mymodule.pyd on Windows OS with Mingw compiler:

$  g++ mymodule.cpp -o mymodule.pyd -g -std=c++1z -shared -fPIC \
   -I"C:\Users\archbox\Miniconda3\pkgs\python-3.6.5-h0c2934d_0\include" \
   C:\Users\archbox\Miniconda3\pkgs\python-3.6.5-h0c2934d_0\libs\Python36.lib

Compilation with setup.py

Build only:

$ python.exe setup.py build

$ python.exe setup.py build 

$ cd build\
lib.win-amd64-3.6\   temp.win-amd64-3.6\
F:\python-module (master -> origin)
λ cd build\lib.win-amd64-3.6\

$ ls
mymodule.cp36-win_amd64.pyd*

$ C:\Users\archbox\Miniconda3\python.exe
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import mymodule as m
 [TRACE] Python module DLL loaded by process PID = <620>
 [TRACE] Attach the debugger with: $ gdb -pid=620

Install:

$ python.exe setup.py install
For uninstall see: question 1550226

$ python.exe setup.py install

 $ C:\Users\archbox\Miniconda3\python.exe setup.py install
 running install
 running build
 running build_ext
 building 'mymodule' extension
 creating build\temp.win-amd64-3.6
 creating build\temp.win-amd64-3.6\Release
 C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox
  /W3 /GL /DNDEBUG /MD -IC:\Users\archbox\Miniconda3\include -IC:\Users\archbox\Miniconda3\include "-IC:\Program Files (x86)\Mi
 crosoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" ... ... 
 ... ....   ... ....   ... ....   ... ....   ... ....   ... ....   ... ....  
 ... ....   ... ....   ... ....   ... ....   ... ....   ... ....   ... ....  
 Finished generating code
 running install_lib
 copying build\lib.win-amd64-3.6\mymodule.cp36-win_amd64.pyd -> C:\Users\archbox\Miniconda3\Lib\site-packages
 running install_egg_info
 Writing C:\Users\archbox\Miniconda3\Lib\site-packages\mymodule-1.0-py3.6.egg-info

Check generated file on Linux:

$ file mymodule.so 

$ du -h mymodule.so 
136K	mymodule.so
136K	total

View symbols:

$ nm  mymodule.so 
0000000000209518 B __bss_start
0000000000005500 t __clang_call_terminate
0000000000209518 b completed.7347
                 U __cxa_atexit@@GLIBC_2.2.5
                 U __cxa_begin_catch@@CXXABI_1.3
                 U __cxa_call_unexpected@@CXXABI_1.3
                 U __cxa_end_catch@@CXXABI_1.3
                 w __cxa_finalize@@GLIBC_2.2.5
                 U __cxa_rethrow@@CXXABI_1.3
0000000000004160 t __cxx_global_var_initmymodule.cpp_
00000000000041a0 t __cxx_global_var_initmymodule.cpp_.1
00000000000041f0 t deregister_tm_clones
000000000020951a B dllinit_hook
.. ... ... ... ... ... ... ...

Load module in Python 3 REPL:

$ python3
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import mymodule as m
 [TRACE] Python module DLL loaded by process PID = <5236> 
 [TRACE] Attach the debugger with: $ gdb -pid=5236
>>> 
>>>

Module Docstring (documentation):

>>> help(m)
Help on module mymodule:

NAME
    mymodule - A sample C++ native-code module for python3.

FUNCTIONS
    computeStatistics(...)
 
    exposedFunction(...)
        Documentation or docstring of function exposedFunction1.
 
    printRange(...)
 
    returnDictionary(...)
 
    returnTuple(...)
 
    tabulateFunction(...)
        Tabulate some mathematical function or callable object
 
    taylorSeriesExp(...)
        taylorSeriesExp(double x, size_t maxiter, double tol) -> double
        Computes exponential of a given value with taylor serie approximation.
        Formula reference:  https://www.mathsisfun.com/algebra/taylor-series.html
 
    testArguments(...)
        Test python arguments: Signature testArguments(int, float, double, const char*)

DATA
    version = 'version 0.1-Alpha'

FILE
    /home/archbox/shared/python-module/mymodule.so

Module version:

>>> m.version
'version 0.1-Alpha'
>>>

Call function exposedFunction()

>>> m.exposedFunction()
 =>> Hello word Python from C++ side.
>>>

Call function testArguments:

>>> s1 = m.testArguments(False, 100, 10.9361, 0.344, "from Python to C++")
 Function: [testArguments] ==> Number of arguments = 5
 Received Python Arguments 
abool[bool] = false
 abool[int] = 0
      aint  = 100
     afloat = 10.9361
    adouble = 0.344
    aword   = 'Received string = from Python to C++'

>>> s1
"'Received string = from Python to C++'"
>>> 

>>> 
>>> s2 = m.testArguments(True, 90, -0.56e6, 9.344e5, "something else more")
 Function: [testArguments] ==> Number of arguments = 5
 Received Python Arguments 
abool[bool] = true
 abool[int] = 1
      aint  = 90
     afloat = -560000
    adouble = 934400
    aword   = 'Received string = something else more'

>>> s2
"'Received string = something else more'"

Call function printRange()

>>> m.printRange(5, 4.5)
 =>> Printing numeric range 
 k[0] = 4.5
 k[1] = 7.5
 k[2] = 10.5
 k[3] = 13.5
 k[4] = 16.5

>>> m.printRange(-5, 4.5)
 =>> Printing numeric range 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Invalid argument: n supposed to be greater than zero.

>>> m.printRange(0, 4.5)
 =>> Printing numeric range 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Invalid argument: n supposed to be greater than zero.

Calll function returnTuple:

>>> tpl = m.returnTuple(3.5, 4.2, 10)

>>> tpl
(49.0, 1.9000000000000004, 10, 'Result')

>>> tpl[0]
49.0

>>> tpl[1]
1.9000000000000004

>>> tpl[2]
10

>>> m.returnTuple(8.5, 4.2, 20)
(101.0, -3.1000000000000014, 20, 'Result')

Call functon returnDictionary:

>>> d = m.returnDictionary(8.5, 8.62, 10)

>>> d
{'alpha': 103.19999999999999, 'size': 30, 'beta': 'Some string'}

>>> d["alpha"]
103.19999999999999

>>> d["beta"]
'Some string'

>>> d["size"]
30

Call function computeStatistics:

>>> xs = [4, 1.2, 9.5, 10.5, 3.4, 5.10, 6.0]

>>> xs
[4, 1.2, 9.5, 10.5, 3.4, 5.1, 6.0]

>>> m.computeStatistics(xs)
 [TRACE] mymodule.cpp(268) - Calling function: computeStatistics
 [TRACE] Arguments = ([4, 1.2, 9.5, 10.5, 3.4, 5.1, 6.0],)
 [TRACE] numberOfElements = 7
item[0] = 4
item[1] = 1.2
item[2] = 9.5
item[3] = 10.5
item[4] = 3.4
item[5] = 5.1
item[6] = 6.0

 (C++) Statics Result 
---------------
 Sum  = 39.7
 Mean = 5.67143
>>>

Call function taylorSerieExp:

>>> help(m.taylorSeriesExp)
Help on built-in function taylorSeriesExp in module mymodule:

taylorSeriesExp(...)
    taylorSeriesExp(double x, size_t maxiter, double tol) -> double
    Computes exponential of a given value with taylor serie approximation.
    Formula reference:  https://www.mathsisfun.com/algebra/taylor-series.html

>>> import math

# Exact value 
>>> math.exp(3.5)
33.11545195869231

# Approximation 
>>> m.taylorSeriesExp(3.5, 10, 0.001)
 [ERROR] Series does not converge.
nan
>>> m.taylorSeriesExp(3.5, 100, 0.001)
33.10588185071678
>>> 

# Exact value 
>>> math.exp(5.0)
148.4131591025766

# Approximation 
>>> m.taylorSeriesExp(5.0, 100, 0.001)
148.37958007973663

>>> m.taylorSeriesExp(5.0, 100, 0.0001)
148.41021027504306
>>>

Call function: tabulateFunction which takes a callable object as argumet.

>>> help(m.tabulateFunction)
Help on built-in function tabulateFunction in module mymodule:

tabulateFunction(...)
    Tabulate some mathematical function or callable object

Pass ordinary functions:

Note: Python ordinary functions are objects with __call__ method:

import math 

>>> math.sqrt.__call__(25)
5.0

>>> math.sqrt.__call__(125)
11.180339887498949
>>> 

>>> m.tabulateFunction(math.exp, -5.0, 5.0, 1.0)
Tabulating range:  ; xmin = -5.0000 ; xmax = 5.0000 ; step = 1.0000
 -5.0000    0.0067
 -4.0000    0.0183
 -3.0000    0.0498
 -2.0000    0.1353
 -1.0000    0.3679
  0.0000    1.0000
  1.0000    2.7183
  2.0000    7.3891
  3.0000   20.0855
  4.0000   54.5982
  5.0000  148.4132
>>> 

>>> a.tabulateFunction(math.sqrt, 0.0, 25.0, 5.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 5.0000
  0.0000    0.0000
  5.0000    2.2361
 10.0000    3.1623
 15.0000    3.8730
 20.0000    4.4721
 25.0000    5.0000

Pass lambda functions:

>>> m.tabulateFunction(lambda x: x, 0.0, 25.0, 4.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 4.0000
  0.0000    0.0000
  4.0000    4.0000
  8.0000    8.0000
 12.0000   12.0000
 16.0000   16.0000
 20.0000   20.0000
 24.0000   24.0000
>>> 

>>> m.tabulateFunction(lambda x: 3 * x + 5, 0.0, 25.0, 4.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 4.0000
  0.0000    5.0000
  4.0000   17.0000
  8.0000   29.0000
 12.0000   41.0000
 16.0000   53.0000
 20.0000   65.0000
 24.0000   77.0000

Class LinearFun:

class LinearFun:
    def __init__(self, a, b):
        self.a = a
        self.b = b
    def __call__(self, x):
        return self.a * x + self.b
    def eval(self, x):
        return self.a * x + self.b

Pass a callable object:

>>> lfun = LinearFun(5, 4)
>>> lfun(3)
19
>>> lfun(6)
34
>>> lfun.eval(3)
19
>>> lfun.eval(6)
34

>>> m.tabulateFunction(lfun, 0.0, 25.0, 4.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 4.0000
  0.0000    4.0000
  4.0000   24.0000
  8.0000   44.0000
 12.0000   64.0000
 16.0000   84.0000
 20.0000  104.0000
 24.0000  124.0000
>>> 
>>> 

>>> a.tabulateFunction(lfun.eval, 0.0, 25.0, 4.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 4.0000
  0.0000    4.0000
  4.0000   24.0000
  8.0000   44.0000
 12.0000   64.0000
 16.0000   84.0000
 20.0000  104.0000
 24.0000  124.0000

>>> lfun.a 
5
>>> lfun.a  = 0
>>> 
>>> m.tabulateFunction(lfun.eval, 0.0, 25.0, 4.0)
Tabulating range:  ; xmin = 0.0000 ; xmax = 25.0000 ; step = 4.0000
  0.0000    4.0000
  4.0000    4.0000
  8.0000    4.0000
 12.0000    4.0000
 16.0000    4.0000
 20.0000    4.0000
 24.0000    4.0000

Native Modules or C++ binding with Pybind11

Overview

Pybind11 is a C++ header-only library that simplifies the creation of Python binding to existing C++ libraries or creating Python native modules. This library is easier and less cumbersome to use than Python raw C API due to the library declarative and intuitive syntax and lots of C++ wrappers.

Use cases:

Create high-performance python modules in C++
Create binding to existing C or C++ libraries.
Embed Python in a C++ application
Access to operating system APIs not exposed in Python standard library

Documentation:

https://pybind11.readthedocs.io/en/stable/

Github Repository:

https://github.com/pybind/pybind11

Conan package reference:

pybind11/2.2.4@conan/stable

Headers:

<pybind11/pybind11.h>
- Main header, that should be always included
<pybind11/stl.h>
- Provides implicit convesion from STL containers to Python types.
<pybind11/functional.h>

Example

Overview

The following sample CMake/conan project contains many PyBind11 examples about binding existing C++ functions and classes and access or call Python objects and APIs from C++.

https://github.com/caiorss/example-project-pybind11

File: CMakeLists.txt

Note: In order to use this CMakeLists.txt file, it is necessary an installation of conan package manager available in $PATH environment variable, in other words, accessible from command line.

#      Author: Caio Rodrigues 
# Description:  Sample CMake + Conan + Pybind11 project
#        Note:  Requires Conan and Python Development files installed 
#----------------------------------------------------------

cmake_minimum_required(VERSION 2.8)
project(pybind11-experiments)
set(CMAKE_CXX_STANDARD 17)


# ============  Conan Boostrap  ======================#

# Download automatically, you can also just copy the conan.cmake file
if(NOT EXISTS "${CMAKE_BINARY_DIR}/conan.cmake")
   message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
   file(DOWNLOAD "https://github.com/conan-io/cmake-conan/raw/v0.13/conan.cmake"
                 "${CMAKE_BINARY_DIR}/conan.cmake")
endif()

include(${CMAKE_BINARY_DIR}/conan.cmake)

conan_cmake_run(REQUIRES   pybind11/2.2.4@conan/stable
                BASIC_SETUP
                BUILD missing)

#============ Additional Commands =============#

# Copy target file to current directory whenerver it is rebuilt
function(copy_after_build TARGET_NAME )
    # Note: CMAKE_CURRENT_LIST_DIR is the directory where is this
    # CMakeLists.txt file.
    set(DESTDIR ${CMAKE_CURRENT_LIST_DIR}/bin/)
    file(MAKE_DIRECTORY ${DESTDIR})

    # Copy binary file to <CMakeLists.txt didctory>./bin
    # after target is compiled.
    add_custom_command(TARGET ${TARGET_NAME} POST_BUILD
      COMMAND ${CMAKE_COMMAND} -E copy
                               $<TARGET_FILE:${TARGET_NAME}> ${DESTDIR}
                               )
endfunction()


#----------------------------------------------#
#    Target Configurations                     #
#----------------------------------------------#

find_package(pybind11 CONFIG REQUIRED)

pybind11_add_module(SampleModule SampleModule.cpp)
# Copy binary to ./bin directory after module is built
copy_after_build(SampleModule)

# Target1: cashflow - Python module basics
#.....................................
pybind11_add_module(cashflow cashflow.cpp)
copy_after_build(cashflow)

File: SampleModule.cpp

Full Source

File: SampleModule.cpp

Header and Namespaces

Header files:

#include <iostream>
#include <string>
#include <vector>
#include <iomanip>
#include <cmath>

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <pybind11/functional.h>

Namespaces:

namespace py = pybind11;
using namespace pybind11::literals;

Bindings C++ Functions, enums and Classes

Bindings to C++ functions, enums and classes without any knowledge of Python API.

Function add:

int add(int i, int j)
{
    return i + j;
}

Registration:

PYBIND11_MODULE(SampleModule, m) {
   // optional module docstring
   m.doc() = "Sample Python built with C++ CeePlusPlus ";

   // Register an ordinary C++ function
   m.def("add",                                 // Function Name in Python side
         &add,                                  // Function Pointer
         "A function which adds two numbers",   // Docstring
         py::arg("x"),                          // Name of argument 1
         py::arg("y") = 10);                    // Name of argument 2 with default value
 ... ...   ... ...   ... ...   ... ...   ... ... 
}

Function vectorCumulativeSum

std::vector<double>
vectorCumulativeSum(std::vector<double> const& xs)
{
    std::vector<double> out(xs.size());
    double sum = 0.0;
    for(size_t i = 0; i < xs.size(); i++)
    {
        sum += xs[i];
        out[i] = sum;
    }
    return out;
}

Registration:

PYBIND11_MODULE(SampleModule, m) {
 ...   ...   ...   ...   ...
  m.def("vectorCumulativeSum", &vectorCumulativeSum);    
 ...   ...   ...   ...   ...
}

Function tabulate1

void tabulate1(double start, double stop, double step,
               std::function<double (double)> func)
{
    if(start > stop || step <= 0)
        throw std::runtime_error(" Error: Invalid domain "
                                 "\n Expected: start < stop && step > 0 "
                                 );

    std::cout << std::setprecision(5) << std::fixed;
    for(double x = start; x <= stop; x+= step)
        std::cout << std::setw(10) << x
                  << std::setw(10) << func(x)
                  << std::endl;
}

Registration:

PYBIND11_MODULE(SampleModule, m) {
 ...   ...   ...   ...   ...
  m.def("tabulate1", &tabulate1);
 ...   ...   ...   ...   ...
}

Enum class MathErrorCode:

enum class MathErrorCode: std::uint32_t
{
     OK           = 0x00
   , OVERFLOW     = 0x01
   , UNDERFLOW    = 0x04
   , NOT_A_NUMBER = 0x08
};

Registration:

PYBIND11_MODULE(SampleModule, m) {
 ...   ...   ...   ...   ...

  py::enum_<MathErrorCode>(m, "MathErrorCode", py::arithmetic())
     .value("OK",           MathErrorCode::OK)
     .value("OVERFLOW",     MathErrorCode::OVERFLOW)
     .value("UNDEFLOW",     MathErrorCode::UNDERFLOW)
     .value("NOT_A_NUMBER", MathErrorCode::NOT_A_NUMBER);
    // .export_values();
 ...   ...   ...   ...   ...
}

Class LinearFunctor:

// Sample "function-object class"
class LinearFunctor
{
public:
    double A = 0, B = 0;

    LinearFunctor();
    LinearFunctor(double a, double b): A(a), B(b){ }

    double GetA() const   { return A; }
    void   SetA(double a) { A = a; }
    double GetB() const   { return B; }
    void   SetB(double b) { B = b; }

    void show() const
    {
        std::cout << " LinearFunction: y(x) = A * x + B" << std::endl;
        std::cout << " => A = " << this->A << " ; B = " << this->B << std::endl;
    }
    std::string toString() const
    {
        std::stringstream ss;
        ss << " LinearFunction: y(x) = A * x + B" << std::endl;
        ss << " => A = " << this->A << " ; B = " << this->B << std::endl;
        return ss.str();
    }
    // Function-call operator
    double operator()(double x)
    {
        return A * x + B;
    }
};

Registration:

PYBIND11_MODULE(SampleModule, m) {
  ... ...   ... ...   ... ...   ... .

    // Register LinearFunction
    py::class_<LinearFunctor>(m, "LinearFunctor")
            .def(py::init<double, double>())             // Register overloaded consructor
            .def("GetA", &LinearFunctor::GetA)            // Reister method GetA()
            .def("GetB", &LinearFunctor::GetB)            // Register method GetB()
            .def("SetA", &LinearFunctor::SetA)            // Reister method GetA()
            .def("SetB", &LinearFunctor::SetB)            // Register method GetB()
            .def("show", &LinearFunctor::show)            // Register method show
            .def("call", &LinearFunctor::operator())      // Register function-call operator with name 'call'
            .def("__call__", &LinearFunctor::operator ()) // Register fun-call operator
            .def("__repr__", &LinearFunctor::toString)    // Register strin representation
            .def_readwrite("A", &LinearFunctor::A)        // Register field A
            .def_readwrite("B", &LinearFunctor::B);       // Register field B

  ... ... 
}

Functions with knowledge of Python API

Function computeFormula
- This function takes a Python tuple py::args as argument and returns a Python dictionary py::dict

// Sample function that takes a python tuple as argument and
// returns a dictionary.
py::dict
computeFormula(py::args args)
{
    py::print(" [INFO] Number of args = {s1} ; args = {s2}",
              "s1"_a = args.size(), "s2"_a = args);
    std::cout << " [INFO] Arguments are = " << py::str(args) << std::endl;
    int    n = py::cast<int>(args[0]);
    double x = py::cast<double>(args[1]);
    double y = py::cast<double>(args[2]);

    py::dict d{ "n"_a =  n, "2x"_a = 2.0 * x, "addxy"_a = (x + y)};
    d["m"] = 5.0 * x + 4.5 * y;
    py::print(" [INFO] Dictionary d = {d} ", "d"_a = d);
    return d;
}

Registration:

m.def("computeFormula", &computeFormula);

Function showDictionary

// Function that takes a Python dictionary as argument
void showDictionary(py::dict map)
{
    std::cout << " ====== Show dictonary =========== " << std::endl;
    for(auto const& item : map)
        std::cout << " key = "
                  << item.first << " ; "
                  << item.second
                  << std::endl;
}

Registration:

m.def("showDictionary",
      &showDictionary,
      "Display dictionary in console" );

Function pyFuncArgsKwargs
- Sample function that takes a python tuple (py::args) and named arguments py::kwargs.

// Sample Python function taking a tuple and dictionary as argument
void pyFuncArgsKwargs(py::args args, py::kwargs kwargs)
{
    double x = py::cast<double>(args[0]);
    int    n = py::cast<double>(args[1]);
    bool   verbose = false;

    py::print(" [TRACE]   args = ", args);
    py::print(" [TRACE] kwargs = ", kwargs);

    std::cout << std::boolalpha;

    if(kwargs.contains("verbose"))
    {
        verbose = py::cast<bool>(kwargs["verbose"]);
    }
    std::cout << " [INFO] verbose option = "
              << verbose << std::endl;

    std::cout << " sqrt(x) = " << std::sqrt(x)
              << " ; 2 * n = " << 2 * n
              << std::endl;
}

Registration:

m.def("dictionaryArgument", &dictionaryArgument);

Function: dictionaryArgument

// Sample function that takes a Python dictionary returning a
// Python tuple as argument.
py::tuple
dictionaryArgument(py::dict dict)
{
    std::cout << " >> Dictionary = " << py::str(dict) << std::endl;
    auto xx = dict["x"];
    auto yy = dict["y"];
    double x = py::cast<double>(xx);
    double y = py::cast<double>(yy);
    std::string name = py::cast<std::string>(dict["name"]);
    double z = std::sqrt(x * x + y * y);

    // Return a tuple on python side
    py::tuple tpl = py::make_tuple(x, y, z, name);
    py::print(" [INFO] Tuple = {} ", tpl);
    return tpl;
}

Registration:

m.def("pyFuncArgsKwargs",
      &pyFuncArgsKwargs,
      "Sample Python function taking a tuple");

Function: callPyObjectMethod
- This funtion takes any Python object as argument and prints its attributed .name and call its method named “.call”. If the object does not have those fields and methods, a runtime error will happen.

// Call Python Objec's Method (Duck typing)
void callPyObjectMethod(py::object obj, double k)
{
    // Print objec's name attribute
    py::print(" [TRACE] obj.name = ", obj.attr("name"));

    std::string name = py::cast<std::string>(obj.attr("name"));
    std::cout << " [COUT] object.name = " << name << std::endl;

    // Invoke object's method .call
    auto call = obj.attr("call");
    py::print(" [TRACE] obj.call(6.0) = ", call(6.0));
    double x = py::cast<double>(call(k));
    py::print(" [TRACE] 100.0 * objct.call(x) = ", 100.0 * x);
}

Registration:

m.def("callPyObjectMethod", &callPyObjectMethod);

Module Registration of Functions and classes

PYBIND11_MODULE(SampleModule, m) {
    // optional module docstring
    m.doc() = "Sample Python built with C++ CeePlusPlus ";

    //----------- Register C++ Functions --------//
    //

    // Register an ordinary C++ function
    m.def("add",                                 // Function Name in Python side
          &add,                                  // Function Pointer
          "A function which adds two numbers",   // Docstring
          py::arg("x"),                          // Name of argument 1
          py::arg("y") = 10);                    // Name of argument 2 with default value

    m.def("cppLambda"
          ,[](double x, double y){ return 3.0 * x + y;}
          ,"A C++ lambda object or functor"
          //,py::arg("x"), py::args("y") = 15
    );

    m.def("vectorCumulativeSum", &vectorCumulativeSum);

    m.def("tabulate1", &tabulate1);

 ... ...  ... ...  ... ...  ... ...  ... ... 
 }

Compile and Running

Compile

The project can be compiled from any IDE that supports CMake or from command line either by invoking CMake or a C++ compiler manually. All its need to build from an IDE is to open the CMakeList.txt.

Clone Repository:

$ cd /tmp 
$ git clone https://github.com/caiorss/example-project-pybind11 pybind11
$ cd pybind11 

$ tree .
.
├── bin
│   └── test-script.py
├── cashflow.cpp
├── CMakeLists.txt
└── SampleModule.cpp

1 directory, 4 files

Build from QTCreator IDE:

Just open the file CMakeLists.txt and click at the hammer button or open this file from command line. Similar procedure can be used with other IDEs. After compiled, the Python modules are copied to directory ./bin.

$ qtcreator CMakeLists.txt &

Build from command line:

Set configuration. Note: Debug build type has debugging symbols enabled and optmization disabled.

$ cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Debug

-- Conan: Automatic detection of conan settings from cmake
-- Conan: Settings= -s;build_type=Debug;-s;compiler=gcc;-s;compiler.version=7;-s;compiler.libcxx=libstdc++11
 ...  ...  ...  ...  ...  ...  ...  ...  ...  ... 

-- Build files have been written to: /tmp/pybind11/build

Compile:

$ cmake --build build --target

Scanning dependencies of target SampleModule [ 25%] Building CXX
object CMakeFiles/SampleModule.dir/SampleModule.cpp.o [ 50%] Linking
CXX shared module lib/SampleModule.cpython-37m-x86_64-linux-gnu.so [
50%] Built target SampleModule

... ... ... ... ... ... ... ... ... ...

The Cmake function copy_after_build copies the python native modules (shared libraries *.so files on Linux or *.pyd on Windows) to the directory ./bin

# Module files 
$ ls bin/
cashflow.cpython-37m-x86_64-linux-gnu.so*      test-script.py
SampleModule.cpython-37m-x86_64-linux-gnu.so*

$ file bin/SampleModule.cpython-37m-x86_64-linux-gnu.so 
bin/SampleModule.cpython-37m-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, with debug_info, not stripped

Running

Enter at directory where is the Python native module SampleModule, in this case, the file: SampleModule.cpython-37m-x86_64-linux-gnu.so

$ cd bin/

$ ls
cashflow.cpython-37m-x86_64-linux-gnu.so*      test-script.py
SampleModule.cpython-37m-x86_64-linux-gnu.so*

$ python3
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

Import SampleModule from REPL:

>>> import SampleModule as t

Get module help:

>>> help(t)

Help on module SampleModule:

NAME
    SampleModule - Sample Python built with C++ CeePlusPlus

CLASSES
    pybind11_builtins.pybind11_object(builtins.object)
        LinearFunctor
        MathErrorCode
 ....      ....      ....      ....      ....     


FUNCTIONS
    add(...) method of builtins.PyCapsule instance
        add(x: int, y: int=10) -> int

        A function which adds two numbers

    callPyObjectMethod(...) method of builtins.PyCapsule instance
        callPyObjectMethod(arg0: object, arg1: float) -> None

    computeFormula(...) method of builtins.PyCapsule instance
        computeFormula(*args) -> dict

    cppLambda(...) method of builtins.PyCapsule instance
        cppLambda(arg0: float, arg1: float) -> float

        A C++ lambda object or functor

    dictionaryArgument(...) method of builtins.PyCapsule instance

 ... ...  ... ...  ... ...  ... ...  ... ...  ... ...

Test function and classes without knowledge of Python API

Function add

>>> help(t.add)

add(...) method of builtins.PyCapsule instance
    add(x: int, y: int=10) -> int

    A function which adds two numbers


>>> t.add(20)
30
>>> t.add(100)
110
>>> t.add(15, 25)
40

Function vectorCumulativeSum

>>> help(t.vectorCumulativeSum)

vectorCumulativeSum(...) method of builtins.PyCapsule instance
    vectorCumulativeSum(arg0: List[float]) -> List[float]

>>> t.vectorCumulativeSum([1, 10, 5, -8, 25, 10])
[1.0, 11.0, 16.0, 8.0, 33.0, 43.0]

Function tabulate1

>>> t.tabulate1(0, 5, 1, lambda x: x * x - 5 * x  + 8)
   0.00000   8.00000
   1.00000   4.00000
   2.00000   2.00000
   3.00000   2.00000
   4.00000   4.00000
   5.00000   8.00000

>>> import math
>>> t.tabulate1(0, 5, 1, lambda x: math.exp(x))
   0.00000   1.00000
   1.00000   2.71828
   2.00000   7.38906
   3.00000  20.08554
   4.00000  54.59815
   5.00000 148.41316

>>> t.tabulate1(0, -5, 1, lambda x: x * x - 5 * x  + 8)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError:  Error: Invalid domain 
 Expected: start < stop && step > 0

Enum MathError

>>> status = t.MathErrorCode.OK

>>> status == t.MathErrorCode.OK 
True

>>> status = t.MathErrorCode.OVERFLOW

>>> status == t.MathErrorCode.OK 
False

>>> status == t.MathErrorCode.OVERFLOW
True

>>> int(t.MathErrorCode.OVERFLOW)
1

>>> int(t.MathErrorCode.NOT_A_NUMBER)
8

Class LinearFunctor

>>> lfun = t.LinearFunctor(10, 5)

>>> lfun
 LinearFunction: y(x) = A * x + B
 => A = 10 ; B = 5

>>> lfun.show()
 LinearFunction: y(x) = A * x + B
 => A = 10.00000 ; B = 5.00000
>>> 

>>> lfun(4)
45.0

>>> lfun(5)
55.0

>>> lfun(10)
105.0
>>> 

// Get coefficients 
//-----------------------
>>> lfun.A
10.0

>>> lfun.B
5.0

>>> lfun.GetA()
10.0

>>> lfun.GetB()
5.0
>>> 

// Change coefficients
//-------------------
>>> lfun.SetA(-5)

>>> lfun.SetB(15)

>>> lfun
 LinearFunction: y(x) = A * x + B
 => A = -5 ; B = 15

>>> lfun.call(5.64)
-13.2

>>> lfun(5.64)
-13.2
>>>

Test function and classes with knowledge of Python API

Function: computeFormula

>>> result = t.computeFormula(100, 5.67, 9.71)
 [INFO] Number of args = {s1} ; args = {s2}
 [INFO] Arguments are = (100, 5.67, 9.71)
 [INFO] Dictionary d = {d} 

>>> result
{'n': 100, '2x': 11.34, 'addxy': 15.38, 'm': 72.04500000000002}

>>> result["n"]
100

>>> result["2x"]
11.34

Function: showDictionary

>>> t.showDictionary({'x': 10.34, 'y': "numerical electronic integrator", "name": "ARM-CPU"})
 ====== Show dictonary =========== 
 key = x ; 10.34
 key = y ; numerical electronic integrator
 key = name ; ARM-CPU
>>> 

>>> t.showDictionary(100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: showDictionary(): incompatible function arguments. The following argument types are supported:
    1. (arg0: dict) -> None

Invoked with: 100

Function: pyFuncArgsKwargs

>>> t.pyFuncArgsKwargs(2.7824, 200)
 [TRACE]   args =  (2.7824, 200)
 [TRACE] kwargs =  {}
 [INFO] verbose option = false
 sqrt(x) = 1.66805 ; 2 * n = 400

>>> t.pyFuncArgsKwargs(2.7824, 200, verbose = True)
 [TRACE]   args =  (2.7824, 200)
 [TRACE] kwargs =  {'verbose': True}
 [INFO] verbose option = true
 sqrt(x) = 1.66805 ; 2 * n = 400

>>> t.pyFuncArgsKwargs(2.7824, 200, verbose = False)
 [TRACE]   args =  (2.7824, 200)
 [TRACE] kwargs =  {'verbose': False}
 [INFO] verbose option = false
 sqrt(x) = 1.66805 ; 2 * n = 400


>>> t.pyFuncArgsKwargs(2.7824, 200, verbose = "aaa")
 [TRACE]   args =  (2.7824, 200)
 [TRACE] kwargs =  {'verbose': 'aaa'}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type 'bool'

Function: dictionaryArgument

>>> result = t.dictionaryArgument({'x': 10.5, 'y': 5.6, 'name': "DUMMY"});
 >> Dictionary = {'x': 10.5, 'y': 5.6, 'name': 'DUMMY'}
 [INFO] Tuple = {}  (10.5, 5.6, 11.899999999999999, 'DUMMY')

>>> result
(10.5, 5.6, 11.899999999999999, 'DUMMY')

>>> result[0]
10.5

>>> result[1]
5.6

>>> result[2]
11.899999999999999
>>>

Function: callPyObjectMethod

class Dummy:
  def __init__(self, name):
      self.name = name

  def call(self, x):
      print(" I was called by C++.")
      return 4.5 * x + 10

>>> obj = Dummy("A dummy object")

>>> t.callPyObjectMethod(obj, 4.5)
 [TRACE] obj.name =  A dummy object
 [COUT] object.name = A dummy object
 I was called by C++.
 [TRACE] obj.call(6.0) =  37.0
 I was called by C++.
 [TRACE] 100.0 * objct.call(x) =  3025.0


>>> t.callPyObjectMethod(obj, 10.0)
 [TRACE] obj.name =  A dummy object
 [COUT] object.name = A dummy object
 I was called by C++.
 [TRACE] obj.call(6.0) =  37.0
 I was called by C++.
 [TRACE] 100.0 * objct.call(x) =  5500.0

R Language (RLang) Native Extension API

R Raw C API

Overview

R Language C data structures

Reference: http://adv-r.had.co.nz/C-interface.html

Data Structure	Description
Vector
REALSXP	Numeric Vector
INTSXP	Integer Vector
LGSXP	Logical Vector
STRSXP	Character Vector
CPLSXP	Complex Vectors

VECSXP	List

Null
NILSXP	Null

Misc
CLOSXP	Function (closure)
ENVSXP	Environment
LISTSXP	Pair list
DOTSXP
SYMSXP	names/symbols

Internal Objects	Note: Only used by C functions, not R functions.
LANGSXP	Language constructs
CHARSXP	-
PROMSXP	-
EXPRSXP	Expressions

Fuctions for coercing Scalars:

asLogical(xs): LGSXP -> int
asInteger(xs): INTSXP -> int
asReal(xs): REALSXP -> double
CHAR(asChar(x)): STRSXP -> const char*

R Types:

All numbers are float point (double 64 bits)
Scalar data are vectors, a single number is vector of size one.

Useful References:

R Internals - C API Documentation
R Native API Cheat Sheet
Foreign function | R Documentation
The Need for Speed Part 1: Building an R Package with Fortran (or C) | Strange Attractors
R’s C interface · Advanced R.
Use of C++ in Packages - The R Blog
Three ways to call C/C++ from R | R-bloggers
Integrate C Language Functions into R With an R Package | NCEAS

Install R language and development tools

Install R development libraries with Anaconda Python Distribution

# Install R language, interpreter 
$ conda install r-base   

# Install R development libraries 
$ conda install r-devtools -c r

Locate where the headers:

$ find ~/opt/anaconda3/ -name "Rin*.h"
/home/dummy/opt/anaconda3/lib/R/include/Rinterface.h
/home/dummy/opt/anaconda3/lib/R/include/Rinternals.h
/home/dummy/opt/anaconda3/pkgs/r-base-3.5.1-h1e0a451_2/lib/R/include/Rinterface.h
/home/dummy/opt/anaconda3/pkgs/r-base-3.5.1-h1e0a451_2/lib/R/include/Rinternals.h


$ ls ~/opt/anaconda3/lib/R/include/
R_ext/     Rdefines.h   R.h           Rinternals.h  Rversion.h
Rconfig.h  Rembedded.h  Rinterface.h  Rmath.h       S.h

Example using R native C API

Source

File:

rnative.cpp

Compilation:

Manual compilation

$ clang++ rnative.cpp -o rnative.so -std=c++1z -g -O0 -Wall -shared -fPIC -I/home/archbox/opt/anaconda3/lib/R/include/

Automatic with R command line tool.

$ R CMD SHLIB rnative.cpp

Source Code

Headers:

#include <iostream>
#include <iomanip>
#include <cmath>
#include <vector> 

#include <R.h>
#include <Rinternals.h>
#include <Rmath.h>

Global object for logging when the library is loaded or unloaded.

struct DLLIntializer{
     DLLIntializer()
     {
         std::cerr << "[TRACE] - DLL loaded into process OK." << std::endl;
     }
     ~DLLIntializer()
     {
         std::cerr << "[TRACE] - DLL unloaded from process OK." << std::endl;
     }
};

DLLIntializer init_hook{};

Function: basicFunction

Computes the 10.0 * x + 4.0 * y where x and y are scalars (vectors of size one)
Note 1: Every argument of a R-function is a SEXP (S-expression structure) that can be a vector, scalar, matrix, R function and so on.
Note 2: Every scalar in R is a vector of size 1.

extern "C"
SEXP basicFunction(SEXP x, SEXP y)
{
    std::cerr << " [TRACE] Calling R from C++" << std::endl;
    // Allocate a vector of size 1 of type real (double)
    // Remember: in R, a scalar is a vector of size one  
    SEXP out = PROTECT(allocVector(REALSXP, 1));
    // asReal(x) => Interprets the first element of the
    // vector x as a double 
    REAL(out)[0] = 10.0 * asReal(x) + 4.0 * asReal(y);
    //     UNPROTECT(< Number of allocations> )
    // OR: UNPROTECT(< Number of protects> )
    UNPROTECT(1);	
    return out;
}

Function: GetVersion

Returns library verion.

extern "C"
SEXP GetVersion()
{
    SEXP version;
    // Allocate 1 string 
    PROTECT(version = allocVector(STRSXP, 1));
    SET_STRING_ELT(version, 0, ::mkChar("version 0.1"));
    UNPROTECT(1);
    return version;
}

Function: Metric

Display vector statistics

extern "C"
auto metrics(SEXP xs) -> SEXP 
{
   size_t size = ::length(xs);
   double* vec =  REAL(xs);
   double sumsq  = 0.0;
   double sum    = 0.0;
   double x;
   for(auto i = 0; i < size; i++){
       x = vec[i];
       sumsq = sumsq + x * x;
       sum   = sum + x;
   }
   std::cout << std::setprecision(4) << std::fixed;
   std::cout << std::endl;
   std::cout << " =>    Mean = " << sum / size << std::endl;
   std::cout << " =>    Norm = " << std::sqrt(sumsq) << std::endl;
   std::cout << " =>    Size = " << size << std::endl;
   // If the function returns nothing, it should return R_NilValue 
   return R_NilValue;
}

Function: linearcomb

Return a linear combination between vectors.

/** Computes the equation: 
  * For each i = 0, 1, 2, ... N - 1
  *   zs_vector[i] = a * xs_vector[i] + b * ys_vector[i] + c 
  *
  * Returns: Vector zs_vector[i]
  */
extern "C"
SEXP linearcomb(SEXP xs_vector, SEXP ys_vector, SEXP a, SEXP b, SEXP c)
{
     size_t size = ::length(xs_vector);
     // Allocate output vector of same size 
     SEXP output = PROTECT(allocVector(REALSXP, size));
     double *pout, *pxs, *pys;
     pout = REAL(output);
     pxs  = REAL(xs_vector);
     pys  = REAL(ys_vector);
     for(size_t i = 0; i < size; i++)
             pout[i] = asReal(a) * pxs[i] + asReal(b) * pys[i] + asReal(c);
     UNPROTECT(1);
     return output;
}

Load into R REPL

Load shared library:

> dyn.load("rnative.so")
[TRACE] - DLL loaded into process OK.

Call function basicFunction

> .Call("basicFunction", 10, 5)
 [TRACE] Calling R from C++
[1] 120

Call function metrics:

> .Call("metrics", 10)
 => Average = 10.0000
 =>    Norm = 10.0000
 =>    Size = 1
NULL

> .Call("metrics", c(4.5, 8.0, 6, -10.0))
 => Average = 2.1250
 =>    Norm = 14.8408
 =>    Size = 4
NULL

Call function linearcomb:

Test function linearcomb which perform linear combination between two vectors:

> .Call("linearcomb", c(1, 2, 3, 4), c(4, 5, 6, 9), 10, 2, 0)
[1] 18 30 42 58

> .Call("linearcomb", c(1, 2, 3, 4), c(4, 5, 6, 9), 10, 2, 5)
[1] 23 35 47 63

> .Call("linearcomb", 4, 6, 10, 1, 2)
[1] 48

> .Call("linearcomb", 4, 6, 10, 5, 2)
[1] 72

End of the interative session:

> quit()
Save workspace image? [y/n/c]: n
[TRACE] - DLL unloaded from process OK.

Those functions need wrappers for making usage easier:

File: rnative.R

dyn.load("rnative.so")

basicFunction <- function(x, y){  .Call("basicFunction", x, y) }

getVersion <- function(){ .Call("GetVersion")}

metrics <- function(xs){ .Call("metrics", xs) }

# Compute: a * xs + b * ys + c
linearcomb <- function(xs, ys, a, b, c){
    .Call("linearcomb", xs, ys, a, b, c)
}


testPackage <- function(){
    print(sprintf("basicFunction(4, 5) = %.3f", basicFunction(4, 5)))
    print(sprintf("basicFunction(10, 6) = %.3f", basicFunction(10, 6)))

    metrics(8)
    metrics(c(3.4, 9.0, 10.0, -15.75, 20.65))
}

This script can be loaded into the REPL with:

> source("rnative.R")
[TRACE] - DLL loaded into process OK.

> getVersion()
[1] "version 0.1"

> metrics(c(4, 5, 10.5, 8.65, 5))

 =>    Mean = 6.6300
 =>    Norm = 15.8453
 =>    Size = 5
NULL
> 

> quit()
Save workspace image? [y/n/c]: n
[TRACE] - DLL unloaded from process OK.

Rcpp Library for easier C++ Integration

Overview

Rcpp is a C++ library that makes easier to write native R functions in C++ and also to create bindings to existing C++ libraries.

Rcpp

Official Web Sites:

http://www.rcpp.org/
http://dirk.eddelbuettel.com/code/rcpp

Doxygen - C++ API Docs:

http://dirk.eddelbuettel.com/code/rcpp/html/index.html

Example - Native R library in C++ with Rcpp

Source Files

Source Files

File: rcxx.cpp
File: CMakeLists.txt

Compilation

Manual:

$ clang++ rcxx.cpp -o rcxx.so -std=c++1z -g -O0 -Wall -shared -fPIC \
   -I/home/archbox/opt/anaconda3/lib/R/include/ \
   -I/home/archbox/opt/anaconda3/lib/R/library/Rcpp/include/

Build with CMake:

$ cmake -Bbuild -H. 

$ cmake --build build --target 
Scanning dependencies of target rcxx
[ 50%] Building CXX object CMakeFiles/rcxx.dir/rcxx.cpp.o
[100%] Linking CXX shared library rcxx.so
[100%] Built target r

$ ls build/
CMakeFiles/  CMakeCache.txt  cmake_install.cmake  Makefile  rcxx.so*

$ cp build/rcxx.so .

File: rcxx.cpp

Headers:

#include <iostream>
#include <cmath>
#include <iomanip>
#include <algorithm>
#include <string>
#include <map>

#include <Rcpp.h>

Global object for logging when the DLL is loaded or unloaded.

struct DLLIntializer{
    DLLIntializer()
    {
        std::cerr << "[TRACE] - DLL loaded into process OK." << std::endl;
    }
    ~DLLIntializer()
    {
        std::cerr << "[TRACE] - DLL unloaded from process OK." << std::endl;
    }
};

// Global object allocated in process' data segment for
// DLL logging during loading and unloading. 
DLLIntializer init_hook{};

Function libraryVersion

Returns the library version as string.

RcppExport
auto libraryVersion() -> SEXP
{
    return Rcpp::wrap("version 1.0 - alpha");
}

Function linearcomb

Performs the transformation:
ys[i] = a * xs[i] + b for every i = 0, 1, 2 … N - 1

/** Compute equation: 
 * for_each(i) => ys[i] = a * xs[i] + b 
 * return ys 
 */
RcppExport
SEXP linearcomb(SEXP a, SEXP b, SEXP xs)
{
    Rcpp::NumericVector vx(xs);    
    double aa = Rcpp::as<double>(a);
    double bb = Rcpp::as<double>(b);
    Rcpp::NumericVector out(vx.size());	
    size_t n = vx.size();
    // std::cerr << " [TRACE] Size of xs = " << n << std::endl;
    for(auto i = 0; i < vx.size(); i++)
       out[i] = aa * vx[i] + bb;
    return out;
}

Function: computeStatistics

RcppExport
auto computeStatistics(SEXP xs) -> SEXP
{
    Rcpp::NumericVector vx{xs};
    double sum = 0.0;
    double sumsq = 0.0;

    std::for_each(vx.begin(), vx.end(), [&](double x)
    {
        sum   += x;
        sumsq += x * x;
    });

    auto result = std::map<std::string, double>
    {
        {"sum",   sum},
        {"sumsq", sumsq},
        {"mean",  sum / vx.size()},
        {"norm",  std::sqrt(sumsq)}
    };
    return Rcpp::wrap(result);
}

Function: computeStatistics2:

RcppExport
auto computeStatistics2(SEXP xs) -> SEXP
{
    Rcpp::NumericVector vx{xs};
    double sum = 0.0;
    double sumsq = 0.0;

    std::for_each(vx.begin(), vx.end(), [&](double x)
    {
        sum   += x;
        sumsq += x * x;
    });
    return Rcpp::List::create(
                    Rcpp::Named("sum", sum)
                   ,Rcpp::Named("sumsq", sumsq)
                   ,Rcpp::Named("mean", sum / vx.size())
                   ,Rcpp::Named("norm", std::sqrt(sumsq))
                );
}

Function tabulate:

RcppExport
SEXP tabulate(SEXP mfunc, SEXP mvec)
{
    Rcpp::Function     func = mfunc;
    Rcpp::NumericVector vec = mvec;
    std::cout << std::setprecision(4) << std::fixed;
    for(auto const& x: vec)
        std::cout << std::setw(10) << x
                  << std::setw(10) << Rcpp::as<double>(func(x))
                  << std::endl;
    // Return nothing on R side
    return R_NilValue;
}

Function: vectorOperations

This function performs arithmetic operation on two input R numeric vectors.

/* Compute:
 *
 *  FOR i = 0 TO N DO:
 *    out[i] = sqrt(va[i] * 4.0 + vb[i] * 5.0
 *  RETURN ouy
 *
 * Note: The vectorized operation are not efficient
 * Rcpp::sqrt(4.0 * va + 5.0 * vb) is equivaent to:
 * Rcpp::sqrt(operatior*(4.0, va) + operator*(5.0, vb))
 * Rcpp::sqrt(operator+(operatior*(4.0, va), operator*(5.0, vb)))
 *
 * Every operator function call is 1 loop, so, the total number
 * of passes over the input data is equal to:
 *  =>  1 (SQRT Call) + 3 (Number of operators) = 4
 *
 * This function requires 4 for-loops over the data,
 * thus a single for-loop is more efficient.
 */
RcppExport
SEXP vectorOperations(SEXP sa, SEXP sb)
{
    using Vector = Rcpp::NumericVector;
    Vector va = sa;
    Vector vb = sb;
    Vector result = Rcpp::sqrt(4.0 * va + 5.0 * vb);
    return result;
}

Function: ShowNormalRandoms

This code demonstrates how to call R functions such as rnorm from C++ side. It takes as parameter the number of normally distributed random numbers that will be generated with mean 20.0 and standard deviation 5.0

RcppExport
SEXP ShowNormalRandoms(SEXP s_size)
{
    size_t n = Rcpp::as<size_t>(s_size);
    std::cout << " [INFO] size = " << n << std::endl;

    // Get R function rnorm for generating N normally distributed
    // random numbers
    Rcpp::Function rnorm("rnorm");

    // Call R built-in function rnorm
    // that generates n normally distributed random numbers with
    // mean 20.0 and standard deviation std = 5.0
    //---------------------------------------------
    // NOTE: Every R fuction returns an S-expression struct
    SEXP sexp = rnorm(Rcpp::Named("n", n),
                      Rcpp::Named("sd", 5.0),
                      Rcpp::Named("mean", 20.0)
                      );
    // Convert s-expression to numeric vector
    Rcpp::NumericVector result{sexp};

    std::cout << std::setprecision(5) << std::fixed;
    size_t k = 0;
    std::for_each(result.begin(), result.end(),
                  [&](auto x) -> void
                   {
                    std::cout << std::setw(10) << k++
                              << std::setw(10) << x
                              << std::endl;
                   });
    // std::for_each(result.begin())
    return R_NilValue;
}

Loading into R REPL

Interactive Session:

> library(Rcpp)

> # Load shared library into current REPL 
> dyn.load("rcxx.so")
[TRACE] - DLL loaded into process OK.

Call function libraryVersion:

> .Call("libraryVersion")
[1] "version 1.0 - alpha"
> 

# Better 
> .Call("libraryVersion", PACKAGE = "rcxx")
[1] "version 1.0 - alpha"

> .Call("libraryVersion", PACKAGE = "rm")
Error in .Call("libraryVersion", PACKAGE = "rm") : 
  "libraryVersion" not available for .Call() for package "rm"

Call function linearcomb

> # Call function from shared library 
> .Call("linearcomb", 10.0, 4.0, c(4.5, 9.8, 5.6, 10.0), PACKAGE = "rcxx")
[1]  49 102  60 104

> .Call("linearcomb", 10.0, 0, c(4.5, 9.8, 5.6, 10.0), PACKAGE = "rcxx")
[1]  45  98  56 100

Call function computeStatistics

> .Call("computeStatistics", c(4.5, 3.5, 9.81, 4.6))
     mean      norm       sum     sumsq 
  5.60250  12.24321  22.41000 149.89610 
> 

> out = .Call("computeStatistics", c(4.5, 3.5, 9.81, 4.6), PACKAGE = "rcxx")
> out
     mean      norm       sum     sumsq 
  5.60250  12.24321  22.41000 149.89610 

> out["mean"]
  mean 
5.6025 

> out["norm"]
    norm 
12.24321 

> out["sum"]
  sum 
22.41

Call function computeStatistics2

> res = .Call("computeStatistics2", c(4.5, 3.5, 9.81, 4.6), PACAKGE = "rcxx")

> res
$sum
[1] 22.41

$sumsq
[1] 149.8961

$mean
[1] 5.6025

$norm
[1] 12.24321

> res$sum
[1] 22.41

> res$mean
[1] 5.6025
> 
> 100 * res$mean
[1] 560.25

Call function tabulate

> .Call("tabulate", sqrt, c(4.5, 3.5, 9.81, 4.6))
    4.5000    2.1213
    3.5000    1.8708
    9.8100    3.1321
    4.6000    2.1448
NULL

> .Call("tabulate", function(x){ 10 * x + 5 }, c(4.5, 3.5, 9.81, 4.6))
    4.5000   50.0000
    3.5000   40.0000
    9.8100  103.1000
    4.6000   51.0000
NULL
>

Call function vectorOperations

> .Call("vectorOperations", c(4.0, 3.0, 2.0, 6.0), c(10.0, 5, 8, 4))
[1] 8.124038 6.082763 6.928203 6.633250

> xs = .Call("vectorOperations", c(4.0, 3.0, 2.0, 6.0), c(10.0, 5, 8, 4))

> xs
[1] 8.124038 6.082763 6.928203 6.633250

> xs * xs
[1] 66 37 48 44

> sqrt(4.0 * 4.0 + 5.0 * 10.0)
[1] 8.124038
>

Call function ShowNormalRandoms

This function invokes the R built-in function rnorm from the C++ side and displays the result on the screen.

> .Call("ShowNormalRandoms", 5, PACKAGE = "rcxx")
 [INFO] size = 5
         0  15.61839
         1  25.08418
         2  22.76518
         3  17.96007
         4  17.60298
NULL

> .Call("ShowNormalRandoms", 10, PACKAGE = "rcxx")
 [INFO] size = 10
         0  17.92591
         1  16.47821
         2  24.35774
         3  19.33816
         4  22.70113
         5  17.37169
         6  14.92140
         7  11.94084
         8  14.29367
         9  12.16308
NULL

References

Unofficial Rcpp API Documentation - The Coatless Professor
Cran - RCPP - Rcpp: Seamless R and C++ Integration
- https://cran.r-project.org/web/packages/Rcpp/index.html
Roman François. Integrating R with C++: Rcpp, RInside and RProtoBuf
- https://www.slideshare.net/romainfrancois/integrating-r-with-c-rcpp-rinside-and-rprotobuf
Roman François. R / C++
- https://www.slideshare.net/romainfrancois/rc-67163677
Hyun Min Kang - Biostatistics 615/815 Lecture 12 - Interfacing C++ and R
- https://genome.sph.umich.edu/w/images/4/47/Bios615-fa12-lec12-presentation.pdf

Rcpp namespace
Convenience Functions in	Description
Fundamental Transcendatal Function
Rcpp::exp	Exponential e^x
Rcpp::log	Natural logarithm base e = 2.718282
Rcpp::log1p	Natural logarithm, however, log(1 + x), where x is a small number
Rcpp::log10	Logarith base 10
Rcpp::sqrt	Square root


Statistcs
Rcpp::sd	Standard Deviation
Rcpp::mean	Mean, or average

Rcpp::sum	Sum of input values
Rcpp::cumsum	Cummulative sum
Rcpp::diff
Rcpp::pmin	Return paralallel maxima of the input values
Rcpp::pmax	Return paralallel minima of the input values

..... and more …

RCPP Class	R type typeof(<OBJ>)
Vector
Rcpp::IntegerVector	integer
Rcpp::NumericVector	numeric
Rcpp::LogicalVector	logical
Rcpp::CharacterVector	character
Rcpp::RawVector	raw…
Rcpp::ComplexVector	complex…

Matrix
Rcpp::IntegerMatrix
Rcpp::NumericMatrix
Rcpp::LogicalMatrix
Rcpp::RawMatrix
Rcpp::ComplexMatrix

Other Types
Rcpp::Environment	environment
Rcpp::Function	function (closure)
Rcpp::XPtr	externalptr
Rcpp::Language	Language

Files

DLL-Binary-Components-SharedLibraries.org

Latest commit

History

DLL-Binary-Components-SharedLibraries.org

File metadata and controls

Shared Libraries - Binary Components

C-Interoperability and Shared Libraries - Binary Components

Overview

Design Guidelines for C-Interfaces

Cross platform Shared Library, C-Interfaces and Language Interoperability

Overview

Header macros

Namespace Linalg

C-interface of Namespace Linalg

Non-polymorphic Class - SampleClass

C-interface for SampleClass

Polymorphic class InterfaceClass

DLL startup function DLLMain

DLL entry point for run32dll.exe

C++ Client program - client1.cpp

Load DLL in Python3 REPL

Python 3 Client Code - Wrapper Module

CSharp Client Code - PInvoke

View symbols exported by the shared library

View exported symbols on Windows (dumpbin)

View exports symbols on Unix-like OSes (nm)

Plugins and Dynamically Loading Classes

Overview

API used for loading shared libraries dynamically

Example

Overview

File: psdk/interfaces.hpp

File: psdk/factory.hpp

File: psdk/loader.hpp

File: PluginA.cpp

File: main.cpp

Return string from a C++ function to Python Ctypes

Python Native Extension API

Overview

Python Raw C API

Module Code

Sources

File: setup.py

File: mymodule.cpp

Compiling and Running Native Module

Native Modules or C++ binding with Pybind11

Overview

Example

Overview

File: CMakeLists.txt

File: SampleModule.cpp

Full Source

Header and Namespaces

Bindings C++ Functions, enums and Classes

Functions with knowledge of Python API

Module Registration of Functions and classes

Compile and Running

Compile

Running

R Language (RLang) Native Extension API

R Raw C API

Overview

Install R language and development tools

Example using R native C API

Source

Load into R REPL

Rcpp Library for easier C++ Integration

Overview

Example - Native R library in C++ with Rcpp

Source Files

Loading into R REPL

References