You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**IncludeOS** is an includable, minimal [unikernel](https://en.wikipedia.org/wiki/Unikernel) operating system for C++ services running in the cloud and on real HW. Starting a program with `#include <os>` will literally include a tiny operating system into your service during link-time.
5
5
6
6
IncludeOS is free software, with "no warranties or restrictions of any kind".
[](https://join.slack.com/t/includeos/shared_invite/zt-5z7ts29z-_AX0kZNiUNE7eIMUP60GmQ)
11
-
12
8
**Note:***IncludeOS is under active development. The public API should not be considered stable.*
13
9
14
-
15
10
## <aname="features"></a> Key features
16
11
17
12
***Extreme memory footprint**: A minimal bootable 64-bit web server, including operating system components and anything needed from the C/C++ standard libraries is currently 2.5 MB.
18
13
***KVM, VirtualBox and VMWare support** with full virtualization, using [x86 hardware virtualization](https://en.wikipedia.org/wiki/X86_virtualization), available on most modern x86 CPUs. IncludeOS will run on any x86 hardware platform, even on a physical x86 computer, given appropriate drivers. Officially, we develop for- and test on [Linux KVM](http://www.linux-kvm.org/page/Main_Page), and VMWare [ESXi](https://www.vmware.com/products/esxi-and-esx.html)/[Fusion](https://www.vmware.com/products/fusion.html) which means that you can run your IncludeOS service on Linux, Microsoft Windows and macOS, as well as on cloud providers such as [Google Compute Engine](http://www.includeos.org/blog/2017/includeos-on-google-compute-engine.html), [OpenStack](https://www.openstack.org/) and VMWare [vcloud](https://www.vmware.com/products/vcloud-suite.html).
19
14
***Instant boot:** IncludeOS on Qemu/kvm boots in about 300ms but IBM Research has also integrated IncludeOS with [Solo5/uKVM](https://github.com/Solo5/solo5), providing boot times as low as 10 milliseconds.
20
15
***Modern C++ support**
21
-
* Full C++11/14/17 language support with [clang](http://clang.llvm.org)5 and later.
16
+
* Full C++11/14/17/20 language support with [clang](http://clang.llvm.org)18 and later.
22
17
* Standard C++ library (STL) [libc++](http://libcxx.llvm.org) from [LLVM](http://llvm.org/).
23
18
* Exceptions and stack unwinding (currently using [libgcc](https://gcc.gnu.org/onlinedocs/gccint/Libgcc.html)).
24
19
**Note:* Certain language features, such as threads and filestreams are currently missing backend support but is beeing worked on.
@@ -31,11 +26,10 @@ A longer list of features and limitations can be found on our [documentation sit
31
26
## Contents
32
27
33
28
-[Getting started](#getting_started)
34
-
-[Dependencies](#dependencies)
35
-
-[Hello world](#hello_world)
36
-
-[Getting started developing packages](#develop_pkg)
-[Building with IncludeOS in editable mode](#editing_includeos)
29
+
-[Dependencies](#dependencies)
30
+
-[Hello world](#hello_world)
31
+
-[Kernel development](#develop_kernel)
32
+
-[Running tests](#running_tests)
39
33
-[Contributing to IncludeOS](#contribute)
40
34
-[C++ Guidelines](#guideline)
41
35
-[Security contact](#security)
@@ -44,241 +38,95 @@ A longer list of features and limitations can be found on our [documentation sit
44
38
45
39
### <aname="dependencies"></a> Dependencies
46
40
47
-
For building IncludeOS services you will need:
48
-
49
-
*[The conan package manager](https://docs.conan.io/en/latest/installation.html) (1.13.1 or newer)
50
-
* cmake, make, nasm (x86/x86_64 only)
51
-
* clang, or alternatively gcc on linux. Prebuilt packages are available for clang 6.0 and gcc 7.3.
52
-
53
-
To boot VMs locally with our tooling you will also need:
54
-
55
-
* qemu
56
-
* python3 packages: psutil, jsonschema
57
-
58
-
The following command will configure conan to use our build profiles and remote repositories. (**Note:** this overwrites any existing conan configuration. Set `CONAN_USER_HOME` to create a separate conan home folder for testing.)
For building and booting IncludeOS services you will need [nix](https://nixos.org) and Linux. Nix will automatically download and set up the correct versions of all the required libraries and compilers.
63
42
64
-
**Note:** If you prefer to set up conan manually the full configuration can be found in the [conan_config](https://github.com/includeos/conan_config.git) repository. You can browse our prebuilt conan packages in [bintray.com/includeos](https://bintray.com/includeos).
43
+
To speed up local builds we also recommend configuring nix with [ccache support](https://nixos.wiki/wiki/CCache) but this is not a requirement. To use ccache, `--arg withCcache true`can be added to most `nix-build` and `nix-shell` commands shown below.
If you want to use a Vagrant box to explore IncludeOS and contribute to IncludeOS development, you can read the getting started with Vagrant. See [etc/vagrant.md](etc/vagrant.md)
45
+
IncludeOS can currently not be built on macOS or Windows.
86
46
87
47
### <ahref="hello_world"></a> Hello World
88
48
89
-
First select an appropriate [conan profile](https://docs.conan.io/en/latest/reference/profiles.html) for the target you want to boot on. `conan profile list` will show the profiles available, including the ones installed in the previous step. When developing for the machine you're currently on, Linux users can typically use `clang-6.0-linux-x86_64`, and MacOS users can use `clang-6.0-macos-x86_64`. You can also make your own.
49
+
A minimal IncludeOS "hello world" looks like a regular C++ program:
90
50
91
-
The following steps let you build and boot the IncludeOS hello world example.
You can use the [hello world repo](https://github.com/includeos/hello_world) as a starting point for developing your own IncludeOS services. For more advanced examples see the [examples repo](https://github.com/includeos/demo-examples) or the integration tests (under ./IncludeOS/test/\*/integration).
103
58
104
-
Once you're done `$ source deactivate.sh` will reset the environment to its previous state.
59
+
A full "Hello world" service with a working nix workflow is available in the [hello world repo](https://github.com/includeos/hello_world). The repository can also be used as a a starting point for developing your own IncludeOS service.
105
60
106
-
### <aname="develop_pkg"></a> Getting started developing packages
61
+
For more advanced service examples see the the integration tests (under ./IncludeOS/test/\*/integration).
107
62
108
-
If you are interested in editing/building our dependency packages on your own, you can checkout our repositories at [includeos/](https://github.com/includeos/). Some of our tools and libraries are listed below under [Tools and Libraries](#libs_tools). You can find the external dependency recipes at [includeos/conan](https://github.com/includeos/conan). Currently we build with `clang-6` and `gcc-7.3.0` compiler toolchains. It is expected that these are already installed on your system (see [Dependencies](#dependencies) for details).
63
+
### <aname="develop_kernel"></a> Kernel development
Prebuilt conan packages are uploaded to our [bintray repository](https://bintray.com/includeos/includeos).
113
-
114
-
We upload to two channels:
115
-
116
-
-`stable`: this channel has all the stable packages.
117
-
-`latest`: this channel will have the latest packages in development/test phase (including stable releases).
118
-
119
-
**Note:** We only guarantee that the **latest 10 packages** are kept in the `latest` channel. All `stable` packages will be kept in the stable channel unless proven unsafe. One suggested workaround is to copy packages into your own repository.
120
-
121
-
To set up our remote, we recommend following the steps listed in [Dependencies](#dependencies).
122
-
123
-
### Search
124
-
125
-
If you want to check if a package exists you can search for it with `conan search`. To list all the available packages on our remote `includeos`, you can use:
126
-
127
-
```text
128
-
$ conan search -r includeos
67
+
```bash
68
+
$ nix-build
129
69
```
130
70
131
-
This should list all the packages we have uploaded to the includeos repository.
132
-
133
-
To find all the stable versions uploaded for a particular package:
The profiles we are using for development can be found in the [includeos/conan_config](https://github.com/includeos/conan_config) repository under `conan_config/profiles/`.
141
-
142
-
The target profiles we have verified are the following:
These profiles should have prebuilt packages available and are tested in CI. If you create a custom profile (or use a different profile provided by us) the dependencies may have to be rebuilt locally.
71
+
This will build the toolchain and all IncludeOS kernel libraries.
150
72
151
-
## <aname="editing_includeos"></a> Building with IncludeOS in editable mode
73
+
Note that the first build will take some time to complete, as the IncludeOS toolchain is rebuilt from source code. This includes clang, llvm, libcxx, musl and so on. There is no nix binary cache available for these files at the moment. Subsequent builds will go much faster when the toolchain has been cached in the local nix-store.
152
74
153
-
If you are a kernel developer or are simple interested in fiddling with our kernel code, you can use the includeos-package in editable mode. When you rebuild the package will then automatically be updated so it can be used by other packages locally. This is useful when working with several interconnected components and you would like to test your changes across several libraries.
75
+
After making changes to the kernel, run `nix-build` again to get new binaries. If you are iterating on changes in one section of the kernel you can speed up the build significantly by using ccache. All `nix-build` and `nix-shell` commands in this section support the optional parameter `--arg withCcache true`.
154
76
155
-
You can read more about how editable mode works in the [Conan documentation](https://docs.conan.io/en/latest/developing_packages/editable_packages.html).
77
+
It's not always practical to rebuild the whole kernel during development. You can get a development shell with a preconfigured environment using `shell.nix`:
156
78
157
-
Below we have written down a few steps to get you started with editable packages and IncludeOS.
158
-
159
-
**Note:** Currently this is an experimental feature on conan version 1.13 and they have mentioned that for future releases the feature is subject to breaking changes.
160
-
161
-
Start by cloning the IncludeOS source code and create a `build` folder. You have to edit `etc/layout.txt` in the source code to point to the `build` folder you created, by updating the `build_dir` variable.
162
-
163
-
The layout will look similar to the example below. You only have to update `build_dir`.
164
-
165
-
```text
166
-
{% set simple=true%}
167
-
168
-
{% set build_dir='build' %}
169
-
170
-
{% if simple==false %}
171
-
{% set arch=settings.arch|string %}
172
-
{% set platform=options.platform|string %}
173
-
{% set build_dir=build_dir+'/'+arch+'/'+platform %}
174
-
{% endif %}
175
-
176
-
[bindirs]
177
-
{#This is so that we can find os.cmake after including conanbuildinfo.cmake #}
178
-
cmake
179
-
180
-
[includedirs]
181
-
{#This is to ensure the include directory in conanbuildinfo.cmake includes our API#}
182
-
api
183
-
184
-
[libdirs]
185
-
{#This is so that we can find our libraries #}
186
-
{{ build_dir }}/plugins
187
-
{{ build_dir }}/drivers
188
-
{{ build_dir }}/lib
189
-
{{ build_dir }}/platform
190
-
191
-
[resdirs]
192
-
{#This is so that we can find ldscript and search for drivers plugins etc#}
193
-
{{ build_dir }}
194
-
195
-
```
196
-
**Note:** in the non simple form it is possible to have multiple build folders from the same source which allows multiple architectures and configurations to be tested from the same source however the complexity increases.
197
-
198
-
You should now be able to set the package in editable mode. The following command will add the package as editable based on the specified layout. We inspect the package to get the version, as this has to match exactly.
199
-
200
-
```text
201
-
$ conan editable add . includeos/$(conan inspect -a version . | cut -d " " -f 2)@includeos/latest --layout=etc/layout.txt
79
+
```bash
80
+
$ nix-shell
202
81
```
203
82
204
-
The package is now in **editable mode** and any dependencies of IncludeOS will pick this IncludeOS package from your local cache.
83
+
Further instructions will be shown for optionally configuring VM networking or overriding the build path when starting the shell.
205
84
206
-
Here is an example on how it looks when its pulled into cache as editable:
85
+
By default th shell will also build the unikernel from `example.nix`. The example unikernel can be booted from within the shell:
207
86
208
-
```text
209
-
$ conan editable list
210
-
includeos/0.14.1-1052@includeos/latest
211
-
Path: ~/git/IncludeOS
212
-
Layout: ~/git/IncludeOS/etc/layout.txt
87
+
```bash
88
+
$ nix-shell
89
+
[...]
90
+
nix$ boot hello_includeos.elf.bin
213
91
```
214
92
215
-
We are now ready to build the package. Assuming the build-folder is called `build` under the includeos source directory the following is enough.
After making changes to the code you can rebuild the package with
224
-
225
-
```text
226
-
$ cd build && make
227
-
or
228
-
$ cmake build --build
229
-
```
230
-
231
-
Once you have made your changes and the code is **finalized** you should verify that the conan package still builds. Remove the editable and do a conan create on the package:
If you want to build a different unikernel than the example, this can be specified with the `--argstr unikernel [path]` parameter. This is primarily used for integration tests. For example, to build and run the stacktrace-test:
symtab or strtab is empty, indicating image may be stripped
104
+
[0] 0x000000000025dcd2 + 0x000: 0x25dcd2
105
+
[1] 0x000000000021097d + 0x000: 0x21097d
106
+
[2] 0x00000000002b370a + 0x000: 0x2b370a
107
+
[3] 0x0000000000210eea + 0x000: 0x210eea
108
+
We reached the end.
236
109
```
237
110
238
-
## Libraries and tools
111
+
To build and run the test VM as a single command:
239
112
240
-
We have moved the libraries and tools created by IncludeOS outside the includeos-repository. You can now find them all in their own repositories inside the IncludeOS organization.
241
-
242
-
To build the libraries and tools, see build instructions in each repository. Typically, the instructions will be in the form:
Vmrunner is a utility developed for booting IncludeOS binaries.
117
+
### <aname="running_tests"></a> Running tests
257
118
258
-
-[Mana](https://github.com/includeos/mana) -
259
-
Mana is a web application framework which is used to build a IncludeOS webserver.
260
-
We have an example named [acorn](https://github.com/includeos/demo-examples/tree/master/acorn) which demonstrates mana's potential.
119
+
You can run all the integration tests using the script `./test.sh`. The tests will run locally in the nix environment. We recommend manually verifying that all the tests pass locally before submitting a new PR to IncludeOS to save review time.
Diskbuilder is a tool used for building disks for IncludeOS.
268
-
269
-
-[NaCl](https://github.com/includeos/NaCl) -
270
-
NaCl is the configuration language tool we have tailored for IncludeOS to allow users to configure various network settings such as firewall rules, vlans, ip configurations etc.
121
+
Individual tests can be run with `nix-shell` directly. See `test.sh` for more details.
271
122
272
123
## <aname="contribute"></a> Contributing to IncludeOS
273
124
274
-
IncludeOS is being developed on GitHub. Create your own fork, send us a pull request, and [chat with us on Slack](https://join.slack.com/t/includeos/shared_invite/zt-5z7ts29z-_AX0kZNiUNE7eIMUP60GmQ). Please read the [Guidelines for Contributing to IncludeOS](http://includeos.readthedocs.io/en/latest/Contributing-to-IncludeOS.html).
125
+
IncludeOS is being developed on GitHub. Create your own fork and send us a pull request. Please read the [Guidelines for Contributing to IncludeOS](http://includeos.readthedocs.io/en/latest/Contributing-to-IncludeOS.html).
275
126
276
127
## <aname="guideline"></a> C++ Guidelines
277
128
278
129
We want to adhere as much as possible to the [ISO C++ Core Guidelines](https://github.com/isocpp/CppCoreGuidelines). When you find code in IncludeOS which doesn't adhere, please let us know in the [issue tracker](https://github.com/includeos/IncludeOS/issues) - or even better, fix it in your own fork and send us a [pull-request](https://github.com/includeos/IncludeOS/pulls).
279
130
280
-
[brew]: https://brew.sh/
281
-
[qemu]: https://www.qemu.org/
282
-
283
131
## <aname="security"></a> Security contact
284
132
If you discover a security issue in IncludeOS please avoid the public issue tracker. Instead send an email to security@includeos.org. For more information and encryption please refer to the [documentation](http://includeos.readthedocs.io/en/latest/Security.html).
0 commit comments