The configuration file used by ops
(such as one passed as a parameter,
e.g. ops --config myconfig.json
) specifies various options and attributes
of code execution, such as files to include, arguments, and environment
variables. This file follows the standard json format.
For a complete sample of a configuration, see our sample.
You can also enable config interpolation for things like secrets, hosts or other configuration that you would like to inject from your environment. This is helpful to reduce duplicate config:
For example:
secret=test ops_render_config=true ops run -c config.json
Will populate 'secret' with 'test':
{
"Env": {
"secret": "$secret"
}
}
The OPS_HOME environment variable points to the location of where ops will store build images and releases locally. By default it is set to ~/.ops . However there are some use-cases where you might want it set to something else like so:
OPS_HOME=/opt/ops
Args
is an array of arguments passed to your program to execute when running the image. Most programs will consider the program name as arg[0] but not all.
{
"Args": ["--report-on-fatalerror", "ex.js"]
}
DisableArgsCopy
, when set to true, disables the auto copy of files from host to image when present in args.
{
"Args": ["ex.js"],
"DisableArgsCopy": true
}
The BaseVolumeSz
is an optional parameter you can pass to specify
the size of the base volume. By default the size is the end of blocks
written by TFS.
To specify 100 megabytes:
{
"BaseVolumeSz": "100m"
}
To specify 1 gigabyte:
{
"BaseVolumeSz": "1g"
}
Boot
sets the path of bootloader file.
{
"Boot": "nanos/output/platform/pc/boot/boot.img"
}
Uefi
indicates whether image should support booting via UEFI. Default is false
.
{
"Uefi": true
}
UefiBoot
sets the path of UEFI bootloader file.
{
"UefiBoot": "nanos/output/platform/pc/boot/bootx64.efi"
}
BuildDir
path to the temporary directory used during image build.
{
"BuildDir": "/tmp/my_app_build_dir"
}
The CloudConfig
configures various attributes about the cloud provider, we want to use with ops.
BucketName
specifies the bucket to store the Ops built image artifacts.
{
"CloudConfig": {
"BucketName": "my-bucket"
}
}
BucketNamespace
is required on uploading files to cloud providers as oci.
{
"CloudConfig": {
"BucketNamespace": "my-namespace"
}
}
This option is supported on AWS only. Please keep in mind that, depending on the dedicated host type, only some instance flavors may be available.
DedicatedHostId
is used to specify the dedicated host id placement on cloud provider.
{
"CloudConfig" : {
"DedicatedHostId": "h-deadbeefdeadbeef1",
"Flavor": "c4.large",
"BucketName":"my-bucket"
}
}
This option is supported on AWS, GCP and Vultr.
DomainName
is used to update a A record DNS entry with the started instance IP.
{
"CloudConfig": {
"DomainName": "ops.city"
}
}
This option is only supported on AWS, GCP.
StaticIP
by cloud provider to assign a public static IP to a NIC.
{
"CloudConfig": {
"StaticIP": "1.2.3.4"
}
}
This option is only supported on AWS, GCP.
If EnableIPv6
is set to true and a VPC is created, the new VPC will have ipv6 support. Otherwise, it doesn't affect the selected VPC ipv6 support.
{
"CloudConfig": {
"EnableIPv6": true
}
}
Flavor
specifies the machine type used to create an instance. Each cloud provider has different types descriptions.
{
"CloudConfig": {
"Flavor": "t2.micro"
}
}
ImageType
allows the user to specify an image type (whose possible values are target platform-specific) when creating an image.
Can be used to create images for Hyper-V generation 2 instances.
{
"CloudConfig": {
"ImageType": "gen2"
}
}
ImageName
specifies the image name in the cloud provider.
It might be used by ops to set "IMAGE_NAME"
and "RADAR_IMAGE_NAME"
environment variables.
{
"CloudConfig": {
"ImageName": "web-server"
}
}
InstanceProfile
sets up an IAM role for an instance. Currently, this is only used for AWS but, in the future, it might be used to set roles for other cloud providers.
{
"CloudConfig": {
"InstanceProfile": "my-iam-rolename"
}
}
KMS
optionally encrypts AMIs if set. 'default' may be used for the default key or a KMS arn may be specified. This is only used for AWS.
{
"CloudConfig": {
"KMS": "default"
}
}
Platform
defines the cloud provider to use with the ops CLI.
Currently supported platforms:
- Default
onprem
- Google Cloud Platform
gcp
- AWS
aws
- Vultr
vultr
- Vmware
vsphere
- Azure
azure
- Openstack
openstack
- Upcloud
upcloud
- Hyper-v
hyper-v
- DigitalOcean
do
- OpenShift
openshift
- Oracle Cloud Infrastructure
oci
- Oracle VM VirtualBox
vbox
- Proxmox VE
proxmox
See further instructions about the cloud provider in dedicated documentation page.
{
"CloudConfig": {
"Platform": "gcp"
}
}
ProjectID
is used in some cloud providers to identify a workspace.
{
"CloudConfig": {
"ProjectID": "proj-1000"
}
}
RootVolume
is an optional set of options one can apply to the
root/base volume of an instance on AWS. These settings include the name,
typeof, iops, size, and throughput.
For more information see the AWS page.
{
"CloudConfig": {
"RootVolume": {
"Name": "my-root-volume",
"Typeof":"gp3",
"Size": 10
}
}
}
SecurityGroup
allows an instance to use an existing security group in the cloud provider.
On AWS, both the name and the id of the security group may be used as value.
{
"CloudConfig": {
"SecurityGroup": "sg-1000"
}
}
Subnet
allows an instance to use an existing subnet in the cloud provider.
On AWS, both the name and the id of the subnet may be used as value.
{
"CloudConfig": {
"Subnet": "sb-1000"
}
}
Tags
is a list of keys and values to provide more context about an instance or an image. There are a set of pre-defined tags to identify the resources created by ops
.
The following example demonstrates the default format used when deploying on different cloud providers.
{
"CloudConfig": {
"Tags": [
{
"key": "instance-owner",
"value": "joe-smith"
},
{
"key": "function",
"value": "web-server"
}
]
}
}
On GCP (only) there is also an extended configuration attribute
that can be used to detail/control the purpose and destination of the tag
{
"CloudConfig": {
"Tags": [
{
"key": "instance-owner",
"value": "joe-smith",
"attribute": {
"image_label": false,
"instance_label": true,
"instance_network": false,
"instance_metadata": false
}
},
{
"key": "function",
"value": "web-server",
"attribute": {
"image_label": true,
"instance_label": true,
"instance_network": false,
"instance_metadata": false
}
},
{
"key": "network-tag",
"value": "http",
"attribute": {
"image_label": false,
"instance_label": false,
"instance_network": true, // will use the tag value
"instance_metadata": false
}
}
]
}
}
VPC
allows instance to use an existing vpc in the cloud provider.
On AWS and DO, both the name and the id of the vpc may be used as value. On DO setting the id (uuid) avoids an extra call to the api at the cost of readability.
{
"CloudConfig": {
"VPC": "vpc-1000"
}
}
Zone
is used in some cloud providers to identify the location where cloud resources are stored.
{
"CloudConfig": {
"Zone": "us-west1-b"
}
}
Dirs
defines an array of directory locations to include into the image.
{
"Dirs": ["myapp/static"]
}
File layout on local host machine:
-myapp
app
-static
-example.html
-stylesheet
-main.css
File layout on VM:
/myapp
app
/static
-example.html
/stylesheet
-main.css
Env
defines a map of environment variables to specify for the image runtime.
{
"Env": {
"Environment": "development",
"NODE_DEBUG": "*"
}
}
Files
defines an array of file locations to include into the image.
{
"Files": ["ex.js"]
}
Kernel
sets the path of kernel image file.
{
"Kernel": "nanos/output/platform/pc/bin/kernel.img"
}
KlibDir
sets the host directory where kernel libs are located.
{
"KlibDir": "nanos/output/klib/bin"
}
Defines a list of klibs to include. For example to run the NTP klib (eg: ntpd):
{
"Klibs":["ntp"]
}
MapDirs
sets map of a local directory to a different path on the guest VM. For example the below
adds all files under /etc/ssl/certs
on host to /usr/lib/ssl/certs
on VM.
{
"MapDirs": {"/etc/ssl/certs/*": "/usr/lib/ssl/certs" },
}
Mounts
is used to mount a volume in an instance.
{
"Mounts": {
"vol": "/files"
}
}
In the case of onprem
provider, the volume can be a host directory (as opposed to a pre-made volume)
that the guest will mount as a 9P filesystem.
{
"Mounts": {
"/tmp/files": "/files"
}
}
See further instructions about volumes in dedicated documentation page.
NameServers
is an array of DNS servers to use for DNS resolution. By default it is Google's 8.8.8.8
.
{
"NameServers": ["10.8.0.1"]
}
NanosVersion
sets nanos version to be used on image manifest.
{
"NanosVersion": "nightly"
}
NightlyBuild
flag forces the use of latest dev builds.
{
"NightlyBuild": true
}
NoTrace
is an array of syscalls to mute tracing for.
{
"NoTrace": ["syscall-here"]
}
Program
specifies the path of the program to refer to on attach/detach.
{
"Program":"lua_5.2.4/lua"
}
ProgramPath
specifies the original path of the program to refer to on attach/detach.
{
"Program":"/ops_apps/lua_5.2.4/lua"
}
RebootOnExit
reboot your application immediately if it crashes (exit code is not 0). Is turned off by default, but you can enable it.
{
"RebootOnExit": true
}
same as
{
"Debugflags": ["reboot_on_exit"]
}
LocalFilesParentDirectory
is the parent directory of the files/directories specified in Files and Dirs.
The default value is the directory from where the ops command is running
{
"LocalFilesParentDirectory": "."
}
TargetRoot
TODO
{
"TargetRoot": "unix"
}
VolumesDir
is the directory used to store and fetch volumes.
{
"VolumesDir": ""
}
PackageBaseURL
provides the URL for downloading the packages.
{
"PackageBaseURL": "https://repo.ops.city/v2/packages"
}
PackageManifestURL
provides the URL to download the manifest file that stores info about all packages.
{
"PackageManifestURL": "https://repo.ops.city/v2/manifest.json"
}
Version
ops package version. Like docker if the user doesn't provide version of the image we consider "latest" as the version.
{
"Version": "v18.9.0"
}
Language
ops package language.
{
"Language": "javascript"
}
Runtime
ops package runtime.
{
"Runtime": "node"
}
Description
ops package description.
{
"Description": "node.js runtime"
}
TargetConfig
provides a limited support to adding cloud provider specific config.
This is done as to not overload the other configuration settings.
Currently only the proxmox
target has support here. For example:
{
"TargetConfig": {
"isoStorageName": "local",
"Arch": "x86_64",
"Machine": "q35",
"Sockets": "1",
"Cores": "1",
"Numa": "0",
"Memory": "512M",
"StorageName": "local-lvm",
"BridgePrefix": "vmbr",
"Onboot": "0",
"Protection": "0"
}
}
The RunConfig
configures various attributes about the runtime of the ops instance, such as allocated memory and exposed ports.
Defines whether hardware acceleration should be enabled in qemu. This option is enabled by default, but will be disabled when Debug is set to true.
{
"RunConfig": {
"Accel": true
}
}
Defines hooks to be ran after instance stops.
{
"RunConfig": {
"AtExit": "echo `date` - START; sh shell_script.sh; echo `date` - DONE"
}
}
Connects the unikernel network interface to a bridge with the name br0
. The bridge name may be overriden with the property BridgeName.
{
"RunConfig": {
"Bridged": true
}
}
Connects the unikernel network interface to a bridge with the name specified. If the bridge does not exist in host machine it is created.
{
"RunConfig": {
"BridgeName": "br1"
}
}
Enable IP forwarding when creating an instance. Only GCP, Azure provider.
{
"RunConfig": {
"CanIPForward": true
}
}
Specifies the number of CPU cores the unikernel is allowed to use.
{
"RunConfig": {
"CPUs": 2
}
}
Specifies the number of GPUs the unikernel is allowed to use. Only GCP provider.
{
"RunConfig": {
"GPUs": 1,
"GPUType": "nvidia-tesla-t4"
}
}
Specifies the type of GPU available. Only GCP provider.
{
"RunConfig": {
"GPUs": 1,
"GPUType": "nvidia-tesla-t4"
}
}
Opens a port in unikernel to allow a connection with the GDB debugger. See further instructions in Debugging. If debug is set to true the hardware acceleration Accel is disabled.
{
"RunConfig": {
"Debug": true
}
}
Defines the default gateway IP of the network interface.
{
"RunConfig": {
"Gateway": "192.168.1.255"
}
}
Define the gdb debugger port. It only takes effect if debug is enabled. By default the GdbPort
is 1234
.
{
"RunConfig": {
"GdbPort": 1234,
"Debug": true
}
}
Sets the name of the image file.
{
"RunConfig": {
"Imagename": "web-server"
}
}
Sets the name of the instance.
{
"RunConfig": {
"InstanceName": "web-server-instance"
}
}
Sets the group of the instance.
{
"RunConfig": {
"InstanceGroup": "my-autoscale-service"
}
}
Defines the IP address of the network interface.
{
"RunConfig": {
"IPAddress": "192.168.1.75"
}
}
The IP address has to be specified along with the netmask and the gateway, so the unikernel can assign the IP address to the network interface.
{
"RunConfig": {
"IPAddress": "192.168.1.75",
"NetMask": "255.255.255.0",
"Gateway": "192.168.1.1"
}
}
Defines the static IPv6 address of the network interface.
{
"RunConfig": {
"IPv6Address": "FE80::46F:65FF:FE9C:4861"
}
}
Configures the amount of memory to allocated to qemu
. Default is 128 MiB.
Optionally, a suffix of "M" or "G" can be used to signify a value in megabytes
or gigabytes respectively.
{
"RunConfig": {
"Memory": "2G"
}
}
Defines whether to emulate a VGA output device in qemu
.
{
"RunConfig": {
"Vga": false
}
}
Defines a list of directories paths in the host machine whose data will be copied to the unikernel.
{
"RunConfig": {
"Mounts": ["./files","./assets"]
}
}
Defines the netmask of the network interface.
{
"RunConfig": {
"NetMask": "255.255.255.0"
}
}
Is a list of pre-configured network cards.Meant to eventually deprecate the existing single-nic configuration. Currently only supported for Proxmox
{
"Nics": [
{
"IPAddress": "192.168.1.75",
"NetMask": "255.255.255.0",
"Gateway": "192.168.1.1",
"BridgeName": "br1"
}
]
}
Starts unikernels in background. You can stop the unikernel using the onprem instances stop command.
{
"RunConfig": {
"Background": true
}
}
A list of ports to expose. Alternatively, you can also use -p
in the command
line.
{
"RunConfig": {
"Ports": ["80", "8008"],
}
}
Enables printing more details about what ops is doing at the moment. Also, enables the printing of warnings and errors.
{
"RunConfig": {
"ShowDebug": true,
}
}
Enables printing errors with more details.
{
"RunConfig": {
"ShowErrors": true,
}
}
Enables printing warnings details.
{
"RunConfig": {
"ShowWarnings": true,
}
}
Connects the unikernel to a network interface with the name specified. If the tap
does not exist in host machine it is created.
{
"RunConfig": {
"TapName": "tap0"
}
}
Opens ports that use UDP protocol.
{
"RunConfig": {
"UDPPorts": ["60", "70-80", "6006"],
}
}
Enables verbose logging for the runtime environment. As of now, it prints the
command used to start qemu
.
{
"RunConfig": {
"Verbose": true
}
}
This property is only used by cloud provider openstack and sets the instance volume size. Default size is 1 GB.
{
"RunConfig": {
"VolumeSizeInGb": 2
}
}
This property is only used by cloud provider gcp to attach an existing cloud disk/volume to the instance being created.
{
"RunConfig": {
"AttachVolumeOnInstanceCreate": true
}
}
There is the concept of the manifest in Nanos where there exists many other config options that don't boil up to ops configuration, however, sometimes you still wish to pass these settings down to the manifest. Certain klibs, in particular, have variables that need to be set. To set them do this:
{
"ManifestPassthrough": {
"my_manifest_setting": "some_value"
}
}
Nanos offers a 'consoles' feature that allows multiple methods for monitoring a running unikernel. By default, the 'serial' and 'vga' consoles are enabled for testing purposes. However, in a production environment, it is highly advised against using these consoles as they significantly slow down the unikernel, rendering it nearly non-functional.
To activate or deactivate consoles, you can use the '+' or '-' symbols. Here is an example configuration:
"ManifestPassthrough": {
"consoles": [
"+net",
"-serial",
"-vga"
]
}
In the above example, +net
enables the net console locally for debugging purposes. The net console
is faster than 'serial' when it comes to testing performance.
The net console also provides two options: netconsole_port
and netconsole_ip
. These options allow
you to send logs to a remote machine for debugging. However, it is advisable to use 'syslog' instead
of net console for production workloads.
Nanos has an 'exec protection' feature that prevents the kernel from executing any code outside the main executable and other 'trusted' files explicitly marked. The program is further limited from modifying the executable file and creating new ones. This flag may also be used on individual files within the children tuple. This prevents the application from exec-mapping anything that is not explicitly mapped as executable.
This is not on by default, however, as many JITs won't work with it turned on.
{
"ManifestPassthrough": {
"exec_protection": "t"
}
}
Some applications expect to have a working directory in a different place than where they have been placed. You can adjust this via the manifest variable 'cwd':
{
"ManifestPassthrough": {
"cwd": "/my_new/path"
}
}
This is an optional configuration setting that allows Nanos to wait for a valid ipv4 address to become available via DHCP using the timeout of 'exec_wait_for_ip4_secs'. If static ip is set than there is no effect. This configuration is not on by default.
{
"ManifestPassthrough": {
"exec_wait_for_ip4_secs": "5"
}
}
This configuration can also be associated to a specific network interface, i.e:
{
"ManifestPassthrough": {
"en2": {
"exec_wait_for_ip4_secs": "5"
}
}
}
This is an optional configuration setting that allows Nanos to wait for a valid ipv6 address to become available via DHCP using the timeout of 'exec_wait_for_ip6_secs'. If static ip is set than there is no effect. This configuration is not on by default.
{
"ManifestPassthrough": {
"exec_wait_for_ip6_secs": "5"
}
}
This configuration can also be associated to a specific network interface, i.e:
{
"ManifestPassthrough": {
"en2": {
"exec_wait_for_ip6_secs": "5"
}
}
}
This is an optional configuration setting that changes the program exit code to 0, if a match is found on the configuration.
Some possible configurations:
{
"ManifestPassthrough": {
"expected_exit_code": "1"
}
}
{
"ManifestPassthrough": {
"expected_exit_code": "!6"
}
}
{
"ManifestPassthrough": {
"expected_exit_code": ["1", "6"]
}
}
{
"ManifestPassthrough": {
"expected_exit_code": "*"
}
}
This is an optional configuration setting that defines the minimum virtual address that a process is allowed to mmap. If set to zero, allow zero-page mappings to occur.
{
"ManifestPassthrough": {
"mmap_min_addr": "0"
}
}
By default, the user application is not allowed to overwrite the program binary file (and interpreter, if present).
This default behavior can be overridden by inserting a program_overwrite
attribute in the root tuple of the manifest.
{
"ManifestPassthrough": {
"program_overwrite": "t"
}
}
This enables tracing calls made by the application binary to dynamic library functions. It works on both pie and no-pie programs, and also works with aslr.
{
"ManifestPassthrough": {
"ltrace": {}
}
}
This is an optional configuration setting that changes the MTU (Maximum Transmission Unit) size of the network interface(s). This doesn't affect TUN interface(s).
{
"ManifestPassthrough": {
"mtu": "1420"
}
}
This is an optional configuration setting used to manage the size (in bytes) of the socket receive buffer. The default buffer size is 208 KB, so to change the size to, say, 512 KB, you could use the following config:
{
"ManifestPassthrough": {
"so_rcvbuf": "524288"
}
}
This is an optional configuration setting that will disable demand paging. This is overriden when ltrace is in use.
{
"ManifestPassthrough": {
"static_map_program": "t"
}
}
This is an optional configuration setting that allows specifying trace flags (i.e. a comma-delimited set of trace message types) in the trace
symbol of the root tuple.
A given trace message is output only if its message type is enabled in the trace flags.
pf
- page-fault-related messages - setsTRACE_PAGE_FAULT
threadrun
- messages output when returning to user threads - setsTRACE_THREAD_RUN
all
- all tracing messages are enabled- for backward compatibility, any unknown trace flag enables all messages output via calls to thread_log(). - sets
TRACE_OTHER
Example: a "trace:pf,other"
value in the manifest enables page-fault-related messages and messages classified as "other".
{
"ManifestPassthrough": {
"trace:pf,threadrun,all": {}
}
}
This is an optional configuration setting that reboots your application immediately if it stops with an exit code that matches the configuration.
This is checked after manifestpassthrough.expected_exit_code
execution (when both configured).
Note: poweroff/stop
operations work normally, regardless of the presence and value of the reboot_on_exit
option, and regardless of the application exit code.
Some possible configurations:
{
"ManifestPassthrough": {
"reboot_on_exit": "!0"
}
}
{
"ManifestPassthrough": {
"reboot_on_exit": "1"
}
}
{
"ManifestPassthrough": {
"reboot_on_exit": ["0", "1", "6"]
}
}
{
"ManifestPassthrough": {
"reboot_on_exit": "*"
}
}
This is an optional configuration setting that keeps a VM running after the user program exits with an exit code that matches the configuration. The VCPUs remain halted.
Some possible configurations:
{
"ManifestPassthrough": {
"idle_on_exit": "0"
}
}
{
"ManifestPassthrough": {
"idle_on_exit": "!0"
}
}
{
"ManifestPassthrough": {
"idle_on_exit": ["0", "1", "6"]
}
}
{
"ManifestPassthrough": {
"idle_on_exit": "*"
}
}
By default the virtio-net driver utilizes as many tx/rx queues as supported by the attached device but limited to the number of cpus exposed to the instance. This setting may be changed via the following config for the guest:
"ManifestPassthrough": {
"en1": {
"io-queues": "2"
}
}
This is an optional configuration that will make the kernel return your chosen strings in the sysname and release fields of the uname syscall. Several programs expect known names or versions, to for example, enable certain features.
The default is set to "Nanos" for the sysname field, and "5.0-<NANOS_VERSION>" for the release.
"ManifestPassthrough": {
"uname": {
"sysname": "Linux",
"release": "5.0.5"
}
}
Transparent HugePage Support (THP) is enabled by default on nanos.
To disable THP support set "transparent_hugepage"
to "never"
.
Any other value, will keep THP enabled.
{
"ManifestPassthrough": {
"transparent_hugepage": "never"
}
}
Debugflags
adds additional debug flags to the runtime.
{
"Debugflags": ["trace:pf,threadrun,all", "debugsyscalls"]
}
idle_on_exit
keeps a VM running after the user program exits successfully (exit code is 0). The VCPUs remain halted.
{
"Debugflags": ["idle_on_exit"]
}
reboot_on_exit
reboot your application immediately if it crashes (exit code is not 0). Is turned off by default, but you can enable it.
{
"Debugflags": ["reboot_on_exit"]
}
TODO