Skip to content

Commit 1f9ecf7

Browse files
feat: fuse-overlayfs configuration (#2685)
* feat: fuse-overlayfs configuration Signed-off-by: dkwon17 <dakwon@redhat.com> * Update Signed-off-by: dkwon17 <dakwon@redhat.com> * Indentation Signed-off-by: dkwon17 <dakwon@redhat.com> * Update Signed-off-by: dkwon17 <dakwon@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update butane version Signed-off-by: dkwon17 <dakwon@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/administration-guide/pages/configuring-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/accessing-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/accessing-fuse.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/enabling-overlay-with-a-configmap.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/using-the-fuse-overlay-storage-driver.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/enabling-overlay-with-a-configmap.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/enabling-overlay-with-a-configmap.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/using-the-fuse-overlay-storage-driver.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/enabling-overlay-with-a-configmap.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update modules/end-user-guide/pages/enabling-overlay-with-a-configmap.adoc Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com> * Update prerequisites Signed-off-by: dkwon17 <dakwon@redhat.com> * Fix indentation formatting Signed-off-by: dkwon17 <dakwon@redhat.com> * Mention Buildah Signed-off-by: dkwon17 <dakwon@redhat.com> * Fix build Signed-off-by: dkwon17 <dakwon@redhat.com> --------- Signed-off-by: dkwon17 <dakwon@redhat.com> Co-authored-by: Jana Vrbkova <jvrbkova@redhat.com>
1 parent 2c9e26c commit 1f9ecf7

File tree

7 files changed

+304
-0
lines changed

7 files changed

+304
-0
lines changed

modules/administration-guide/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@
8080
*** xref:configuring-cluster-roles-for-users.adoc[]
8181
*** xref:configuring-advanced-authorization.adoc[]
8282
*** xref:removing-user-data-in-compliance-with-the-gdpr.adoc[]
83+
** xref:configuring-fuse.adoc[]
8384
* xref:managing-ide-extensions.adoc[]
8485
** xref:extensions-for-microsoft-visual-studio-code-open-source.adoc[]
8586
* xref:managing-workloads-using-the-che-server-api.adoc[]
Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
:_content-type: PROCEDURE
2+
:description: Configuring fuse-overlayfs
3+
:keywords: administration-guide, configuring, fuse
4+
:navtitle: Configuring fuse-overlayfs
5+
:page-aliases:
6+
7+
[id="configuring-fuse-overlayfs"]
8+
= Configuring fuse-overlayfs
9+
10+
By default, the Universal Developer Image (UDI) contains Podman and Buildah which you can use to build and push container images within a workspace.
11+
However, Podman and Buildah in the UDI are configured to use the `vfs` storage driver which does not provide copy-on-write support.
12+
For more efficient image management, use the fuse-overlayfs storage driver which supports copy-on-write in rootless environments.
13+
14+
15+
== Enabling container access to /dev/fuse for OpenShift
16+
17+
To use fuse-overlayfs, you must make `/dev/fuse` accessible to workspace containers first.
18+
19+
[WARNING]
20+
====
21+
Creating `MachineConfig` resources on an OpenShift cluster is a potentially dangerous task, as you are making advanced, system-level changes to the cluster.
22+
23+
View the link:https://docs.openshift.com/container-platform/{ocp4-ver}/post_installation_configuration/machine-configuration-tasks.html#machine-config-overview-post-install-machine-configuration-tasks[MachineConfig documentation] for more details and possible risks.
24+
25+
====
26+
27+
.Prerequisites
28+
29+
* The link:https://docs.openshift.com/container-platform/{ocp4-ver}/installing/install_config/installing-customizing.html#installation-special-config-butane-install_installing-customizing[Butane] tool (`butane`) is installed in the operating system you are using.
30+
31+
* An active `{orch-cli}` session with administrative permissions to the destination OpenShift cluster. See {orch-cli-link}.
32+
33+
.Procedure
34+
35+
. Set the environment variable based on the type of your OpenShift cluster: a single node cluster, or a multi node cluster with separate control plane and worker nodes.
36+
+
37+
* For a single node cluster, set:
38+
+
39+
[subs="+quotes,+attributes,+macros"]
40+
----
41+
$ NODE_ROLE=master
42+
----
43+
+
44+
* For a multi node cluster, set:
45+
[subs="+quotes,+attributes,+macros"]
46+
+
47+
----
48+
$ NODE_ROLE=worker
49+
----
50+
51+
. Set the environment variable for the OpenShift Butane config version. This variable is the major and minor version of the OpenShift cluster. For example, `4.12.0`, `4.13.0`, or `4.14.0`.
52+
+
53+
[subs="+quotes,+attributes,+macros"]
54+
----
55+
$ VERSION=4.12.0
56+
----
57+
+
58+
. Create a `MachineConfig` resource that creates a drop-in CRI-O configuration file named `99-podman-fuse` in the `NODE_ROLE` nodes. This configuration file makes access to the `/dev/fuse` device possible for certain pods.
59+
+
60+
[source,subs="+quotes,+macros"]
61+
----
62+
cat << EOF | butane | oc apply -f -
63+
variant: openshift
64+
version: ${VERSION}
65+
metadata:
66+
labels:
67+
machineconfiguration.openshift.io/role: ${NODE_ROLE}
68+
name: 99-podman-dev-fuse-${NODE_ROLE}
69+
storage:
70+
files:
71+
- path: /etc/crio/crio.conf.d/99-podman-fuse <1>
72+
mode: 0644
73+
overwrite: true
74+
contents: <2>
75+
inline: |
76+
[crio.runtime.workloads.podman-fuse] <3>
77+
activation_annotation = "io.openshift.podman-fuse" <4>
78+
allowed_annotations = [
79+
"io.kubernetes.cri-o.Devices" <5>
80+
]
81+
[crio.runtime]
82+
allowed_devices = ["/dev/fuse"] <6>
83+
EOF
84+
----
85+
<1> The absolute file path to the new drop-in configuration file for CRI-O.
86+
<2> The content of the new drop-in configuration file.
87+
<3> Define a `podman-fuse` workload.
88+
<4> The pod annotation that activates the `podman-fuse` workload settings.
89+
<5> List of annotations the `podman-fuse` workload is allowed to process.
90+
<6> List of devices on the host that a user can specify with the `io.kubernetes.cri-o.Devices` annotation.
91+
+
92+
. After applying the `MachineConfig` resource, scheduling will be temporarily disabled for each node with the `worker` role as changes are applied. View the nodes' statuses.
93+
+
94+
[subs="+quotes,+attributes,+macros"]
95+
----
96+
$ oc get nodes
97+
----
98+
+
99+
Example output:
100+
+
101+
[subs="+quotes,+attributes,+macros"]
102+
----
103+
NAME STATUS ROLES AGE VERSION
104+
ip-10-0-136-161.ec2.internal Ready worker 28m v1.27.9
105+
ip-10-0-136-243.ec2.internal Ready master 34m v1.27.9
106+
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.27.9
107+
ip-10-0-142-249.ec2.internal Ready master 34m v1.27.9
108+
ip-10-0-153-11.ec2.internal Ready worker 28m v1.27.9
109+
ip-10-0-153-150.ec2.internal Ready master 34m v1.27.9
110+
----
111+
+
112+
. Once all nodes with the `worker` role have a status `Ready`, `/dev/fuse` will be available to any pod with the following annotations.
113+
+
114+
[source,yaml,subs="+quotes,+attributes"]
115+
----
116+
io.openshift.podman-fuse: ''
117+
io.kubernetes.cri-o.Devices: /dev/fuse
118+
----
119+
120+
.Verification steps
121+
122+
. Get the name of a node with a `worker` role:
123+
+
124+
[subs="+attributes,+quotes"]
125+
----
126+
$ oc get nodes
127+
----
128+
129+
. Open an `oc debug` session to a worker node.
130+
+
131+
[subs="+attributes,+quotes"]
132+
----
133+
$ oc debug node/__<nodename>__
134+
----
135+
136+
. Verify that a new CRI-O config file named `99-podman-fuse` exists.
137+
+
138+
[subs="+attributes,+quotes"]
139+
----
140+
sh-4.4# stat /host/etc/crio/crio.conf.d/99-podman-fuse
141+
----
142+
143+
== Using fuse-overlayfs for Podman and Buildah within a workspace
144+
Users can follow xref:end-user-guide:using-the-fuse-overlay-storage-driver.adoc[] to update existing workspaces to use the fuse-overlayfs storage driver for Podman and Buildah.

modules/end-user-guide/nav.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@
1515
** xref:starting-a-workspace-from-a-raw-devfile-url.adoc[]
1616
** xref:basic-actions-you-can-perform-on-a-workspace.adoc[]
1717
** xref:authenticating-to-a-git-server-from-a-workspace.adoc[]
18+
** xref:using-the-fuse-overlay-storage-driver.adoc[]
19+
*** xref:accessing-fuse.adoc[]
20+
*** xref:enabling-overlay-with-a-configmap.adoc[]
1821
* xref:using-che-in-team-workflow.adoc[]
1922
** xref:first-time-contributors.adoc[]
2023
** xref:benefits-of-pull-requests-review-in-che.adoc[]
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
:_content-type: PROCEDURE
2+
:description: Accessing /dev/fuse from your workspaces.
3+
:keywords: user-guide, fuse, overlay, device, /dev/fuse, pod, overrides
4+
:navtitle: Accessing /dev/fuse
5+
:page-aliases:
6+
7+
[id="accessing-fuse"]
8+
= Accessing /dev/fuse from a workspace
9+
10+
You must have access to `/dev/fuse` to use fuse-overlayfs. This section describes how to make `/dev/fuse` accessible to workspace containers.
11+
12+
.Prerequisites
13+
14+
* The administrator has enabled access to `/dev/fuse` by following xref:administration-guide:configuring-fuse.adoc[].
15+
* Determine a workspace to use fuse-overlayfs with.
16+
17+
.Procedure
18+
19+
. Use the `pod-overrides` attribute to add the required annotations defined in xref:administration-guide:configuring-fuse.adoc[] to the workspace. The `pod-overrides` attribute allows merging certain fields in the workspace pod's `spec`.
20+
+
21+
[subs="+quotes,+attributes,+macros"]
22+
----
23+
$ {orch-cli} patch devworkspace __<DevWorkspace_name>__ \
24+
--patch '{"spec":{"template":{"attributes":{"pod-overrides":{"metadata":{"annotations":{"io.kubernetes.cri-o.Devices":"/dev/fuse","io.openshift.podman-fuse":""}}}}}}}' \
25+
--type=merge
26+
----
27+
28+
.Verification steps
29+
30+
. Start the workspace and verify that `/dev/fuse` is available in the workspace container.
31+
+
32+
[subs="+attributes,+quotes"]
33+
----
34+
$ stat /dev/fuse
35+
----
36+
37+
After completing this procedure, follow the steps in xref:enabling-overlay-with-a-configmap.adoc[] to use fuse-overlayfs for Podman.
Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
:_content-type: PROCEDURE
2+
:description: You can enable fuse-overlayfs storage driver for Podman
3+
:keywords: overlay, configmap, podman, storage, driver, fuse-overlayfs
4+
:navtitle: Enabling fuse-overlayfs with a ConfigMap
5+
:page-aliases:
6+
7+
[id="enabling-fuse-overlay-with-a-configmap"]
8+
= Enabling fuse-overlayfs with a Configmap
9+
10+
You can define the storage driver for Podman and Buildah in the `~/.config/containers/storage.conf` file. Here are the default contents of the `/home/user/.config/containers/storage.conf` file in the UDI container:
11+
12+
.`storage.conf`
13+
[source]
14+
----
15+
[storage]
16+
driver = "vfs"
17+
----
18+
19+
To use fuse-overlayfs, `storage.conf` can be set to the following:
20+
21+
.`storage.conf`
22+
[source]
23+
----
24+
[storage]
25+
driver = "overlay"
26+
27+
[storage.options.overlay]
28+
mount_program="/usr/bin/fuse-overlayfs" <1>
29+
----
30+
<1> The absolute path to the `fuse-overlayfs` binary. The `/usr/bin/fuse-overlayfs` path is the default for the UDI.
31+
32+
You can do this manually after starting a workspace. Another option is to build a new image based on the UDI with changes to `storage.conf` and use the new image for workspaces.
33+
34+
Otherwise, you can update the `/home/user/.config/containers/storage.conf` for all workspaces in your {orch-namespace} by creating a ConfigMap that mounts the updated file. See xref:mounting-configmaps.adoc[].
35+
36+
.Prerequisites
37+
38+
* The administrator has enabled access to `/dev/fuse` by following xref:administration-guide:configuring-fuse.adoc[].
39+
40+
* A workspace with the required annotations are set by following xref:accessing-fuse.adoc[]
41+
42+
43+
[NOTE]
44+
====
45+
Since ConfigMaps mounted by following xref:mounting-configmaps.adoc[this guide] mounts the ConfigMap's data to all workspaces, following this procedure will set the storage driver to fuse-overlayfs for all workspaces. Ensure that your workspaces contain the required annotations to use fuse-overlayfs by following xref:accessing-fuse.adoc[].
46+
====
47+
48+
.Procedure
49+
50+
. Apply a ConfigMap that mounts a `/home/user/.config/containers/storage.conf` file in your {orch-namespace}.
51+
+
52+
[source,yaml,subs="+quotes,+attributes,+macros"]
53+
----
54+
kind: ConfigMap
55+
apiVersion: v1
56+
metadata:
57+
name: fuse-overlay
58+
labels:
59+
controller.devfile.io/mount-to-devworkspace: 'true'
60+
controller.devfile.io/watch-configmap: 'true'
61+
annotations:
62+
controller.devfile.io/mount-as: file
63+
controller.devfile.io/mount-path: /home/user/.config/containers
64+
data:
65+
storage.conf: |
66+
[storage]
67+
driver = "overlay"
68+
69+
[storage.options.overlay]
70+
mount_program="/usr/bin/fuse-overlayfs"
71+
----
72+
73+
.Verification steps
74+
75+
. Start the workspace containing the required annotations and verify that the storage driver is `overlay`.
76+
+
77+
[subs="+attributes,+quotes"]
78+
----
79+
$ podman info | grep overlay
80+
----
81+
82+
+
83+
Example output:
84+
+
85+
[subs="+attributes,+quotes"]
86+
----
87+
graphDriverName: overlay
88+
overlay.mount_program:
89+
Executable: /usr/bin/fuse-overlayfs
90+
Package: fuse-overlayfs-1.12-1.module+el8.9.0+20326+387084d0.x86_64
91+
fuse-overlayfs: version 1.12
92+
Backing Filesystem: overlayfs
93+
----

modules/end-user-guide/pages/getting-started-with-che.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@ If your organization is already running a {prod-short} instance, you can get sta
1414
* xref:starting-a-workspace-from-a-raw-devfile-url.adoc[]
1515
* xref:basic-actions-you-can-perform-on-a-workspace.adoc[]
1616
* xref:authenticating-to-a-git-server-from-a-workspace.adoc[]
17+
* xref:using-the-fuse-overlay-storage-driver.adoc[]
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
:_content-type: CONCEPT
2+
:description: Using the fuse-overlayfs storage driver for Podman and Buildah
3+
:keywords: fuse, overlay, fuse-overlayfsm, podman, buildah, storage, driver
4+
:navtitle: Using the fuse-overlayfs storage driver for Podman and Buildah
5+
:page-aliases:
6+
7+
[id="using-the-fuse-overlay-storage-driver-for-podman-and-buildah"]
8+
= Using the fuse-overlayfs storage driver for Podman and Buildah
9+
10+
By default, newly created workspaces that do not specify a devfile will use the Universal Developer Image (UDI).
11+
The UDI contains common development tools and dependencies commonly used by developers.
12+
13+
Podman and Buildah are included in the UDI, allowing developers to build and push container images from their workspace.
14+
15+
By default, Podman and Buildah in the UDI are configured to use the `vfs` storage driver.
16+
For more efficient image management, use the fuse-overlayfs storage driver which supports copy-on-write in rootless environments.
17+
18+
You must meet the following requirements to fuse-overlayfs in a workspace:
19+
20+
. The administrator has configured `/dev/fuse` access on the cluster by following xref:administration-guide:configuring-fuse.adoc[].
21+
. The workspace has the necessary annotations for using the `/dev/fuse` device. See xref:accessing-fuse.adoc[].
22+
. The `storage.conf` file in the workspace container has been configured to use fuse-overlayfs. See xref:enabling-overlay-with-a-configmap.adoc[].
23+
24+
.Additional resources
25+
* link:https://github.com/devfile/developer-images[Universal Developer Image]

0 commit comments

Comments
 (0)