Update documentation related to the prefetcher (#1049)

* Update documentation related to the prefetcher Update changelog to include changes in the prefetcher and document the unstable configuration to set maximum prefetch window size. Signed-off-by: Monthon Klongklaew <monthonk@amazon.com> * PR comments Signed-off-by: Monthon Klongklaew <monthonk@amazon.com> * Change wording Signed-off-by: Monthon Klongklaew <monthonk@amazon.com> * Apply PR suggestion Co-authored-by: Daniel Carl Jones <danny@danielcarl.info> Signed-off-by: Monthon Klongklaew <monthonk@amazon.com> --------- Signed-off-by: Monthon Klongklaew <monthonk@amazon.com> Co-authored-by: Daniel Carl Jones <danny@danielcarl.info>
awslabs · Oct 10, 2024 · 0415b5c · 0415b5c
1 parent 548c0de
commit 0415b5c
Show file tree

Hide file tree

Showing 2 changed files with 17 additions and 0 deletions.
diff --git a/doc/CONFIGURATION.md b/doc/CONFIGURATION.md
@@ -358,3 +358,16 @@ We welcome feedback on how this works for your applications and workloads.
 ## Logging
 
 By default, Mountpoint emits high-severity log information to [syslog](https://datatracker.ietf.org/doc/html/rfc5424) if available on your system. You can change what level of information is logged, and to where it is logged. See [LOGGING.md](LOGGING.md) for more details on configuring logging.
+
+## Unstable configurations
+
+Configurations in this section are experimental and may be removed or superseded by other configurations in any future release.
+
+### Maximum prefetch window size
+
+Mountpoint downloads object contents into memory ahead of read requests in order to optimize sequential-read throughput. The prefetch window size determines how much data Mountpoint would prefetch data into memory and the value can be different for each file handle. These window sizes are adjusted automatically based on available system memory, scaling up to a maximum of 2 GiB per file handle by default.
+
+This should work well for most use cases but if you need to use a different value, you can set the maximum prefetch window size to an arbitrary number with the `UNSTABLE_MOUNTPOINT_MAX_PREFETCH_WINDOW_SIZE` environment variable at mount time, providing a value in bytes.
+
+> [!WARNING]
+> Overriding the default maximum prefetch window size may reduce overall read throughput.
diff --git a/mountpoint-s3/CHANGELOG.md b/mountpoint-s3/CHANGELOG.md
@@ -1,5 +1,9 @@
 ## Unreleased
 
+### New features
+
+* Mountpoint now automatically adjusts its prefetcher read window size based on available system memory. This reduces the risk of Mountpoint potentially consuming all available system memory in cases where a large number of file handles are read from concurrently. ([#1013](https://github.com/awslabs/mountpoint-s3/pull/1013))
+
 ### Breaking changes
 
 * When configured to log to a directory, Mountpoint now includes a random string following the timestamp in the file name.