Total and free size of filesystem reported in sites doesn't match actual values

[fluiddot]

Quick summary

The total and free size of filesystem reported in sites doesn't match the ones from the machine. This can lead to issues in plugins that rely on these values.

Steps to reproduce

1. Open the app.
2. Create/start a site.
3. Edit the wp-config-php file of the site and add the following code at the bottom:

define( 'WP_DEBUG', true );
define( 'WP_DEBUG_LOG', true );
error_log("Disk Free Space: " . print_r(disk_free_space('/'), true));
error_log("Disk Total Space: " . print_r(disk_total_space('/'), true));

4. Navigate to the site.
5. Open the logs file located in <SITE_PATH>/wp-content/debug.log.
6. Observe that disk space values don't match the ones from the machine. E.g.:
   • Disk Free Space: 2048000000 (2GB)
   • Disk Total Space: 4096000000 (4GB)

What you expected to happen

The size of filesystem matches the values of the machine.

What actually happened

The size of the filesystem doesn't match the values of the machine.

Impact

All

Available workarounds?

No but the app is still usable

Platform

Mac Silicon

Logs or notes

No response

[fluiddot]

I explored the Playground code and noticed that the free and total values are fixed values. Let me briefly explain the references I found while debugging this:

Free space

• PHP's function disk_free_space calculates the value using the filesystem function statfs:
  • https://github.com/php/php-src/blob/c69dedabb23e7c3c212f31a00251536da0ab61bf/ext/standard/filestat.c#L192
  • https://github.com/php/php-src/blob/c69dedabb23e7c3c212f31a00251536da0ab61bf/ext/standard/filestat.c#L234-L238
• The calculation isf_bsize * f_bavail, which values represent Optimal transfer block size and Free blocks available to unprivileged user (documentation).

Total space

• PHP's function disk_total_space calculates the value using the filesystem function statfs:
  • https://github.com/php/php-src/blob/c69dedabb23e7c3c212f31a00251536da0ab61bf/ext/standard/filestat.c#L107
  • https://github.com/php/php-src/blob/c69dedabb23e7c3c212f31a00251536da0ab61bf/ext/standard/filestat.c#L152-L156
• The calculation isf_bsize * f_blocks, which values represent Optimal transfer block size and Total data blocks in filesystem (documentation).

Filesystem values

• The values used in the above calculations are obtained from NODEFS.statfs function which returns value is represented with the class statfs.
• The implementation of this function can be seen here for the PHP version 8.1:

HEAP32[(((buf) + (4)) >> 2)] = 4096;
HEAP32[(((buf) + (40)) >> 2)] = 4096;   // f_bsize - Block size
HEAP32[(((buf) + (8)) >> 2)] = 1e6;     // f_blocks - Total blocks
HEAP32[(((buf) + (12)) >> 2)] = 5e5;    // f_bfree - Free blocks (not used in calculation)
HEAP32[(((buf) + (16)) >> 2)] = 5e5;    // f_bavail - Available blocks
HEAP32[(((buf) + (20)) >> 2)] = FS.nextInode;
HEAP32[(((buf) + (24)) >> 2)] = 1e6;
HEAP32[(((buf) + (28)) >> 2)] = 42;
HEAP32[(((buf) + (44)) >> 2)] = 2;
HEAP32[(((buf) + (36)) >> 2)] = 255;

• As you can see, the values are fixed so as outlined in the issue, they don't reflect the actual total and free size of the file system. Here are the results:
  • Total available size: 4096 * 1^6 = 4.096.000.000 (4GB).
  • Total free size: 4096 * 5^5 = 2.048.000.000 (2GB).

As a next step, we'd need to explore if this is set by Playground when compiling PHP source. Or if on the other hand, this is set by Emscripten.

UPDATE: As shared in #341 (comment), the values are set by Emscripten here.

[wojtekn]

CC @adamziel

[jeroenpf]

@fluiddot I went a bit deeper and it looks like these values are set in emscripten:

https://github.com/emscripten-core/emscripten/blob/7f8a05dd4e37cbd7ffde6d624f91fd545f7b52e3/src/library_syscall.js#L804
https://github.com/emscripten-core/emscripten/blob/7f8a05dd4e37cbd7ffde6d624f91fd545f7b52e3/system/lib/wasmfs/syscalls.cpp#L1530

With the various types of available VFS's it could be complicated to give a meaningful output of statfs which is probably why they settled on a stub with, as they call it, sane values.

@adamziel Do you have any ideas of how we can proceed with this? I'm happy to have a deeper look but I'm out of my depth here so some pointers would be very welcome :-)

[adamziel]

It seems like it's reporting the heap size, not the disk size. The 2GB comes up on this Emscripten doc page but not directly tied with this problem.

The original commit introducing these constants in Emscripten says:

// None of the constants here are true. We're just returning safe and
// sane values.

It sounds like this could be solved with an Emscripten PR that delegates the statfs call to the FS object which in turn delegates it to the underlying FS backend like NODEFS and falls back to the hardcoded constants if that backend doesn't support the statfs call. Opening an Emscripten issue first would be useful to validate the approach with the team.

[jeroenpf]

This approach could work. Some filesystems might not be relevant because they only work in the browsers. For NODEFS this will work and I was able to test that with some custom code:

  function ___syscall_statfs64(path, size, buf) {

      let defaults = {
      	bsize: 4096,
      	frsize: 4096,
      	blocks: 1e6,
      	bfree: 5e5,
      	bavail: 5e5,
      	files: FS.nextInode,
      	ffree: 1e6,
      	fsid: 42,
      	flags: 2,
      	namelen: 255
      }

    try {
	    path = SYSCALLS.getStr(path);

	    if (ENVIRONMENT_IS_NODE ) { // need to check here if FS is nodefs too.
		    const node = FS.lookupPath(SYSCALLS.getStr(path), { follow: true }).node;
		    const stats = fs.statfsSync(node.mount.opts.root);
		    defaults.bsize = stats.bsize;
		    defaults.frsize = stats.bsize;
		    defaults.blocks = stats.blocks;
		    defaults.bfree = stats.bfree;
		    defaults.bavail = stats.bavail;
		    defaults.files = stats.files;
		    defaults.ffree = stats.ffree;
		    defaults.fsid = stats.type;
	    }

      HEAP32[buf + 4 >> 2] = defaults.bsize;
      HEAP32[buf + 40 >> 2] = defaults.frsize;
      HEAP32[buf + 8 >> 2] = defaults.blocks;
      HEAP32[buf + 12 >> 2] = defaults.bfree;
      HEAP32[buf + 16 >> 2] = defaults.bavail;
      HEAP32[buf + 20 >> 2] = defaults.files;
      HEAP32[buf + 24 >> 2] = defaults.ffree;
      HEAP32[buf + 28 >> 2] = defaults.fsid;
      HEAP32[buf + 44 >> 2] = defaults.flags;
      HEAP32[buf + 36 >> 2] = defaults.namelen;
      return 0;
    } catch (e) {
      if (typeof FS == "undefined" || !(e.name === "ErrnoError"))
        throw e;
      return -e.errno;
    }
  }

As far as i can see the values are correctly added to the shared memory and read by the systemcall inside WASM.

I will create an issue on emscripten. There's probably a bit more that we need to consider and each filesystem might need different defaults or ways to determine more accurate numbers.