Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fsFixture test util #8856

Merged
merged 17 commits into from
Aug 16, 2023
Merged

Add fsFixture test util #8856

merged 17 commits into from
Aug 16, 2023

Conversation

lettertwo
Copy link
Member

@lettertwo lettertwo commented Feb 25, 2023

fsFixture is test utility that is meant to make writing integration test cases easier by allowing us to colocate the filesystem-bound fixtures with the tests.

Example

A test from the javascript suite converted to fsFixture

  it('should produce a basic JS bundle with CommonJS requires', async function () {
    await fsFixture(overlayFS, __dirname)`
      commonjs
        index.js:
          var local = require('./local');
          // eslint-disable-next-line no-unused-vars
          var url = require('url');

          module.exports = function () {
            return local.a + local.b;
          };

        local.js:
          exports.a = 1;
          exports.b = 2;`;
    
    // The fixture is a set of files defined by the above `fsFixture`
    let b = await bundle(path.join(__dirname, '/commonjs/index.js'), {
      inputFS: overlayFS,
    });

    let output = await run(b);
    assert.equal(typeof output, 'function');
    assert.equal(output(), 3);
  });
See the original
  it('should produce a basic JS bundle with CommonJS requires', async function () {
    // The fixture is a set of files at the path `/integration/commonjs/`
    let b = await bundle(
      path.join(__dirname, '/integration/commonjs/index.js'),
    );

    let output = await run(b);
    assert.equal(typeof output, 'function');
    assert.equal(output(), 3);
  });

The test itself becomes a bit more verbose, but hopefully it's obvious that the fixtures themselves are now closer to the test.

While this example shows converting an existing test, fsFixture is probably most useful for authoring new tests, as it allows us to iterate on the fixtures themselves along with tests without switching between files and directories.

Changes to MemoryFS

Use cases for fsFixture revealed a couple of edge cases that are addressed in this PR:

  • Reading from the root dir ('/') didn't work due to a couple of assumptions about filepaths
  • The bitmasks used to detect an entry type (file, directory, symlink) didn't always mask the mode correctly, which could result in it being impossible to distinguish a symlink from a file.

Grammar

fsFixture defines a simple DSL with just a handful of rules:

  • directories are defined on a line by themselves:
    fsFixture(fs)`
      dirname
    `
  • directories can be nested with two spaces:
    fsFixture(fs)`
      dirname
        nested
    `
  • or on one line with /:
    fsFixture(fs)`
      dirname/nested
    `
  • symlinks are defined with a ->:
    fsFixture(fs)`
      symlink -> target
    `
  • files are defined with a ::
    fsFixture(fs)`
      emptyfile:
    `
  • files contents are defined as everything after the ':' until the indentation level matches the file's:
    fsFixture(fs)`
      file: contents
      anotherfile:
        multiline
        contents
      emptyfile:
    `

Since fsFixture is a tagged template literal, it is possible to embed expressions using the ${embedded expression} syntax.

See the tests in this PR for more examples!

Converting existing tests

A CLI can be found at scripts/to-fs-fixture.js that is capable of doing some pretty thorough code modification to existing tests:

Usage: to-fs-fixture [options] <files...>

Convert test files to use fsFixture.

Arguments:

  files             One or more files to be transformed in-place. Required.

Options:
  --cleanup         Cleanup old fixtures after running
  --dry-run         Dry run (implies --verbose)
  --grep <pattern>  Only transform tests matching this pattern. Can be repeated. (default: [])
  --keep <glob>     Keep fixtures matching glob. Can be repeated. (default: [])
  --verbose         Verbose output
  --yes             Say yes to all prompts
  -h, --help        output usage information

By default, it will prompt for modifying each test it finds, and it can display a diff of the change it will apply, with options to skip, which makes running this tool on an existing test suite an incremental process.

@parcel-benchmark
Copy link

parcel-benchmark commented Feb 25, 2023

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.47s +16.00ms
Cached 260.00ms -28.00ms 🚀

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

React HackerNews ✅

Timings

Description Time Difference
Cold 4.09s +60.00ms
Cached 383.00ms +5.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/PermalinkedComment.b41e2791.js 4.07kb +0.00b 422.00ms +33.00ms ⚠️
dist/UserProfile.2a2fa310.js 1.51kb +0.00b 422.00ms +33.00ms ⚠️
dist/NotFound.13a965e5.js 399.00b +0.00b 421.00ms +32.00ms ⚠️
dist/logo.8dd07848.png 244.00b +0.00b 303.00ms -18.00ms 🚀

Cached Bundles

Bundle Size Difference Time Difference
dist/PermalinkedComment.b41e2791.js 4.07kb +0.00b 356.00ms -23.00ms 🚀
dist/UserProfile.2a2fa310.js 1.51kb +0.00b 356.00ms -23.00ms 🚀
dist/NotFound.13a965e5.js 399.00b +0.00b 356.00ms -23.00ms 🚀
dist/logo.8dd07848.png 244.00b +0.00b 293.00ms -18.00ms 🚀

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 37.99s -671.00ms
Cached 2.26s +137.00ms ⚠️

Cold Bundles

Bundle Size Difference Time Difference
dist/index.ca7cdd59.js 3.79mb +0.00b 17.13s -883.00ms 🚀
dist/component-lazy.1b33c14d.js 59.63kb +0.00b 5.62s -975.00ms 🚀
dist/component.bb2f7991.js 58.01kb +0.00b 5.35s -278.00ms 🚀
dist/component.1c22aee9.js 18.81kb +0.00b 5.35s -273.00ms 🚀
dist/media-viewer-analytics-error-boundary.c1511b0d.js 3.32kb +0.00b 12.13s +2.83s ⚠️
dist/ru.896915b9.js 2.94kb +0.00b 8.86s +2.26s ⚠️
dist/date.7b2f9581.js 2.07kb +0.00b 5.62s -281.00ms
dist/code.ef3dfa9c.js 1.69kb +0.00b 5.62s -281.00ms
dist/16.87c743d1.js 1.48kb +0.00b 5.35s -273.00ms 🚀
dist/16.dd50aef4.js 1.41kb +0.00b 5.35s -272.00ms 🚀
dist/heading5.023a8f1f.js 1.36kb +0.00b 6.37s +471.00ms ⚠️
dist/action.361730a6.js 1.15kb +0.00b 5.62s -281.00ms
dist/16.9e7cc0d9.js 1.13kb +0.00b 5.35s -273.00ms 🚀
dist/decision.36a0b771.js 1.10kb +0.00b 5.62s -281.00ms
dist/16.8d078bd1.js 1.08kb +0.00b 5.35s -273.00ms 🚀
dist/16.bb53313d.js 1.08kb +0.00b 5.35s -273.00ms 🚀
dist/16.0d8c3c9e.js 1.06kb +0.00b 5.62s -281.00ms
dist/16.c0880b62.js 992.00b +0.00b 5.35s -272.00ms 🚀
dist/16.99296be0.js 964.00b +0.00b 5.35s -273.00ms 🚀
dist/16.c16ee42d.js 957.00b +0.00b 5.35s -273.00ms 🚀
dist/16.dcf139e7.js 951.00b +0.00b 5.62s -281.00ms
dist/16.26c3d518.js 912.00b +0.00b 5.35s -273.00ms 🚀
dist/16.f76b9cae.js 906.00b +0.00b 5.35s -273.00ms 🚀
dist/16.fb327623.js 906.00b +0.00b 5.35s -273.00ms 🚀
dist/16.f2056258.js 905.00b +0.00b 5.35s -273.00ms 🚀
dist/16.4e7dec68.js 904.00b +0.00b 5.35s -272.00ms 🚀
dist/16.f6395317.js 876.00b +0.00b 5.62s -281.00ms
dist/16.24326b68.js 855.00b +0.00b 5.35s -273.00ms 🚀
dist/index.html 248.00b +0.00b 6.38s +870.00ms ⚠️

Cached Bundles

Bundle Size Difference Time Difference
dist/component-lazy.1b33c14d.js 59.63kb +0.00b 6.14s +530.00ms ⚠️
dist/codeViewerRenderer.f99075be.js 2.74kb +0.00b 11.47s +3.00s ⚠️
dist/ro.a6eff34a.js 612.00b +0.00b 8.29s +2.00s ⚠️

Three.js ✅

Timings

Description Time Difference
Cold 3.06s -116.00ms
Cached 328.00ms -3.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

Bundle Size Difference Time Difference
dist/Three.js 572.22kb +0.00b 960.00ms -67.00ms 🚀

Click here to view a detailed benchmark overview.

@mattcompiles
Copy link
Contributor

mattcompiles commented Mar 2, 2023

@lettertwo Curious about your choice to create a DSL for this vs a typed API?

e.g.

fsFixture(overlayFS, __dirname)
  .file(
    "/commonjs/index.js",
    `var local = require('./local');
     // eslint-disable-next-line no-unused-vars
     var url = require('url');

     module.exports = function () {
       return local.a + local.b;
     };`
  )
  .file(
    "/commonjs/local.js",
    `exports.a = 1;
     exports.b = 2;`
  );

A chainable API would also have the benefit of being composable via utility functions.

@lettertwo
Copy link
Member Author

lettertwo commented Mar 2, 2023

@lettertwo Curious about your choice to create a DSL for this vs a typed API?

Well, normally I'd recommend (to myself) against a DSL, but in this case, I think the brevity (and the narrow domain) of the DSL could be benefit.

When i started thinking about this, I couldn't see an API that could improve much over:

overlayFS.chdir(__dirname);
await overlayFS.mkdirp('commonjs');
await overlayFS.writeFile('commonjs/index.js', `...`);

But totally open to including something if you've got more thoughts on what a builder API could look like.

EDIT: Maybe you weren't actually describing a builder API, but rather just a chainable file method. That is probably fairly simple to add, if you think it'd be your preference!

@mattcompiles
Copy link
Contributor

@lettertwo I think there are benefits to both approaches. I definitely wouldn't want to block your good work here, was just curious mostly.

Long term I can see some extra APIs being useful as we might want to perform a similar operation across many fixtures.

E.g. addReact(fixture)

Copy link
Contributor

@mattcompiles mattcompiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Only thing I'm not confident to approve is the memory fs changes. 😅

type Fixture = FixtureRoot | FixtureChild;
type FixtureChild = FixtureDir | FixtureFile | FixtureLink;

export function fsFixture(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel like we should move this to its own package? I think this could be useful in projects outside of parcel eventually.

@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch 2 times, most recently from 9024822 to fb39c4f Compare June 30, 2023 21:23
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch 2 times, most recently from 62c473b to 58af4a1 Compare July 10, 2023 22:08
Content is no longer delimited by double quotes.

Instead, it is matched starting at a colon (:) and over subsequent lines
that have an indent level of 1 more than the filename's indent level.

tl;dr: multline line content is nested under a filename, e.g.,:

  filename:
    content
    over multiple
    lines
Previously, a root dirname like '/' could not be read because
the name would be combined with another path separator, like '//'.
Additionally, the directory listing would include the root directory itself.
Adds a bitmask to compute the type from the mode.

Previously, files would report as both a file type and a symbolic link type.
Could be useful for porting over existing test fixtures
This uses the `path.sep` as the default `cwd`, which fixes
errors on Windows like `FSError: ENOENT: \ does not exist`.

Additionally, it normalizes filepaths to the fully resolved `cwd`,
which on Windows resembles `C:\foo\bar`.
@lettertwo lettertwo merged commit 5478f4a into v2 Aug 16, 2023
16 checks passed
@lettertwo lettertwo deleted the lettertwo/fs-fixture branch August 16, 2023 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants