Skip to content

FlowFuse/file-server

Repository files navigation

A Basic Object Store for use with FlowFuse

Authorisation

All requests should include a Authorization header with a Bearer token assigned by the FlowFuse platform to identify

End Points

File Storage

  • Create/Replace

    POST /v1/files/:teamId/:projectId/[path and filename]

    Content-Type: application/octet-stream

  • Append

    POST /v1/files/:teamId/:projectId/[path and filename]

    With Header FF_MODE: append

    Content-Type: application/octet-stream

  • Read File

    GET /v1/files/:teamId/:projectId/[path and filename]

    Content-Type: application/octet-stream

  • Delete File

    DELETE /v1/files/:teamId/:projectId/[path and filename]

  • Check team quota usage

    GET /v1/quota/:teamId

    Content-Type: application/json

Context Store

  • Set stored values

    POST /v1/context/:projectId/:scope

    Content-Type: application/json

    Body:

    [
        { "key": "x", "value": { "foo": "bar" } },
        { "key": "y.y", "value": 100 },
    ]
  • Get stored values

    GET /v1/context/:projectId/:scope?key=x[&key=y.y]

    Content-Type: application/json

    Response:

    [
        { "key": "x", "value": { "foo": "bar" } },
        { "key": "y.y", "value": 100 }
    ]
  • Get keys for a scope

    GET /v1/context/:projectId/:scope/keys

    Content-Type: application/json

    Response:

    [
        "x",
        "y"
    ]
  • Delete scope

    DELETE /v1/context/:projectId/:scope

  • Clean unused scopes from the store

    POST /v1/context/:projectId/clean

    Content-Type: application/json

    Body:

    [
        "nodeId", "flowId"
    ]

Configuration

Configuration is read from etc/flowforge-storage.yml

host: 0.0.0.0
port: 3001
base_url: http://flowforge:3000
driver:
  type: localfs
  options:
    root: var/root
telemetry:
  backend:
    prometheus:
      enabled: true
  • base_url - Where to reach the core FlowForge platform
  • driver
    • type - can be s3, localfs or memory (for testing)
    • options - will vary by driver
  • telemetry
    • backend
      • prometheus
        • enabled - turns on the /metrics endpoint to track resource usage

File Storage

S3

The following can be any of the options for the S3Client Contructor, see here

  • options
    • bucket - name of S3 Bucket (required)
    • region - AWS Region
    • endpoint - S3 ObjectStore Endpoint (if not using AWS S3)
    • forcePathStyle: true/false
    • credential
      • accessKeyId - AccountID/Username
      • secretAccessKey - SecretKey/Password
host: '0.0.0.0'
port: 3001
base_url: http://forge.default
driver:
  type: s3
  options:
    bucket: flowforge-files
    credentials:
      accessKeyId: XXXXXXXXXXXXXXXXXXX
      secretAccessKey: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    forcePathStyle: true
    region: us-east-1

LocalFS

  • options
    • root - path to store team files, relative path will apply to FLOWFORGE_HOME

Memory

This driver is purely to make testing easier, it has no configuration options.

Context Storage

Sequelize

This driver can use either PostgreSQL or SQLite to hold context values.

To use with PostgreSQL configure as follows:

context:
  type: sequelize
  options:
    type: postgres
    host: 127.0.0.1
    port: 5432
    database: ff-context
    username: user
    password: password

To use with SQLite configure as follows:

context:
  type: sequelize
  options:
    type: sqlite
    storage: ff-context.db

Where context.options.storage is the filename of the SQLite database, by default it will be written to the var directory if a fully qualified path is not provided.

Environment variables

  • FLOWFORGE_HOME default /opt/flowforge-file-storage
  • PORT overrides value in config file, default 3001

Development

Testing

npm run test

NOTE: This will run all tests for all backends and requires a running postgres database

To prepare postgres for the tests, use the following procedure (tested on Linux and WSL2 ubuntu + docker)...

docker run --rm --name postgres -e POSTGRES_PASSWORD=secret -e POSTGRES_USER=postgres -p 5432:5432 postgres:14

Testing (without postgres)

npm run test:nopg

Alternatively, you can set env variable TEST_POSTGRES=false

export TEST_POSTGRES=false
npm run test