v1.8.10

Changes

A bunch of relatively small stuff:

• Use MailerSend instead of Postmark in the starter app.
• Upgrade htmx version in starter app to 1.9.10 (thanks @behrica).
• Add com.biffweb/test-xtdb-node function and add more detailed tests to the starter app.
• Make Dockerfile set :biff/host to 0.0.0.0, and add a PORT env var for overriding :biff/port.
• Move documentation to a separate repo: https://github.com/jacobobryant/biffweb.com
• Make Tailwind installation more robust.
• Make merge-context a no-op if XTDB has been removed.
• Change #inst "1970" to (java.util.Date.) in use-beholder.
• Make example XTDB_JDBC_URL value more descriptive.
• Rename use-xt -> use-xtdb, use-tx-listener -> use-xtdb-tx-listener

Upgrade instructions

• Update your Biff dependencies to :git/tag "v1.8.10", :git/sha "146f2b1"

Optional

• If you'd like to use MailerSend instead of Postmark, see 4cd2808 and cc7dcdf. MailerSend is cheaper and has a free tier with 3k emails/month. It's also a bit easier to set up in some cases.
• If you'd like to write automated tests that interact with XTDB, see 146f2b1.

v1.8.1

Changes

• Add biff/use-htmx-refresh component which auto-refreshes the browser whenever you save a file. If there are compilation errors, displays those instead of refreshing.

Upgrade instructions

• Update your Biff dependencies to :git/tag "v1.8.1", :git/sha "93baa98"
• Apply these changes to your project: set the :biff.refresh/enabled config option to true in dev, and add biff/use-htmx-refresh to your components, right before the biff/use-jetty component.

v1.8.0

See the announcement post for commentary.

Changes

• Use Aero to parse config.
• Switch from bb tasks to plain clj-based tasks.
• When using :db.op/upsert on a document that doesn't yet exist, use the provided :xt/id value if there is one instead of always using (random-uuid) as the value.
• Make use-xtdb get the value of :biff.xtdb.jdbc/jdbcUrl from the :biff/secret function if it's defined as a secret.
• Documentation fixes/improvements.
• Make server-setup.sh delete Trenchman artifacts after installing.
• Add libs to forward Java logs to slf4j.
• Update cider-nrepl to 0.45.0

Thank you to @mathisto, @carlos, @olavfosse, and @wunki for contributions.

Upgrade instructions

• Check the upgrade instructions for any previous releases, in case there are optional upgrades you'd like to apply.
• Update your Biff dependencies (in deps.edn, bb/deps.edn and/or tasks/deps.edn) to :git/tag "v1.8.0", :git/sha "3b2fa14"

Optional: upgrade config to use Aero

Apply the changes in this commit to your project:

1. Copy the new resources/config.edn file into your project. (NOTE: this new file should be saved at resources/config.edn, not config.edn).
2. Copy the new resources/config.template.env file into your project. Copy it to both resources/config.template.env and config.env.
3. Add /config.env to your .gitignore file.
4. Edit resources/config.edn and config.env to include any custom config that you previously put in config.edn or secrets.env. For example, you'll need to copy the values of COOKIE_SECRET and JWT_SECRET from secrets.env into config.env. resources/config.edn will be parsed with Aero. To specify different config values for different environments, you'll do e.g. {:example/foo #profile {:prod "bar" :dev "baz"} ... instead of {:prod {:example/foo "bar" ...} :dev {:example/foo "baz" ...} .... Here's some more migration advice.
5. Test your config. Add the check-config function to your repl.clj file. Then run bb dev and evaluate (check-config) from your editor. (You can also do (biff/pprint (check-config)) if needed.) Make sure everything looks correct.
6. Run git add .; git commit.
7. If your app is already deployed, run bb deploy and then bb prod-repl. Evaluate (check-config) again to make sure the new config is available in production.
8. Edit your main namespace and replace biff/use-config and biff/use-secrets with biff/use-aero-config. Run bb dev and make sure everything works locally. Then make a new commit, run bb deploy, and ensure everything works in production.

NOTE: this will make your application use the new Aero config, however, bb tasks will still read from the old config.edn file. To completely upgrade, you'll need to switch to clj tasks.

Optional: switch from bb tasks to clj tasks

If you switch to clj tasks, you must first upgrade your config to use Aero as described above.

Apply the changes in this commit to your project:

1. Add the :dev and :prod aliases to your deps.edn file. Replace :local/root "../libs/tasks" with :git/url "https://github.com/jacobobryant/biff", :git/tag "v1.8.0", :git/sha "3b2fa14", :deps/root "libs/tasks" and change com.example to your main namespace.
2. Add the dev/tasks.clj file to your project.
3. Edit your -main and start functions like so.
4. If you set up a server with server-setup.sh, SSH into it as root, edit /etc/systemd/system/app.service, and change the ExecStart line to ExecStart=/bin/sh -c "mkdir -p target/resources; clj -M:prod". Then run systemctl daemon-reload. Also make the same change in your server-setup.sh file. (Note: new projects also have server-setup.sh set BIFF_PROFILE=prod instead of BIFF_ENV=dev, but BIFF_ENV is still recognized.)
5. Run clj -M:dev dev to make sure everything works locally, then (if you did step 4) run clj -M:dev deploy and make sure it works in prod.
6. Recommended: add alias biff='clj -M:dev' to your .bashrc file.

Run clj -M:dev --help (or biff --help if you set up the alias) for a list of available commands. e.g. instead of running bb dev and bb deploy, you'll now run clj -M:dev dev and clj -M:dev deploy.

Optional: make your app work with Docker/Uberjars

After you upgrade to clj tasks, you can run clj -M:dev uberjar to generate an Uberjar for your app. You can also add this Dockerfile to your app. The recommended way for most people to deploy Biff is still the old way (setting up a DigitalOcean droplet with server-setup.sh and deploying with clj -M:dev deploy). But for those who know they want to deploy another way, clj -M:dev uberjar and the Dockerfile are provided as a convenience.

You'll need to make a few additional changes to your project. See this commit:

• Add (:gen-class) to your main namespace.
• Update css-path and js-path so they don't break when called inside an Uberjar.
• Add .dockerignore to your project.

Optional: clean up

If you followed the instructions above, there will be several files and directories that you no longer need. Feel free to delete them:

• bb.edn
• bb/ (previously this folder was named tasks/)
• config.edn and secrets.env (these aren't checked into git by default, so be sure you've migrated all your config over to the new files before you delete them)

Additionally, you can move your repl.clj file into the new dev directory and your test.clj file into a new test directory. You'll also need to edit deps.edn and your on-save function—see this commit.

Optional: rename "plugins" to "modules"

This is a purely cosmetic change, but if you'd like to stay in line with new projects, the change is a simple find-and-replace.

v0.7.15

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.7.15", :sha "b730c85", ...}
• Update your Biff dependency in bb/deps.edn to {:tag "v0.7.15", :sha "b730c85", :deps/root "tasks", ...}

Changes

• Refactored bb deploy and other tasks so that it attempts to start an ssh-agent session if one isn't already running. This way, your SSH key decryption password will only be requested once per command. If Biff is unable to start an ssh-agent session, bb deploy will only ask for your password twice (in most cases). These changes also fix a bug in v0.7.11 where bb deploy was completely broken if you didn't already have an ssh-agent session running.
• Add a :biff.tasks/skip-ssh-agent config option which, if set to true, will disable the ssh-agent functionality described above.
• Fix regressions in multiple commands, including bb dev and bb deploy, so that they work again on Windows (without WSL).
• Fix a bug in bb css (and several other commands by extension, like bb deploy) where it crashed if you didn't have npm on your path.
• Check if the downloaded tailwindcss binary is corrupted/incomplete, and prompt the user to delete it if so. #166
• Fix a bug in the authentication plugin where the 6-digit code flow didn't correctly update new users' sessions. #167 (Thanks @Invertisment)

v0.7.11

Update 16 Sep 2023:

There were a few regressions in this release. I've updated the upgrade instructions below to have you upgrade to a commit with the fixes. I'll write up separate release notes for that soon.

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.7.11", :sha "6428c7f", ...} {:tag "v0.7.14", :sha "aad7173", ...}
• Update your Biff dependency in bb/deps.edn to {:tag "v0.7.11", :sha "6428c7f", :deps/root "tasks", ...} {:tag "v0.7.14", :sha "aad7173", :deps/root "tasks", ...}

Optional

• Move the default middleware stack into your project so that it's easier to modify and debug: d8c83c4
• Move the default on-error handler into your project so that it's easier to modify: 7622c40
• Add a main.js file to your project for cases where you need more client-side logic than what hyperscript can comfortable handle: 2e943bb

Added

• The com.biffweb/s3-request function is used for uploading/downloading files to/from S3-compatible services. Adapted from the image upload howto.

Changes

• bb deploy now uses rsync (if available) instead of git push. Closes #164, #155
• Improved logic in bb css / bb dev for selecting the right Tailwind command and installing the standalone binary.
• com.biffweb/eval-files! now returns the result from clojure.tools.namespace.reload/track-reload instead of nil, which will be helpful for #117.
• Template project changes (discussed above in Upgrade instructions > Optional).

Fixed

• bb soft-deploy now works on Mac. Closes #152
• The authentication plugin no longer blocks occasionally when generating sign-in codes. Closes #163
• Various documentation updates.
• Fixed a websocket-related bug in the tutorial where messages weren't received if you open the same chat channel with the same user in two different tabs.

v0.7.9

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.7.9", :sha "7890f70", ...}
• Update your Biff dependency in tasks/deps.edn to {:tag "v0.7.9", :sha "7890f70", :deps/root "tasks", ...}

Optional

• Rename the tasks directory to bb, and change the dependency in bb.edn to {:local/root "bb"}. Otherwise, the bb tasks command fails (at least on recent versions of bb--I think it worked previously).

Changes

• Make Tailwind installation more robust
• Add bb css task
• Rename tasks directory to bb in new projects
• Replace optional :biff.tasks/tailwind-build config option with :biff.tasks/tailwind-file in new projects
• Add support for chime error handlers (#160 -- thanks @LucianoLaratelli)
• Make secrets parsing in dev consistent with prod (#159 -- thanks @rads)

v0.7.4

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.7.4", :sha "3ff1256", ...}
• Update your Biff dependency in tasks/deps.edn to {:tag "v0.7.4", :sha "3ff1256", :deps/root "tasks", ...}

Optional

To upgrade, apply the changes in the given commits to your project manually. These changes are all stylistic and mainly beneficial for new projects; feel free to skip them.

• Stop using biff/system, biff/start-system, and biff/refresh, which are now deprecated: 9bcd672
• Stop using biff/use-when: ae18d26
• Stop using biff/use-wrap-ctx (its functionality has been moved into biff/use-jetty): c7349ff
• Stop putting feat in your namespaces: b2417d3
• Rename features to plugins: f5d1021
• Rename sys, req, and opts to the more general ctx (short for "context"): 7d00391

Other changes

• Added :biff.auth/get-user-id option to the auth plugin (thanks @momerath42)
• Updated server-setup.sh to install additional deps that aren't always included in Ubuntu installations (thanks @KarolisL)
• Updated Get Started and Reference documentation. The documentation has also been moved into the Biff repository.
• Updated some dependencies, including XTDB 1.23.0 -> 1.23.1.
• Made the default email validator function in the auth plugin disallow emails with spaces in them.
• bb soft-deploy (and by extension, bb prod-dev) now correctly pushes all files that are tracked in git instead of a hard-coded list.
• A new :biff.tasks/on-soft-deploy config option replaces the old :biff.tasks/soft-deploy-fn option (see the first commit referenced above).

Documentation changes

• Documentation source has been moved into the Biff repository.
• Touched up and consolidated the Get Started section.
• "Project Structure" and "System Composition" have been replaced with Architecture.
• Schema now recommends defining attribute schema inside the document schema.
• htmx includes a new Beyond htmx section.
• Authentication has been renamed to Security and now includes a discussion of authorization, CSRF and CORS.
• New pages: Configuration and Babashka tasks.
• Production has been slightly reorganized and now mentions DigitalOcean uptime checks, workers, and container-based deployment.

v0.7.0

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.7.0", :sha "6a5ba0a", ...}
• Update your Biff dependency in tasks/deps.edn to {:tag "v0.7.0", :sha "6a5ba0a", :deps/root "tasks", ...}
• Optional: follow the authentication plugin upgrade instructions below.

Email code signin

In addition to the existing email link authentication flow, where users sign in by clicking a link in an email, Biff can now send
six-digit codes via email. After submitting their email address, users enter the codes into the same browser tab. While signin links generally have a bit less friction, they do sometimes have problems on mobile devices. PWAs on iOS don't share cookies with the main browser, so signin links don't work at all for those. In addition, mobile email clients often open signin links in an embedded browser. The user is authenticated successfully, but if they visit your site in the main browser again, they'll be unauthenticated.

The code authentication flow solves these problems, since the entire flow happens in the same browser tab. You can use either signin links or signin codes, or a combination. The example project uses links for creating an account and codes for signing in to an existing account.

Authentication plugin

Previously, the backend routes for authentication were defined in your project's code, in an auth.clj file. Biff now provides an authentication-plugin function, which can be included with the rest of your app's features:

(def features
  [app/features
   (biff/authentication-plugin {})
   home/features
   schema/features
   worker/features])

This replaces the old auth.clj file. UI and email templates are still defined in your project's code, so they're easily customizable.

See the reference docs and API docs for more details.

Upgrading

Follow the changes in this commit:

1. Remove the auth.clj file and replace it with the biff/authentication-plugin function.
2. Copy the new email.clj file into your project. If you're not using Postmark, rewrite the send-postmark function to use your chosen email provider. Add a :biff/send-email #'email/send-email line to your system map.
3. Update the malli-opts var so that it includes schema defined in your feature maps. Update schema.clj so that it exposes its schema via a feature map.
4. Update the pages in home.clj so they use the new /auth/send-link, /auth/send-code and other routes.

Changed

• Remove Java 11 check from new project script. It was too brittle, and most people are probably on at least Java 11 by now anyway.
• Use SecureRandom/getInstanceStrong for generating new project secrets.
• The tutorial now instructs you to create your project with bb -e "$(curl -s https://biffweb.com/new-project.clj)" tutorial, which ensures that your project is based on the same version of Biff that the tutorial was written against.

Fixed

• The com.biffweb/wrap-internal-error function actually works now.

v0.6.0

Upgrade instructions

• Follow the upgrade instructions for any previous releases.
• Update your Biff dependency in deps.edn to {:tag "v0.6.0", :sha "b3027b9", ...}
• Update your Biff dependency in tasks/deps.edn to {:tag "v0.6.0", :sha "b3027b9", :deps/root "tasks", ...}
• Optional: follow the secrets management upgrade instructions below.
• Optional: apply these changes to use Biff's reorganized middleware.
• Optional: apply this change so that htmx requests will include the CSRF token even when they aren't inside a (biff/form ...).

Secrets management

In new projects, secrets are now stored in a secrets.env file instead of being kept in config.edn with the rest of your configuration. There is a new biff/use-secrets component which sets the :biff/secret key to a function which takes a single argument and returns the corresponding secret. For example, if secrets.env and config.edn contain the following:

# secrets.env
export POSTMARK_API_KEY=abc123

;; config.edn
{:prod {:postmark/api-key "POSTMARK_API_KEY"
...

Then calling (secret :postmark/api-key) would return "abc123".

This makes secrets management pluggable. If you wanted to store secrets somewhere else (like Hashicorp Vault), you could provide your own :biff/secret function. A second benefit is that it's easier to not shoot yourself in the foot--previously if you e.g. logged the value of the system map, the logs would contain all your secrets.

Upgrading

Apply the changes in this commit to your project:

1. Remove the old biff/use-random-default-secrets component, and add the new biff/use-secrets component.
2. Create a secrets.env file containing all your secrets. Be sure to include export on each line, otherwise they won't work in production.
3. Edit config.edn and change the values of any secrets to be the name of the relevant environment variable (e.g. change :postmark/api-key "abc123" to :postmark/api-key "POSTMARK_API_KEY").
4. Update your Clojure files to use the new :biff/secret function whenever you need to access a secret.
5. If you've written custom tasks for bb dev, bb soft-deploy, bb deploy, or bb run-cmd, apply these changes.
6. Add secrets.env to .gitignore

If you need different versions of a secret for dev and prod, you can store them in separate environment variables:

;; config.edn
{:prod {:stripe/secret-key "STRIPE_SECRET_KEY"
        ...}
 :dev {:stripe/secret-key "STRIPE_TEST_SECRET_KEY"
       ...}}

# secrets.env
export STRIPE_SECRET_KEY=foobar
export STRIPE_TEST_SECRET_KEY=bazquux

Changed

• Biff's middleware has been simplified and made more REPL-friendly. All of the middleware can now be modified at runtime without needing to call biff/refresh.
• XTDB has been upgraded from v1.22.1 to v1.23.0 (see the release notes).
• Minor cleanup in the example project.

v0.5.7

Quick fix for a regression in v0.5.6, with a couple more small things thrown in since I can't help myself.

Upgrade instructions:

• Follow the upgrade instructions for any previous releases.
• Optionally, update your Biff dependency in deps.edn and tasks/deps.edn to {:tag "v0.5.7", :sha "02b6bd5", ...}. (This isn't actually necessary since this release only touches files outside of the library code, but you can update the dependency as a reminder that you've followed these instructions if you like.)
• Change local/deps in bb.edn to a unique name for your project, e.g. com.example/tasks (see e0fdc8b).
• Optionally, add "-J-XX:+CrashOnOutOfMemoryError" to your :biff.tasks/clj-args setting (see 02b6bd5).

Fixed:

• Java version detection in new-project.clj is actually more reliable now.
• New projects now get a unique name for the bb tasks dep, which fixes the stale classpath/Could not find namespace errors which sometimes occurred when running tasks.

Added:

• The :biff.tasks/clj-args option in config.edn now includes "-J-XX:+CrashOnOutOfMemoryError" by default. This will cause apps in production to restart in the case of an OOM instead of continuing to run in a possibly partially broken state. It will also save the stack trace to a file.