update documents for new version

Author: Johann150

CHANGELOG.md

@@ -6,6 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+
+## [3.0.0] - 2021-03-27
 Thank you to @ddevault for contributing to this release.
 
 ### Added
@@ -216,7 +218,8 @@ Thank you to @m040601 for contributing to this release.
 
 ## [1.0.0] - 2020-05-21
 
-[Unreleased]: https://github.com/mbrubeck/agate/compare/v2.5.3...HEAD
+[Unreleased]: https://github.com/mbrubeck/agate/compare/v3.0.0...HEAD
+[3.0.0]: https://github.com/mbrubeck/agate/compare/v2.5.3...v3.0.0
 [2.5.3]: https://github.com/mbrubeck/agate/compare/v2.5.2...v2.5.3
 [2.5.2]: https://github.com/mbrubeck/agate/compare/v2.5.1...v2.5.2
 [2.5.1]: https://github.com/mbrubeck/agate/compare/v2.5.0...v2.5.1

Cargo.lock

@@ -1,6 +1,6 @@
 [package]
 name = "agate"
-version = "2.5.3"
+version = "3.0.0"
 authors = ["Matt Brubeck <mbrubeck@limpet.net>", "Johann150 <johann+agate@qwertqwefsday.eu>"]
 description = "Very simple server for the Gemini hypertext protocol"
 keywords = ["server", "gemini", "hypertext", "internet", "protocol"]

Cargo.toml

@@ -38,19 +38,11 @@ Download the source code and run `cargo build --release` inside the source repos
 You can use the install script in the `tools` directory for the remaining steps if there is one for your system. If there is none, please consider contributing one to make it easier for less tech-savvy users!
 ***
 
-2. Generate a self-signed TLS certificate and private key.  For example, if you have OpenSSL 1.1 installed, you can use a command like the following.  (Replace the hostname `example.com` with the address of your Gemini server.)
-
-```
-openssl req -x509 -newkey rsa:4096 -keyout key.rsa -out cert.pem \
-    -days 3650 -nodes -subj "/CN=example.com"
-```
-
-3. Run the server. You can use the following arguments to specify the locations of the content directory, certificate and key files, IP address and port to listen on, host name to expect in request URLs, and default language code(s) to include in the MIME type for for text/gemini files: (Again replace the hostname `example.com` with the address of your Gemini server.)
+2. Run the server. You can use the following arguments to specify the locations of the content directory, IP address and port to listen on, host name to expect in request URLs, and default language code to include in the MIME type for for text/gemini files: (Replace the hostname `example.com` with the address of your Gemini server.)
+If you have not done it yourself, Agate will generate a private key and certificate for you on the first run, using the specified hostname(s). See the section Certificates below for more.
 
 ```
 agate --content path/to/content/ \
-      --key key.rsa \
-      --cert cert.pem \
       --addr [::]:1965 \
       --addr 0.0.0.0:1965 \
       --hostname example.com \
@@ -107,7 +99,7 @@ Agate will use this MIME type instead of what it would guess, if the file is fou
 If a line violates the format or looks like case 3, but is incorrect, it might be ignored. You should check your logs. Please know that this configuration file is first read when a file from the respective directory is accessed. So no log messages after startup does not mean the `.meta` file is okay.
 
 Such a configuration file might look like this:
-```text
+```
 # This line will be ignored.
 **/*.de.gmi: ;lang=de
 nl/**/*.gmi: ;lang=nl
@@ -135,17 +127,59 @@ any non-hidden file in the `nl` directory ending in `.gmi` (including in non-hid
 
 ### Logging Verbosity
 
-Agate uses the `env_logger` crate and allows you to set the logging verbosity by setting the default `RUST_LOG` environment variable. For more information, please see the documentation of `env_logger`.
+Agate uses the `env_logger` crate and allows you to set the logging verbosity by setting the default `RUST_LOG` environment variable. To turn off all logging use `RUST_LOG=off`. For more information, please see the documentation of `env_logger`.
 => https://docs.rs/env_logger/0.8 documentation of `env_logger` crate
 
 ### Virtual Hosts
 
 Agate has basic support for virtual hosts. If you specify multiple `--hostname`s, Agate will look in a directory with the respective hostname within the content root directory.
 For example if one of the hostnames is `example.com`, and the content root directory is set to the default `./content`, and `gemini://example.com/file.gmi` is requested, then Agate will look for `./content/example.com/file.gmi`. This behaviour is only enabled if multiple `--hostname`s are specified.
-Agate does not support different certificates for different hostnames, you will have to use a single certificate for all domains (multi domain certificate).
+Agate also supports different certificates for different hostnames, see the section on certificates below.
 
 If you want to serve the same content for multiple domains, you can instead disable the hostname check by not specifying `--hostname`. In this case Agate will disregard a request's hostname apart from checking that there is one.
 
+### Certificates
+
+Agate has support for using multiple certificates with the `--certs` option. Agate will thus always require that a client uses SNI, which should not be a problem since the Gemini specification also requires SNI to be used.
+
+Certificates are by default stored in the `.certificates` directory. This is a hidden directory for the purpose that uncautious people may set the content root directory to the currrent director which may also contain the certificates directory. In this case, the certificates and private keys would still be hidden. The certificates are only loaded when Agate is started and are not reloaded while running. The certificates directory may directly contain a key and certificate pair, this is the default pair used if no other matching keys are present. The certificates directory may also contain subdirectories for specific domains, for example a folder for `example.org` and `portal.example.org`. Note that the subfolders for subdomains (like `portal.example.org`) should not be inside other subfolders but directly in the certificates directory. Agate will select the certificate/key pair whose name matches most closely. For example take the following directory structure:
+
+```
+.certificates
+|-- cert.pem     (1)
+|-- key.rsa      (1)
+|-- example.org
+|   |-- cert.pem (2)
+|   `-- key.rsa  (2)
+`-- portal.example.org
+    |-- cert.pem (3)
+    `-- key.rsa  (3)
+```
+
+This would be understood like this:
+* The certificate/key pair (1) would be used for the entire domain tree (exceptions below).
+* The certificate/key pair (2) would be used for the entire domain tree of `example.org`, so also including subdomains like `secret.example.org`. It overrides the pair (1) for this subtree (exceptions below).
+* The certificate/key pair (3) would be used for the entire domain tree of `portal.example.org`, so also inclduding subdomains like `test.portal.example.org`. It overrides the pairs (1) and (2) for this subtree.
+
+Using a directory named just `.` causes undefined behaviour as this would have the same meaning as the top level certificate/key pair (pair (1) in the example above).
+
+The files for a certificate/key pair have to be named `cert.der` and `key.der` respectively. The certificate has to be a X.509 certificate in a DER format file and has to include a subject alt name of the domain name. The private key has to be in DER format and must be either an RSA, ECDSA or Ed25519 key.
+If the `--hostname` argument is used, Agate will generate certificates and Ed25519 certificates for each hostname specified.
+
+## Logging
+
+All requests will be logged using this format:
+```
+<local ip>:<local port> <remote ip or dash> "<request>" <response status> "<response meta>"[ error:<error>]
+```
+The "error:" part will only be logged if an error occurred. This should only be used for informative purposes as the status code should provide the information that an error occurred. If the error consisted in the connection not being established (e.g. because of TLS errors), the status code `00` will be used.
+
+There are some lines apart from these that might occur in logs depending on the selected log level. For example the initial "Listening on..." line or information about listing a particular directory.
+
+## Security considerations
+
+If you want to run agate on a multi-user system, you should be aware that all certificate and key data is loaded into memory and stored there until the server stops. Since the memory is also not explicitly overwritten or zeroed after use, the sensitive data might stay in memory after the server has terminated.
+
 # Changelog
 
 All notable changes to this project will be documented in this file.
@@ -154,7 +188,31 @@ The format is based on Keep a Changelog and this project adheres to Semantic Ver
 => https://keepachangelog.com/en/1.0.0/ Keep a Changelog home page
 => https://semver.org/spec/v2.0.0.html Semantic versioning standard v2.0.0
 
-## [Unreleased]
+## [3.0.0] - 2021-03-27
+Thank you to @ddevault for contributing to this release.
+
+### Added
+* Support for ECDSA and Ed25519 keys.
+* Agate now generates certificates and keys for each `--hostname` that is specified but no matching files exist. (#41)
+
+### Changed
+* The ability to specify a certificate and key with `--cert` and `--key` respectively has been replaced with the `--certs` option. (#40)
+  Certificates are now stored in a special directory. To migrate to this version, the keys should be stored in the `.certificates` directory (or any other directory you specify).
+  This enables us to use multiple certificates for multiple domains.
+* The certificate and key file format has been changed from PEM to DER. This simplifies loading certificate and key files without relying on unstable portions of other crates.
+  If you want to continue using your existing certificates and keys, please convert them to DER format. You should be able to use these commands if you have openssl installed:
+```
+openssl x509 -in cert.pem -out cert.der -outform DER
+openssl rsa -in key.rsa -out key.der -outform DER
+```
+  Since agate will automatically generate certificates from now on, the different format should not be a problem because users are not expected to handle certificates unless experienced enough to be able to handle DER formatting as well.
+
+### Fixed
+* Agate now requires the use of SNI by any connecting client.
+* All log lines are in the same format now:
+  `<local ip>:<local port> <remote ip or dash> "<request>" <response status> "<response meta>" [error:<error>]`
+  If the connection could not be established correctly (e.g. because of TLS errors), the status code `00` is used.
+* Messages from modules other than Agate itself are not logged by default.
 
 ## [2.5.3] - 2021-02-27
 Thank you to @littleli and @06kellyjac for contributing to this release.