version: freeze 0.8.1 docs

docs/protocol/specification.md

@@ -103,7 +103,7 @@ When the client sends a [simple query](#simple-query), the server will respond w
 
 ### Pipeline
 
-A pipeline is a type of query that is used to send multiple queries to the server at once, and the server will return the responses in the same order. There is nothing special about the structure of pipelines. Consider [reviewing this section on pipelines](/querying/#pipelines).
+A pipeline is a type of query that is used to send multiple queries to the server at once, and the server will return the responses in the same order. There is nothing special about the structure of pipelines. Consider [reviewing this section on pipelines](/querying#pipelines).
 
 **⚠️ Illegal packet error escape**: If the client incorrectly encodes a pipeline (even though some of the first queries may be encoded correctly), the server will execute the correctly encoded queries and then instead of sending any further responses it will respond with a `0xFF` byte. This is equivalent to [*Query Error 25*](errors#query-errors).
 

docusaurus.config.js

@@ -125,7 +125,7 @@ module.exports = {
           lastVersion: 'current',
           versions: {
             current: {
-              label: '0.8.2',
+              label: '0.8.1',
               path: '/',
             }
           },

versioned_docs/version-0.8.1/13.libraries.md

@@ -0,0 +1,35 @@
+---
+id: libraries
+title: Client drivers
+---
+
+
+## Official drivers
+
+The drivers that are officially maintained can be expected to be feature complete and stable. We currently support the following drivers:
+
+| Language/Framework             | Links                                                                                               | Status   | Package manager command |
+| ------------------------------ | --------------------------------------------------------------------------------------------------- | -------- | ----------------------- |
+| Rust                           | [GitHub](https://github.com/skytable/client-rust), [crates.io](https://crates.io/crates/skytable)   | Stable ✅ | `cargo add skytable`    |
+| NodeJS (JavaScript/TypeScript) | [GitHub](https://github.com/skytable/client-nodejs), [npm](https://npmjs.com/package/skytable-node) | Beta 🏗   | `npm i skytable-node`   |
+
+:::info More lanugages
+The team is always looking to support more languages and we wish we could ship more drivers. But due to limited
+resources we haven't been able to. If you're willing to write a driver (it's super easy to), jump into our chat!
+:::
+
+## Community powered
+
+List of community powered drivers (sorted alphabetically):
+
+| Language/Framework               | Client source code                                            | Upstreamed | License    | Remarks                                                                           |
+| -------------------------------- | ------------------------------------------------------------- | ---------- | ---------- | --------------------------------------------------------------------------------- |
+| **C#/.NET**                      |                                                               | ❗          |            |                                                                                   |
+|                                  | [C# driver](https://github.com/martinmolin/skytable-dotnet)   | ❗          | Apache-2.0 | Available on [NuGet](https://www.nuget.org/packages/Skytable.Client)              |
+| **Go**                           |                                                               | ❗          |            |                                                                                   |
+|                                  | [Go driver](https://github.com/No3371/go-skytable)            | ❗          | Apache-2.0 | Written from scratch                                                              |
+|                                  | [Go driver](https://github.com/satvik007/skytable-go)         | ❗          | Apache-2.0 | Easy for migration from go-redis                                                  |
+| **JavaScript/TypeScript (Node)** |                                                               | ❗          |            |                                                                                   |
+|                                  | [NodeJS Driver](https://github.com/zhangyuannie/skytable.js)  | ❗          | Apache-2.0 | Available on [npm](https://www.npmjs.com/package/skytable.js)                     |
+| **PHP**                          |                                                               | ❗          |            |                                                                                   |
+|                                  | [PHP client driver](https://github.com/hkulekci/skytable-php) | ❗          | MIT        | Can be found on [Packagist](https://packagist.org/packages/hkulekci/skytable-php) |

versioned_docs/version-0.8.1/14.benchmarking.md

@@ -0,0 +1,47 @@
+---
+id: benchmarking
+title: Benchmarking
+---
+
+All of Skytable's components are developed from scratch in-house, which makes it hard (if not impossible) to utilize other benchmarking suites. This is why all of Skytable's release bundles come with the `sky-bench` benchmarking tool. It's important to understand how the tool works to understand what you're benchmarking and what the results mean.
+
+The overall goal with the benchmark tool is to simulate how the database would perform in the real-world and reduce synthetic factors.
+
+:::info
+The ability to simulate different workloads is currently being worked on. This means that across releases, the default workload that the benchmark tool runs may vary. See the release notes to see if the benchmark workload has changed.
+:::
+
+## Benchmark workload
+
+The workload that the engine currently uses (as of v0.8.1) is the following:
+- A model `bench.bench` is created with a primary key (of type `string` and with a column of type `uint8`)
+- Multiple clients are created (simulating "application servers")
+- Queries are run against **unique rows**. This means that unlike `redis-benchmark` **every query group touches a different row, as it would generally do in the real-world**
+- The following queries (collectively a "query group") are run for each unique row:
+  - The row is first created with an `INSERT`
+  - All columns of the row are returned with a `SELECT`
+  - The integer column is incremented with an `UPDATE`
+  - The row is finally removed with a `DELETE`
+- **By default, 1,000,000 rows are created and manipulated**
+- **The time taken for each row to be sent, read back and decoded into a readable form is recorded** (time taken to parse into actual language structures such as maps and lists) towards the total time taken, once again unlike many other benchmark tools
+- In total 4,000,000 queries are run (by default)
+
+:::caution
+The benchmark tool will create a space `bench` and a model `bench` and will completely remove the space and associated data once the benchmark is complete. **Do not use this space!**
+:::
+
+## Running the benchmark
+
+Now that you know how the benchmark engine works, go ahead and benchmark for yourself.
+
+1. Run:
+   ```sh
+   sky-bench --password <root_password>
+   ```
+   **Note**: You can ignore the `--password` argument if you have already set it using the `SKYDB_PASSWORD` environment variable
+2. The benchmark engine will run the full workload (as described earlier)
+3. Witness 4,000,000 queries being executed in real-time. Good luck and enjoy the results!
+
+:::tip
+You can tune the number of threads, connections, rows created and such to your liking to simulate the environment that you think matches your production setting.
+:::

versioned_docs/version-0.8.1/15.deployment.md

@@ -0,0 +1,16 @@
+---
+id: deployment
+title: Deployment
+---
+
+Here are some recommendations for deployment:
+1. **Make sure you have enough memory and storage!** The server will start returning errors when your server runs out of resources, as you'd expect.
+2. **When deploying on docker**:
+   - Try to map the volume to a local path. We've had unwarranted data losses when we accidentally ended up running a `docker system prune`
+   - Make sure you have your networking configured right. For example, if you don't map your ports correctly, you'll not be able to access the database outside docker (without running `docker inspect` to find the IP of the container)
+   - Keep a separate folder for all your docker containers:
+     - Create a directory in your home directory like: `$HOME/docker-data`
+     - Then for every Docker image (whether it is Skytable or any other container) create a subfolder in that directory and map
+       the subfolder as a volume for persistence
+3. **Check your firewall**: You want to be very sure about this! If you're starting your database on a different server (as you should; ideally you keep your database server separate from the host running your application server) then make sure that the ports are open and allowed
+4. **Set up virtual memory and monitoring**: To avoid exhausting resources, set up monitoring on your node and enable virtual memory to temporarily avoid OOMs

versioned_docs/version-0.8.1/16.limits.md

@@ -0,0 +1,37 @@
+---
+id: limits
+title: Limits
+---
+
+We've made every effort to provide a robust querying interface, but there are some **temporary limitations** that we think you 
+should know about. We aim to remove the limitations over the next few releases which we expect should happen fairly quickly.
+
+Skytable's limitations primarily come from a bunch of concerns:
+- **Performance and scalability**: Most of our design decisions are influenced by concerns about performance. For example, it's very hard to efficiently scale multi-column indexes.
+- **Reliability**: how reliable is the execution of the task? If it's like walking on eggshells, then we're not going to implement it (for example, unreliable distributed locking)
+- **Security**: If it can't be run securely, then it's off our list
+
+## Temporary limitations
+
+- **DML in collections are still limited**: You'll be able to easily `INSERT` data into any collections but the manipulations on them are currently limited.:
+  - `SELECT` will return the entire collection and cannot yet return a single element
+  - `UPDATE` can append elements to *non-nested* collections but can't do the same for nested collections
+  - `DELETE` can't remove individual elements
+- **Models cannot be `volatile` yet**. If you've used Skytable before, you'd know that you could previously create `volatile` 
+  models which were used as *caching tables* as in they didn't persist data across restarts. The `volatile` feature has been 
+  temporarily removed because we're working on integrating it with the new storage engine.
+
+:::tip Nested collections will get an upgrade
+We understand that collections are fundamental to the complexity of today's data and hence we're working on this! The only reason our team spends so much time *perfecting* features is because it has match our design philosophy.
+
+> You can have 100 fancy but slow features, or 10 powerful and performant features.
+
+If it scales, we ship it. **We're on it!**
+:::
+
+## Soft limitations
+
+Following Skytable's design philosophy that closely encompasses NoSQL systems, the following soft limitations are set:
+- **Only one index**: Right now, the only index that you can use is the primary index (primary_key -> row). This is due to  concerns about performance and scale
+- **No mass updates**: We consider mass updates, such as setting `counter += 1` to every row in a model with multi-million rows
+to be very slow and bad for performance. Hence, we do not allow mass updates at this time.

versioned_docs/version-0.8.1/2.installation.md

@@ -0,0 +1,101 @@
+---
+id: installation
+title: Installation
+---
+
+Getting started with Skytable involves choosing a mode of installation, downloading any required files and then starting up the database. You can choose to either use:
+
+- [**Native binaries (recommended)**](#native-binaries): This is what is generally recommended for the best performance. You will need to download a bundle and then start the server binary; no expert knowledge required
+- [**Using a Debian package (recommended)**](#debian-package): If you're deploying on Ubuntu or any other Debian based Linux distribution, then consider using this method. Configuration files, users and passwords are autogenerated.
+- [**A Docker image**](#docker-image): We generally recommend using a Docker image for experimenting with Skytable on your local system during development and you want to keep your local system *clean*. If you want to use a Docker image for deployment, you're always free to do so!
+  > **Note:** You might experience slightly degraded performance from the storage engine due to Docker engine's abstractions.
+
+:::tip
+All client tools (such as `skysh` and `sky-bench`) *can* use the `SKYDB_PASSWORD` variable for authentication. If you're using Skytable in a testing environment and frequently need to use `skysh`, you may consider setting this variable to your password to avoid having to pass the `--password` argument every time.
+
+However, we strongly recommend **not** using it outside testing environments.
+:::
+
+## Native binaries
+
+To use native binaries you need to download a bundle which is simply a ZIP file with all the necessary binaries that you'll ever need to develop on and deploy Skytable.
+
+1. **First download the latest bundle** for your platform. You can find [download links on the releases page](https://github.com/skytable/skytable/releases).
+2. **Unzip the ZIP file**. You'll find the following binaries in the extracted archive:
+   - `skyd`: This is the database server binary which when started runs as a daemon, serving requests
+   - `skysh`: This is the Skytable shell and it provides a very helpful interactive REPL database client
+   - `sky-bench`: This is the benchmarking tool that you can use to load test Skytable
+3. **Start up the server**. You need to choose a `root` password for the `root` account which will have complete control over the database.
+
+    ```bash
+    ./skyd --auth-root-password=<your root password>
+    ```
+
+    **Replace with your own secure password!**
+
+    Explanation:
+    - `--auth-root-password`: sets the root password
+
+The server starts up at `localhost:2003` and is ready to run queries.
+
+:::info
+Your operating system might sometimes not let you run binaries directly. On Unix based systems, you'll need to run: `chmod +x skyd skysh sky-bench`.
+
+And on Windows systems you might need to right-click on the binaries and click on "unblock"
+:::
+
+## Debian package
+
+Find the correct `*.deb` file [from the releases page](https://github.com/skytable/skytable/releases). Now simply run:
+
+```sh
+sudo dpkg -i <file name>.deb
+```
+
+The package will:
+
+- **Generate a root password:** Watch the terminal output!
+- **Create a `systemd` unit**: So you can start and stop the process using `systemd` like `systemd start skyd`
+- **Generate a configuration**: Your configuration is stored in `/var/lib/skytable/config.yaml`. Go ahead and modify it if you need to!
+
+## Docker image
+
+:::info You must have docker set up!
+
+- Use [this great guide from Docker](https://docs.docker.com/engine/install/) to install and get started
+- To be able to run `docker run` and related commands, you may need administrative privileges
+:::
+
+### Simple setup
+
+1. **Download the bundle**: To be able to run queries you need to download the bundle as described above
+2. **Start the container**:
+
+    ```shell
+    docker run -d --name skydb -p 2003:2003 skytable/skytable:latest
+    ```
+
+:::tip
+The password for the Skytable instance on the Docker container is auto-generated. Run `docker logs -f skydb` and you'll see a log
+message with the generated password.
+:::
+
+### With persistence
+
+1. **Download the bundle**: To be able to run queries you need to download the bundle as described above
+2. **Create the data directory**: To ensure that our database is persistent and all our data doesn't vanish as soon as the container is terminated, we'll map the data directory to an actual directory on our local system.
+    > **Note:** Create a folder called `skytable` in a convenient location. We recommend having a directory in `$HOME/docker-containers` where you can store the Skytable container's data and any other containers that you might use. It's a great way to keep things organized.
+3. **Create your configuration**: [Download this template file](https://raw.githubusercontent.com/skytable/skytable/next/examples/config-files/template.yaml) and place it into the directory you created. Update the password with your `root` password of choice.
+4. **Start the container**:
+
+     ```shell
+     docker run -d --name skydb \
+        -v $HOME/docker-containers/skytable:/var/lib/skytable \
+        -p 2003:2003 \
+        skytable/skytable:latest
+     ```
+
+     Explanation:
+     - This starts a container with name `skydb`
+     - It maps the folder (as discussed earlier) `$HOME/docker-containers/skytable` from your local file system to `/var/skytable` (in the container's file system)
+     - Maps port `2003` on the host to the containers port `2003` so that you can use the command-line client `skysh` without having to inspect the container's IP address