docs: add postgresql and vs code setup pages with example scripts/files

Author: ashenBlade

docs/dev_scripts.md

@@ -0,0 +1,83 @@
+# Development scripts
+
+Most development operations can be effectively optimized using shell scripts. This page describes various scripts that can help in database development and debugging.
+
+During development/debugging you will be working with specific database version and installation, because you do not want to mess everything up with different versions.
+
+We will use the following architecture:
+
+- `./` - project root with all source files
+    - `./build` - binary installation directory: `bin`, `share`, `lib`, `include`
+    - `./data` - directory with database installation (`PGDATA`)
+    - `./scripts` - special development scripts (this section describes them)
+        - `build.sh` - running `configure` script and building database
+        - `run.sh` - managing database installation: create, start, stop, etc...
+        - `clean.sh` - cleaning files: database, build artifacts, etc...
+        - `env.sh` - generated file with exported environmental variables
+
+Each file contains it's own `--help` flag which show available options and usage examples.
+
+> Scripts can be found in [GitHub repository](https://github.com/ashenBlade/postgres-dev-helper/tree/master/misc/scripts).
+
+## `build.sh`
+
+In previous section we discussed how the database is compiled, so this script will do that - run `configure` and build database.
+
+But that's not all. Usually you will work with single server, no multi-server cluster. So it would be more convenient to just run `pg_ctl start` or `initdb` that will work with the specified installation or just type `psql` to quickly connect to server.
+
+So, after `configure` script has executed we will create env file, that will store PostgreSQL special environmental variables.
+
+Taking into an account default values for database/user/password/etc... we can generate env file in the following way:
+
+```sh
+PGINSTDIR="$(pwd)/build"
+./configure --prefix="$PGINSTDIR" # ...
+
+# shell env file
+SHENVFILE="$(pwd)/scripts/env.sh"
+cat <<EOF >"$SHENVFILE"
+export PGINSTDIR="$PGINSTDIR"
+export PGDATA="$(pwd)/data"
+export PGHOST="localhost"
+export PGPORT="5432"
+export PGUSER="postgres"
+export PGDATABASE="postgres"
+export PATH="$PGINSTDIR/bin:\$PATH"
+LD_LIBRARY_PATH="\${LD_LIBRARY_PATH:-''}"
+export LD_LIBRARY_PATH="\$PGINSTDIR/lib:\$LD_LIBRARY_PATH"
+EOF
+```
+
+Using this file you can just source it and then use all binaries inside database installation path that will work with specific database installation.
+
+```bash
+source ./scripts/env.sh
+
+# These commands use binaries under "./build/bin"
+# and work with database in "./data"
+initdb
+pg_ctl start
+```
+
+## `run.sh`
+
+There are 4 main operations with database:
+
+- create
+- start
+- connect using psql
+- stop
+
+Script `run.sh` will support all these operations and will be just a wrapper around: `initdb`, `pg_ctl` and `psql`.
+
+## `clean.sh`
+
+As the name implies, this script is responsible for cleaning.
+
+It cleans:
+
+- build artifacts
+- installation files
+- created database
+
+There is no magic in this file.

docs/img/vscode_setup/build_task_threads_number.png

@@ -5,6 +5,8 @@ This is a VS Code extension to assist PostgreSQL Hackers - source code developer
 ## Table of contents
 
 - [Configuration](./configuration.md)
+- [PostgreSQL setup](./postgresql_setup.md)
+- [Development scripts](./dev_scripts.md)
 - [VS Code setup](./vscode_setup.md)
 - Tutorials:
     - [Create extension](./create_extension.md)

docs/index.md

@@ -0,0 +1,42 @@
+# PostgreSQL setup
+
+## Building from source code
+
+PostgreSQL build is 2 staged: run `configure` script and then `make`.
+
+### `configure`
+
+PostgreSQL uses autoconf to setup special `pg_config.h` header file. It contains lots of macros describing target environment, compiler capabilities, etc...
+
+To create this file you first run `./configure` script and pass various flags.
+The set of flags differs between versions, but for development we can outline required:
+
+```bash
+$ ./configure --prefix=$(pwd)/build \
+              --enable-debug \
+              --enable-cassert \
+              --enable-tap-tests \
+              --enable-depend \
+              CFLAGS="-O0 -g3"
+```
+
+What we used:
+
+- `--prefix=$(pwd)/build` - tells to install all binaries inside current working directory, thus you can safely work with multiple versions of PostgreSQL without disrupting the work of others
+- `--enable-debug` - add debug symbols to result binary. This is required for debugging and variable exploring.
+- `--enable-cassert` - enable `Assert` macros that check state consistency. Otherwise it is easy to violate internal state.
+- `--enable-tap-tests` - enable using of TAP-tests using Perl. For this you may need to install Perl on your system.
+- `--enable-depend` - enable tracking of changed files in your system, so you will rebuild only required files, without rebuilding full project.
+- `CFLAGS="-O0 -g3"` - tell the compiler to:
+    - `-O0` - use lowest optimization level (so we can see all variables)
+    - `-g3` - include as much debug symbols as we can
+
+> Prefer using `-g3` level if you are using PostgreSQL Hacker Helper extension, because it allows to use macro definitions and other features that can significantly improve debug experience.
+
+### `make`
+
+The next step is actual building the sources. This is done much easier - just run `make`.
+
+To speed up compilation you can provide number of parallel threads by `-j [THREADS]` flag (or omit to use all available - `make -j`).
+
+When the building is done run `make install`, so all binaries will be installed in `build` directory (directory that we specified as `configure` step).

docs/postgresql_setup.md

@@ -8,7 +8,7 @@ Here is shown VS Code setup for PostgreSQL debugging.
 
 This is the main extension we are talking about. It significantly simplifies development and debugging of source code.
 
-[Link](https://marketplace.visualstudio.com/items?itemName=ash-blade.postgresql-hacker-helper).
+[Link to the extension](https://marketplace.visualstudio.com/items?itemName=ash-blade.postgresql-hacker-helper).
 
 This is the only extension I recommend installing, because there are no alternatives to it.
 For the further extensions you are free to choose that suit you - no restrictions, just suggestions.
@@ -56,6 +56,8 @@ Moreover, you may have multiple different versions of PostgreSQL installed on yo
 
 ## `launch.json`
 
+> [Link to file on GitHub](https://github.com/ashenBlade/postgres-dev-helper/tree/master/misc/vscode/launch.json)
+
 File `.vscode/launch.json` describes debug session configuration: name, debugger, path to binary/pid to attach, launch args, etc...
 When we are talking about PostgreSQL you should remember that it has multi-process architecture, not multi-threaded, this defines how we start debugging.
 Next, typical configurations for different uses cases will be presented.
@@ -113,7 +115,7 @@ They are separate binaries, so you can launch them directly, but usually they in
 
 We can pass it directly using flags, but a better idea would be to use environment variables, because different binaries can use different flags.
 
-All frontend utilities are located in own `src/bin/UTILITY_NAME` directory and after building each directory contains it's binary.
+After installation all frontend utilities will be located in `PGINSTDIR` - directory specified during database setup, so all you have to do is just pick required binary in it.
 
 For example configuration for `pg_ctl` would be:
 
@@ -125,7 +127,7 @@ For example configuration for `pg_ctl` would be:
             "name": "pg_ctl",
             "type": "cppdbg",
             "request": "launch",
-            "program": "${workspaceFolder}/src/bin/pg_ctl/pg_ctl",
+            "program": "${workspaceFolder}/build/bin/pg_ctl",
             "cwd": "${workspaceFolder}",
             "args": [
                 "status"
@@ -145,12 +147,188 @@ Here we are debugging `pg_ctl status` command (see `"args"`) and pass `PGDATA` e
 
 The value of it can be any, but in the example I suppose that for development purposes your installation in `data` directory in the repository itself.
 
-> A better idea than passing environment variables would be to pass environmental variable *file*.
-> It have 2 benefits against manual specifying:
->
-> 1. This file can be automatically generated during database creation
-> 2. If you have configuration for multiple binaries, then you do not have to enter the same parameters - just pass this env file.
+A better idea than passing environment variables would be to pass environmental variable *file*.
+It have 2 benefits against manual specifying:
+
+1. This file can be automatically generated during database creation
+2. If you have configuration for multiple binaries, then you do not have to enter the same parameters - just pass this env file.
+
+On [Development scripts](./dev_scripts.md#buildsh) page are shown useful development scripts, one of them (`build.sh`) will create special `.env` file, which contains all generated environment variables, so you can just specify this file:
+
+```json
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "pg_ctl",
+            "type": "cppdbg",
+            "request": "launch",
+            "program": "${workspaceFolder}/build/bin/pg_ctl",
+            "cwd": "${workspaceFolder}",
+            "args": [
+                "status"
+            ],
+            "envFile": "${workspaceFolder}/scripts/.env"
+        }
+    ]
+}
+```
+
+### CoreDump
+
+Sometimes SEGFAULT happens or `Assert` test fails. In such cases a coredump will be created (you can enable them using `ulimit -c unlimited` shell command from root).
+
+C/C++ extension has special configuration for debugging CoreDump:
+
+```json
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "pg_ctl",
+            "type": "cppdbg",
+            "request": "launch",
+            "program": "${workspaceFolder}/src/backend/postgres",
+            "cwd": "${workspaceFolder}",
+            "coreDumpPath": "${input:coreDumpPath}"
+        }
+    ],
+    "inputs": [
+        {
+            "id": "coreDumpPath",
+            "type": "promptString",
+            "description": "Enter path to CoreDump"
+        }
+    ]
+}
+```
+
+To start debugging core dump you have to enter path to core dump file in prompt.
+
+> PostgreSQL Hacker Helper also works with core dumps.
 
 ## `tasks.json`
 
-TODO
+> [Link to file on GitHub](https://github.com/ashenBlade/postgres-dev-helper/tree/master/misc/vscode/tasks.json)
+
+Another useful functionality of VS Code are Tasks. In short, Task - is a named command, which can be any shell script.
+
+The schema of task entry is very simple:
+
+```json
+{
+    "label": "Name of task",
+    "detail": "Description of a task",
+    "command": "path/to/command",
+    "args": [
+        "additional",
+        "args",
+        "to",
+        "command"
+    ],
+}
+```
+
+On [Development scripts](./dev_scripts.md) page we have defined some useful scripts and now it's time to integrate them into IDE.
+
+This file can be very large, so here we define the most common and necessary.
+
+### Build
+
+To build we can use `build.sh` script. It accepts `--build` flag, that starts building.
+But it also accepts number of threads to use and we will use it.
+
+```json
+{
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "label": "Build",
+            "command": "${workspaceFolder}/scripts/build.sh",
+            "args": [
+                "--build",
+                "-j",
+                "${input:threads}"
+            ],
+            "detail": "Run build and install DB with all contribs",
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            }
+        }
+    ],
+    "inputs": [
+        {
+            "id": "threads",
+            "type": "pickString",
+            "options": [ "1", "2", "3", "4", "5", "6", "7", "8" ],
+            "default": "8",
+            "description": "Number of threads to use"        
+        }
+    ]
+}
+```
+
+As you can see this task runs our `build.sh` script and passes number of threads to it.
+And number of threads is prompted interactively using quick-pick - you can see example on screenshot below.
+
+![Quick pick with number of threads](./img/vscode_setup/build_task_threads_number.png)
+
+More than that, we also added a `"group"` member. What did it give us? Now we can run build just by using shortcut `Ctrl + Shift + B`. And that's it - build is running!
+
+### Running PSQL
+
+Next step after compilation is a database creation and connecting to it. For this task we have `run.sh` script which manages database instance.
+
+It may seem annoying to constantly run the same task sequence: `Init database` -> `Run database` -> `Run PSQL`. So we will combine all 3 commands into single task.
+
+```json
+{
+    "label": "Run psql",
+    "detail": "Run psql",
+    "command": "${workspaceFolder}/scripts/run.sh",
+    "args": [
+        "--init-db",
+        "--run-db",
+        "--psql"
+    ],
+    "isBackground": true,
+}
+```
+
+Now just after database compilation has completed you just types `psql` in `Run task` prompt and clicks required task. That's it - you are running PSQL and can send queries!
+
+### Start terminal in database environment
+
+Complex tasks are not that simple, so sometimes you have to do everything manually. For such kind of work you must start a new terminal session using `env.sh` script defined earlier. It will setup environment variables in such way, so you will be working with compiled binaries and database instance.
+
+```json
+{
+    "label": "Run terminal",
+    "detail": "Run terminal with imported environment variables specific to environment",
+    "presentation": {
+        "echo": true,
+        "reveal": "always",
+        "focus": true,
+        "panel": "shared",
+        "showReuseMessage": false,
+        "clear": false
+    },
+    "command": "/usr/bin/sh",
+    "args": [
+        "-c",
+        ". ${workspaceFolder}/scripts/env.sh; $SHELL"
+    ],
+    "isBackground": true
+},
+```
+
+After launching this task a new shell with all PG env variables will be created.
+
+### Other scripts
+
+In `tasks.json` file you can find more predefined tasks for VS Code. They include:
+
+- Running tests (using `Ctrl + Shift + T` shortcut)
+- Managing database instance
+- Cleaning files