TDP-Setup

[PACordonnier]

TL;DR: Installing tdp-lib, collection is hard. We need a way to allow Users to use their own infrastructure through a tdp-lib installer.

TDP Stack 1.0 has been released, it has been followed by TDP Collection 0.1 and TDP-Manager (including lib, server and ui).

Now we are facing the issue of actually installing Collection and Manager. What I mean by that is:

1. I have hosts available, where I want to install TDP Stack
2. I have a host where I decided I will launch Ansible Job from it.
3. This host will serve as a "TDP Master". I want to be able to deploy tdp using the CLI, as well as running the TDP server and UI on it.

As of today there are no easy way to "Install TDP-manager". The current procedure is kinda like:

1. Create an Ansible "environment" including your hosts.ini with the machines you are deploying to.
2. You need to use a compatible version of Ansible. You can setup a venv for this.
3. Clone tdp-collection. Your ansible needs to be configured to use this collection.
4. Clone tdp-lib, server and ui, checkout the tag for the release, do the appropriate pip install or npm install.
5. tdp-lib and server needs to be available in the PATH (we are using venv for that).
6. Create a tdp-vars-overiddes with the changes you want your cluster to be initialized with.
7. If you use airgap cluster, download all the files from TDP Stack 1.0 releases in files directory. You can use the tdp-releases-uris.txt from getting-started. Create a symbolic from the collection to this directory.
8. If you use the default topology, copy the topology from the collection into your inventory. If not, create your own topology.
9. Initialize the tdp_variables (merging tdp_vars_default and your overrides).
10. Initialize the database used by tdp_lib.
11. Setup environment variables used as configuration for tdp_lib, tdp_server, tdp_ui and even for vanilla ansible (tdp_vars).
12. Start the server and the UI

The actively maintained tdp-getting-started do all of this with a combinaison of versioned files (e.g: overrides, use of vagrant), bash setup script (setup submodules, download releases) and tdp-lib's tdp init (initialize database and tdp_vars).

Git clone and checkout as well as doing 'pip install' are not appropriate ways of distributing tdp manager.

I think we need to provide final users (not active tdp maintainers) a way to bootstrap TDP for their environment.

I think the roadmap to achieve this is as follow:

• Distribute tdp-lib, tdp-server 0.1 through pypi. tdp-ui on npm
• Distribute the collection on ansible-galaxy
• create a tdp-setup cli which is does most of the stuff listed above, but leverage distribution of packages on galaxy, pip and npm

I think this feature can be named "TDP-setup" but I would like to consider using an arbitrary name, not related to TDP-*'. I think it needs to be a project on its own (it could be as simple as a bash script). Some features leverage features in tdp lib (such as init), but since TDP-lib has no knowledge of tdp server nor UI. I think having a dedicated TDP-Setup also allows to release the project as a whole since it includes TDP-Stack version (trough release file) and TDP-Manager version. I think the project might be further extended to manage upgrade both for the stack and the manager.

[rpignolet]

I agree on the principle but not the content. The publication on pypi, npm, ansible galaxy is not a pre-requisite. pip, npm, ansible-galaxy are able to use git as source which can allow us to focus on tdp-setup.

Once we are comfortable with the installation, we can look for the publication.

[rpignolet]

TDP Manager uses Ansible to make installing TDP Manager components easier for the user. For me, it is important that the user understands the different components of TDP Manager and making a script that does everything for him does not go in that direction at all.

I agree to script the configuration and initialization of TDP Manager but not prerequisites.

The prerequisites are:

1. A Python runtime compatible with tdp-lib, ansible and tdp-server (Use of a virtual env is recommended)
2. A http web server compatible export Next.js from tdp-ui
3. Configuring the Ansible Controller (ansible configuration, ssh configuration, inventory configuration with topologies)
4. Download TDP's Ansible collections (tdp-collection, tdp-collection-extras, etc.)
5. Downloading binaries from the TDP stack to the Ansible Controller
6. Creating tdp_vars_overrides

All of these prerequisites are related to how each user works. It will be difficult to make a generic script which corresponds to all the uses.

For the rest, an Ansible role could do very well:

1. Initialization of the tdp_vars taking into account the tdp_vars_overrides of the prerequisites
2. Initializing the tdp-lib database
3. Configuring tdp-server (service unit systemd for example)
4. Initializing the tdp-server database
5. Starting tdp-server
6. Configure tdp-ui (nginx configuration for example)

[Edouard-R]

I think the best way to start is to write a shell script and share it in a private repository.