diff --git a/docs/day1/architecture.rst b/docs/day1/architecture.rst deleted file mode 100644 index 70cb565f..00000000 --- a/docs/day1/architecture.rst +++ /dev/null @@ -1,151 +0,0 @@ -============ -Architecture -============ - -Please familiarize yourself with the `architecture of Dataverse `_ -if not already done: it helps a lot knowing how things are connected in Dataverse -to also understand using it as a Kubernetes application. - -In this chapter you may find detailed documentation about how things are -connected together in this Kubernetes application in a visual way (UML sequence diagrams). -In doubt consult the scripts and descriptors in this repository. - -Initial Deployment Procedure ----------------------------- - -The below image shows all necessary steps by "you" (the user activity on the left) -or (preferably) your deployment framework (like Kustomize.io, Helm or similar) -on your behalf for a new deployment of Dataverse. It also explains what happens -in the background on an overview level. For more details please look at the demos or code. - -.. uml:: - :width: 100% - - @startuml - !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" - - actor User - participant "<$secret>\nSecrets" as S - participant "<$cm>\nConfigMap" as CM - participant "<$pod>\nPostgreSQL" as P - participant "<$pod>\nDataverse" as D - participant "<$job>\nBootstrap Job" as BJ - participant "<$job>\nConfigure Job" as CJ - participant "<$pod>\nSolr" as Solr - - == Deploy application == - - create S - User -> S: Deploy Secrets - create CM - User -> CM: Deploy ConfigMap - note over P: Optional! - create P - User -> P: Deploy PostgreSQL - CM -> P: Pass username +\ndatabase name - S -> P: Pass password - P -> P: Init database - - create Solr - User -> Solr: Deploy Solr from iqss/solr-k8s - Solr -> Solr: Init container:\nFix volume permissions - - create D - User -> D: Deploy Dataverse from iqss/dataverse-k8s - D -> D: Init container:\nFix volume permissions - D <<-->> P: wait for - D <<-->> Solr: wait for - D -> D: Deploy app - note right: see also in detail at\n"Container Startup" - D -> P: Persistance Framework:\nCreate tables - P --> D: Done - - == Bootstrapping == - - create BJ - User -> BJ: Deploy Bootstrapping Job - S -> BJ: Pass db password\n+API key - CM -> BJ: Pass settings - BJ <<-->> P: wait for - BJ <<-->> Solr: wait for - BJ <<-->> D: wait for - ...After Dataverse, Solr and PostgreSQL have been deployed successfully...... - BJ -> P: Additional SQL init - BJ -> D: Bootstrapping w/ setup-all.sh\n(Metadata, user, root dataverse, ...) - activate D - BJ -> D: Configure Solr location\n+ admin contact - BJ -> D: Block API with unblock-key - D -> P: Store settings - return - destroy BJ - - == Further Configuration == - create CJ - User -> CJ: Deploy Configure Job - S -> CJ: Pass API key - CM -> CJ: Pass settings - CJ <<-->> D: wait for - ...After Dataverse ready...... - CJ -> D: Configure Dataverse DB-based\nsettings via API - activate D - D -> P: Store settings - return - destroy CJ - - == Start using == - User -> D: Start accessing Dataverse - @enduml - - - - - -Dataverse Container Startup ---------------------------- - -When the Kubernetes pod containing the application server container starts, -one of the following happens, dependent on the type of image you are using. - -Glassfish release flavor -^^^^^^^^^^^^^^^^^^^^^^^^ -This happens when using the image `iqss/dataverse-k8s `_ or a derived image. - -.. uml:: - - @startuml - !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" - - participant "<$pod>\nContainer" as K - participant Tini - note right Tini: "Tiny init"\ngithub.com/krallin/tini - participant "Entrypoint" as E - participant "Init script" as I - participant "Appserver" as A - - create Tini - K -> Tini: Start - - create E - Tini -> E: Start - create I - E -> I: Start - - create A - I -> A: Start - activate A - I -> A: Configure password aliases - I -> A: Configure keys for S3 - I -> A: Configure resources - I -> A: Configure Dataverse\nJVM options - I -> A: Stop - destroy A - I -> I: Symlink WAR & more - - create A - E -> A: Start in foreground - activate A - E --> Tini: exec(): replace with Appserver - destroy E - Tini -> A: Keep running until container stops - A -> A: Autodeploy WAR - @enduml diff --git a/docs/day1/config.rst b/docs/day1/config.rst index d355d1cb..94cb0570 100644 --- a/docs/day1/config.rst +++ b/docs/day1/config.rst @@ -123,6 +123,34 @@ is a bad idea. It's always a good idea to put it in revision control. You might consider providing a `CronJob` for scheduled, regular updates. +Details of the configuration job +################################ + +.. uml:: + + @startuml + !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" + + actor User + participant "<$secret>\nSecrets" as S + participant "<$cm>\nConfigMap" as CM + participant "<$pod>\nPostgreSQL" as P + participant "<$pod>\nDataverse" as D + participant "<$job>\nConfigure Job" as CJ + + create CJ + User -> CJ: Deploy Configure Job + S -> CJ: Pass API key + CM -> CJ: Pass settings + CJ <<-->> D: wait for + ...After Dataverse ready...... + CJ -> D: Configure Dataverse DB-based\nsettings via API + activate D + D -> P: Store settings + return + destroy CJ + @enduml + Alternative approaches ###################### diff --git a/docs/day1/index.rst b/docs/day1/index.rst index 692d790a..fc8afb11 100644 --- a/docs/day1/index.rst +++ b/docs/day1/index.rst @@ -7,7 +7,8 @@ Day 1 - Deployment :caption: Contents: :hidden: - ./architecture + ./init-deploy + ./job-bootstrap ./config ./secrets ./resources diff --git a/docs/day1/init-deploy.rst b/docs/day1/init-deploy.rst new file mode 100644 index 00000000..722c4169 --- /dev/null +++ b/docs/day1/init-deploy.rst @@ -0,0 +1,56 @@ +================== +Initial Deployment +================== + +Please familiarize yourself with the `architecture of Dataverse `_ +if not already done: it helps a lot knowing how things are connected in Dataverse +to also understand using it as a Kubernetes application. + +The below UML sequence diagram shows all necessary steps by "you" (the user activity on the left) +or (preferably) your deployment framework (like Kustomize.io, Helm or similar) +on your behalf for a new deployment of Dataverse. It also explains what happens +in the background on an overview level. + +When you are done with the initial deployment, you have to :doc:`bootstrap ` +and :doc:`configure `. + +Maybe you should just read :doc:`/get-started/index` and follow that before going into details. + +.. uml:: + + @startuml + !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" + + actor User + participant "<$secret>\nSecrets" as S + participant "<$cm>\nConfigMap" as CM + participant "<$pod>\nPostgreSQL" as P + participant "<$pod>\nDataverse" as D + participant "<$pod>\nSolr" as Solr + + create S + User -> S: Deploy Secrets + create CM + User -> CM: Deploy ConfigMap + note over P: Optional! + create P + User -> P: Deploy PostgreSQL + CM -> P: Pass username +\ndatabase name + S -> P: Pass password + P -> P: Init database + + create Solr + User -> Solr: Deploy Solr from iqss/solr-k8s + Solr -> Solr: Init container:\nFix volume permissions\nDeploy schemas + + create D + User -> D: Deploy Dataverse from iqss/dataverse-k8s + D -> D: Init container:\nFix volume permissions + D <<-->> P: wait for + D <<-->> Solr: wait for + D -> D: Deploy app + note right: see also in detail at\n"Container Startup" + D -> P: Persistance Framework:\nCreate tables + P --> D: Done + + @enduml diff --git a/docs/day1/job-bootstrap.rst b/docs/day1/job-bootstrap.rst new file mode 100644 index 00000000..e4bfb861 --- /dev/null +++ b/docs/day1/job-bootstrap.rst @@ -0,0 +1,46 @@ +============= +Bootstrap Job +============= + +After deploying every components of Dataverse on Kubernetes for the first time +(see :doc:`./init-deploy`), you will need to bootstrap your installation. +That will create a superadmin user, root dataverse and block important API endpoints. + +It will also set the option ``:SolrHostColonPort``, configuring where Dataverse +can find the Solr Search index. It will default to ``solr:8983``, but can be +overridden by setting a hostname or IP in ``SOLR_K8S_HOST`` via ``ConfigMap`` +(see :doc:`config`). + +.. uml:: + + @startuml + !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" + + actor User + participant "<$secret>\nSecrets" as S + participant "<$cm>\nConfigMap" as CM + participant "<$pod>\nPostgreSQL" as P + participant "<$pod>\nDataverse" as D + participant "<$job>\nBootstrap Job" as BJ + participant "<$pod>\nSolr" as Solr + + create BJ + User -> BJ: Deploy Bootstrapping Job + S -> BJ: Pass db password\n+API key + CM -> BJ: Pass settings + BJ <<-->> P: wait for + BJ <<-->> Solr: wait for + BJ <<-->> D: wait for + + ... After Dataverse, Solr and PostgreSQL have been reached successfully... ... + + BJ -> P: Additional SQL init + BJ -> D: Bootstrapping w/ setup-all.sh\n(Metadata, user, root dataverse, ...) + activate D + BJ -> D: Configure Solr location\n+ admin contact + BJ -> D: Block API with unblock-key + D -> P: Store settings + return + destroy BJ + + @enduml diff --git a/docs/images/index.rst b/docs/images/index.rst index a6648222..48ef044d 100644 --- a/docs/images/index.rst +++ b/docs/images/index.rst @@ -21,3 +21,59 @@ images to be used for Dataverse deployment and maintenance. build dataverse-k8s solr-k8s + +Container Startup +----------------- + +Solr Search index +^^^^^^^^^^^^^^^^^ + +Described in detail at :doc:`/day2/job-index`. + +Dataverse Application Server +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When the Kubernetes pod containing the application server container starts, +one of the following happens, dependent on the type of image you are using. + +The following happens when using the :doc:`dataverse-k8s` or a derived image. + +.. uml:: + + @startuml + !includeurl "https://raw.githubusercontent.com/michiel/plantuml-kubernetes-sprites/master/resource/k8s-sprites-unlabeled-25pct.iuml" + + participant "<$pod>\nContainer" as K + participant Tini + note right Tini: "Tiny init"\ngithub.com/krallin/tini + participant "Entrypoint" as E + participant "Init script" as I + participant "Appserver" as A + + create Tini + K -> Tini: Start + + create E + Tini -> E: Start + create I + E -> I: Start + + create A + I -> A: Start + activate A + I -> A: Configure password aliases + I -> A: Configure keys for S3 + I -> A: Configure resources + I -> A: Configure Dataverse\nJVM options + I -> A: Stop + destroy A + I -> I: Symlink WAR & more + + create A + E -> A: Start in foreground + activate A + E --> Tini: exec(): replace with Appserver + destroy E + Tini -> A: Keep running until container stops + A -> A: Autodeploy WAR + @enduml