Automate a multi machine ArcGIS Enterprise deployment on Windows using Chef Server
This workflow describes how to deploy ArcGIS Enterprise on a set of Windows machines using Chef Server.
To follow this workflow for Linux machines, see Automate a multi machine ArcGIS Enterprise deployment on Linux using Chef Server.
Note: The main goal of this workflow is to demonstrate the deployment process. Chef cookbook can configure any ArcGIS Enterprise deployment pattern. Please refer to the ArcGIS Enterprise documentation to pick the ideal deployment architecture for your organization.
- Five Windows machines in the same domain with access to the internet.
- Setups for ArcGIS Enterprise. Note the name of your file. For setup files named "Setup.exe," you should define them in the JSON configuration files using the
setupsvariable; setup files named in a format such as "ArcGIS_Server_Windows_1081_175203.exe" are self-extracting and should be defined using thesetups_archivevariable. - Software authorization files for ArcGIS Enterprise (Portal for ArcGIS and ArcGIS GIS Server) and for ArcGIS GeoAnalytics Server. All software and authorization files should be for the same version.
- A SSL certificate file for the domain name of the base ArcGIS Enterprise machine.
- Starter Kit for Chef Server. See the options below under Prepare Chef Server.
Below is the intended mapping for our five machines and deployment roles.
- Machine 1 -> Chef workstation
- Machine 2 -> Base ArcGIS Enterprise (baseenterprise)
- Machine 3 -> Big Data Store (bds)
- Machine 4 -> File server for GeoAnalytics (fileserver)
- Machine 5 -> GeoAnalytics Server (gaserver)
-
Make sure WinRM service is running and already set up for remote management on all machines participating in this deployment. By default, WinRM service should be running on Windows machines. You can verify this on individual machines by running the command below in your command prompt:
winrm quickconfig -q
If WinRM service is not configured on any machine, the above command will configure it.
- Make sure you have one Windows user (local or domain) with administrator privileges on each machine. This user is used in subsequent PowerShell scripts for deployment purposes.
Most organizations are using Chef Server to automate deployment. If you don’t have a Chef Server, then there are multiple options available to set it up. You can download the setup from the Chef website and install it on your machine. Alternatively, you can use Hosted Chef Server provided by Chef (SaaS) or Chef Automate. If you have an AWS account, then you can use AWS opsworks for Chef to set up Chef Server.
For this workflow, we are using Hosted Chef Server.
- Sign up for a free trial of Hosted Chef if you don't already have an account.
- Sign in to the Chef Server management console, create an organization and download the starter kit for the organization (Administration > Organizations > Starter Kit).
If you don’t have a Chef workstation set up, the following steps can be used to set it up as a reference.
2. On the Chef workstation machine, install Chef Workstation.
- On the Chef Workstation machine, unzip the starter kit
starter-kit.zip - After unzipping, the path should be
C:\chef-repo.
- Copy your encryption key (.pem file) and paste it inside folder
C:\chef-repo\.chef.
- Double click the Desktop Shortcut
CW PowerShell - or click Windows button and start
CW PowerShell
8. Retrieve your server certificate and verify your workstation can communicate with the Chef server:
knife ssl fetch
knife ssl check
knife client list
Download the cookbooks and roles zip files corresponding to your version of ArcGIS Enterprise. Unzip the contents of the cookbooks file into the C:\chef-repo\cookbooks directory on your Chef workstation machine.
knife cookbook upload --all
Unzip your arcgis-enterprise-windows-roles.zip file and move the two folders into C:\chef-repo\roles and C:\chef-repo\environments folders respectively.
Open the file in a text editor like Notepad++. The file has a number of sample values meant to be replaced. Substitute values for these nodes specific to your deployment, like REPOMACHINE name, file paths, user names, and passwords.
knife upload environments\demo.json
A bootstrap is a process that installs the chef-client on a target system so that it can run as a chef-client and communicate with a Chef server. Replace username and password in the command below with those specific to your deployment. Replace HOSTNAME2 with system/host name of your Machine 2. The --bootstrap-version parameter allows you to state the version of the Chef Client you want to use; recommended versions can be found in the Esri-cookbooks-and-Chef-Infra-client-version-compatibility wiki page. For the --secret-file parameter define the path to the file that contains the encryption key (.pem file) that is included in your Hosted Chef Server Starter Kit.
knife bootstrap -o winrm HOSTNAME2 -U 'username' -P 'password' --node-name baseenterprise --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
username refers to the local or domain Windows administrator user available on Machine 2. password refers to the password of your Windows administrator user.
Update values for nodes like REPOMACHINE, yourdomainname, file paths, user names, and passwords, and upload it to the Chef server.
knife upload roles\baseenterprise.json
knife node run_list set baseenterprise 'role[baseenterprise]'
knife node environment set baseenterprise demo
5. Run chef-client on nodes in the 'baseenterprise' role. Replace username and password as mentioned in step 1.
knife winrm 'role:baseenterprise' 'chef-client' --winrm-shell elevated -x 'username' -P 'password'
The Chef run can take around 1-2 hours. After the Chef run finishes, you have Portal, Server and Data Store installed on Machine 2 with the appropriate configuration, as depicted in the above architecture diagram. You can access your ArcGIS Enterprise portal using the URL format https://<yourdomainname>/portal/home.
Replace username and password in the command below with those specific to your deployment. Replace HOSTNAME3 with system/host name of Machine 3.
knife bootstrap -o winrm HOSTNAME3 -U 'username' -P 'password' --node-name bds --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
username refers to the local or domain Windows administrator user available on Machine 3. password refers to the password of your Windows administrator user.
2. Modify node values in the file 'C:\chef-repo\roles\bds.json' available on your Chef workstation machine with values specific to your deployment, and upload it to the Chef server.
knife upload roles\bds.json
knife node run_list set bds 'role[bds]'
knife node environment set bds demo
5. Run chef client on nodes in the 'bds' role. Replace username and password as mentioned in step 1.
knife winrm 'role:bds' 'chef-client' --winrm-shell elevated -x 'username' -P 'password'
After the Chef run finishes, a spatiotemporal big data store is installed on Machine 3 and registered with the ArcGIS Server installed on Machine 2, as depicted in the architecture diagram.
Most organizations probably already have a file server in place. If you do, then you can use your existing file server by providing its file path to the GeoAnalytics Server you set up. Refer to the Chef parameter list on how to define it. The following steps are just an example to configure a file server for you if you don’t already have one.
Replace username and password in command below with those specific to your deployment. Replace HOSTNAME4 with system/host name of Machine 4.
knife bootstrap -o winrm HOSTNAME4 -U 'username' -P 'password' --node-name fileserver --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
username refers to the local or domain Windows administrator user available on Machine 4. password refers to the password of your Windows administrator user.
2. Modify node values in 'C:\chef-repo\roles\fileserver.json' file on your Chef workstation machine with values specific to your deployment and upload them to the Chef server.
knife upload roles\fileserver.json
knife node run_list set fileserver 'role[fileserver]
knife node environment set fileserver demo
5. Run chef-client on nodes in the 'fileserver' role. Replace username and password as mentioned in step 1.
knife winrm 'fileserver' 'chef-client' --winrm-shell elevated -x 'username' -P 'password'
After the Chef run finishes, a file server is configured on Machine 4, as depicted in the architecture diagram.
Replace username and password in command below with those specific to your deployment. Replace HOSTNAME5 with system/host name of Machine 5.
knife bootstrap -o winrm HOSTNAME5 -U 'username' -P 'password' --node-name gaserver --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
username refers to the local or domain Windows administrator user available on Machine 4. password refers to the password of your Windows administrator user.
2. Modify node values in 'C:\chef-repo\roles\gaserver.json' file on your Chef workstation machine with values specific to your deployment and upload them to the Chef server.
knife upload roles\gaserver.json
knife node run_list set gaserver 'role[gaserver]'
knife node environment set gaserver demo
5. Run chef-client on nodes in the 'gaserver' role. Replace username and password as mentioned in step 1.
knife winrm 'gaserver' 'chef-client' --winrm-shell elevated -x 'username' -P 'password'
After the Chef run finishes, ArcGIS Server is installed on Machine 5, as depicted in the architecture diagram.
You can verify deployments by accessing URLs https://yourdomainname/portal and https://yourdomainname/server.
1. Modify 'C:\chef-repo\roles\cleanarcgisenterprise.json' file on your Chef workstation,and upload it to your Chef server.
knife upload roles\cleanarcgisenterprise.json
knife node run_list set baseenterprise 'role[cleanarcgisenterprise]'
knife node run_list set fileserver 'role[cleanarcgisenterprise]'
knife node run_list set gaserver 'role[cleanarcgisenterprise]'
knife node run_list set bds 'role[cleanarcgisenterprise]'
knife winrm 'role:cleanarcgisenterprise' 'chef-client' --winrm-shell elevated -x 'username' -P 'password'
After the Chef run finishes, ArcGIS Server, Portal for ArcGIS, ArcGIS Data Store, and ArcGIS Web Adaptor for IIS will be uninstalled from respective nodes. It also cleans up content directories like arcgisserver, arcgisportal and arcgisdatastore.
The above workflow was designed to make the deployment process easier to understand and recover from mistakes. In production deployments, instead of deploying one machine at a time, it is recommended to split and script the process into three phases:
A sample PowerShell script would be:
knife bootstrap -o winrm HOSTNAME2 -U 'username' -P 'password' --node-name baseenterprise --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
knife bootstrap -o winrm HOSTNAME3 -U 'username' -P 'password' --node-name bds --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
knife bootstrap -o winrm HOSTNAME4 -U 'username' -P 'password' --node-name fileserver --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
knife bootstrap -o winrm HOSTNAME5 -U 'username' -P 'password' --node-name gaserver --bootstrap-version '15.14.0' --secret-file "C:/chef-repo/.chef/your-chef-server-encryption-key-file.pem" --chef-license accept
knife upload roles\baseenterprise.json
knife upload roles\bds.json
knife upload roles\fileserver.json
knife upload roles\gaserver.json
knife node run_list set baseenterprise 'role[baseenterprise]'
knife node run_list set bds 'role[bds]'
knife node run_list set fileserver 'role[fileserver]'
knife node run_list set gaserver 'role[gaserver]'
knife node environment set baseenterprise demo
knife node environment set bds demo
knife node environment set fileserver demo
knife node environment set gaserver demo
knife winrm 'role:baseenterprise' 'chef-client' --winrm-shell elevated -x 'DOMAIN\USER' -P 'password'
knife winrm 'role:bds' 'chef-client' --winrm-shell elevated -x 'DOMAIN\USER' -P 'password'
knife winrm 'role:fileserver' 'chef-client' --winrm-shell elevated -x 'DOMAIN\USER' -P 'password'
knife winrm 'role:gaserver' 'chef-client' --winrm-shell elevated -x 'DOMAIN\USER' -P 'password'