Skip to content

Update Draft for Docker

Jump to bottom
Joe Wicentowski edited this page Mar 4, 2024 · 6 revisions

Update your history.state.gov Development Environment (Draft for moving folks to Docker)

These directions assume you have a fully configured history.state.gov Development Environment and that you need to update your system so that it contains the latest operating system, software dependencies, core applications, and configurations.

Note: The changes described in this draft version of our "Update" document mark the most significant change to the HSG Development Environment in years. Namely, they change the way we interact with eXist-db. Instead of running eXist directly as an application, we will install it through an application called Docker.

Install system updates

  1. From the Apple menu  in the top-left corner of your screen, choose System Settings. Click General in the sidebar, then click Software Update on the right. Or in earlier versions of macOS, choose Apple menu  > System Preferences, then click Software Update.

  2. Open the App Store (using the Spotlight (🔍) icon in the menu bar, search for App Store; or in Finder, select Go > Applications, and find App Store in the list of applications), select the Updates tab, and install all available updates.

Update the Apple Developer Command Line Tools

  1. After installing all system updates, you will need to install the current version of the Apple Developer Command Line Tools. To do so, open Terminal (using Spotlight, search for Terminal; or in Finder, select Go > Utilities), and paste in the following command:

     xcode-select --install

  2. If this command returns a message that says, note: Command line tools are already installed, then you can skip to the next step. Otherwise, you will see a dialog window with 3 buttons. Select Install, and let the installation complete. Or, it might tell you that you already have these tools installed, in which case you should proceed.

Update Homebrew

  1. We need to update Homebrew. To do so, enter the following command:

     brew update && brew upgrade

    This may take several minutes if it's been a long time since you've performed this action.

Update HSG software dependencies

  1. With Homebrew updated, we will use it to make sure you have all of the HSG software dependencies installed. Run the following commands to remove old software that is no longer needed and install our new required software:

     brew uninstall --zap exist-db temurin8

 brew install --cask docker zulu11

 mkdir ~/Downloads/hsg-project-downloads

    If prompted, enter the password for your the user account on your Mac. If any of the applications are already installed, Homebrew will skip them, retaining the existing installation.

  2. We are no longer using the "nvm" utility. The official directions for uninstalling it can be found at https://github.com/nvm-sh/nvm?tab=readme-ov-file#uninstalling--removal, but here are directions adapted for our systems.

     nvm_dir="${NVM_DIR:-~/.nvm}"

 nvm unload

 rm -rf "$nvm_dir"

    Then, run the following command to open the .zshrc file in oXygen:

     open -a "Oxygen XML Editor" ~/.zshrc

    oXygen will present a dialog asking what file type to use when opening this file. Select "Text" from the list of file types.

    When the file opens in oXygen, delete the following 3 lines from the file:

     export NVM_DIR="$HOME/.nvm"
 [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
 [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

    These may be the only 3 lines in this file, in which case you should delete all 3 lines.

    Then save and close the .zshrc file.

  3. Run brew doctor to check your Homebrew installation, and follow any instructions to resolve problems that it reports. Keep running brew doctor until it reports:

    Your system is ready to brew.

    Sometimes the problems reported by brew doctor are inscrutable. Contact Joe if you are unable to decipher these.

  4. Make sure that the following icons are in your Dock: Docker, oXygen XML Editor, GitHub Desktop, and OpenVPN Connect. If any are not there, go to Finder, and select Go > Applications, and drag the respective applications' icons into the dock.

Set up eXist using Docker Desktop

Instead of starting eXist via a Dock icon, we will now use Docker Desktop, a utility that encapsulates applications and their dependencies. These steps will provide you with a fully encapsulated instance of history.state.gov, which is ready to start with all packages and data pre-installed.

Note: The steps in this segment of the directions will trigger a large download, of approximately 15 GB. Please be sure you have a fast broadband connection before beginning.

  1. Start Docker Desktop from the icon in your Dock.

  2. Advance through the setup screens, accepting the terms of service, using recommended settings, and installing the recommended "Rosetta" software update.

  3. When prompted to log into Docker Hub, select skip.

  4. When prompted to complete survey, select skip.

  5. This completes the first time steps for starting Docker Desktop, and you will be looking at Docker Desktop's "Dashboard" screen.

  6. In the Search field at the top of Docker Desktop's Dashboard, enter joewiz/hsg-project, and an entry will appear beneath the Images tab, labeled joewiz/hsg-project. Select the Pull button to the right of this entry. This will trigger the download of the hsg-project image, containing eXist and the HSG code and data repositories.

  7. Once the image has finished downloading, click on the Images tab in the left sidebar. Under Local, you should see an entry called joewiz/hsg-project. Toward the right side of this entry and under Actions, click on the ▶ (triangle-shaped) icon.

  8. A dialog window, "Run a new container" will appear. Click on Optional settings to reveal a form. Enter these values:

    • Container name: hsg-project-1.0
    • Ports: In these 2 fields, enter:
      • 8080
      • 8443
    • Volumes: In the field on the left side, select your Downloads/hsg-project-downloads folder. In the field on the right side, enter /tmp/hsg-project-downloads
    • Environmental Variables: Leave this field blank.

  9. Select the Run button

  10. Docker Desktop will switch you to the Containers pane (in the listing of panes in the left sidebar), and within that pane the Logs tab. You will see logging messages showing that eXist is starting. You'll know that eXist finishes its startup when, after a few moments, you see the logging message, "Server has started, listening on: ..."

  11. Return to the list of containers by selecting the Containers tab in the left sidebar. Find the container named hsg-project-1.0. The icons in this entry's Actions section let you start and stop eXist via the ▶ (triangle-shaped) icon (which starts the image) and the ⏹ (square-shaped) icon (which stops it). The container should already be running.

Set up eXist

With the hsg-project-1.0 image running in Docker Desktop, eXist is ready, with a full copy of history.state.gov running at http://localhost:8080/exist/apps/hsg-shell/! (No more "Deploy all repositories to localhost"!)

However, the version of HSG included in the hsg-project-1.0 Docker image is not updated automatically with the latest commits made to every repository on GitHub. The following steps are needed once to prepare eXist for you to deploy updated versions of any repositories.

  1. Go to the login screen for eXist's Dashboard app at http://localhost:8080/exist/apps/dashboard/login.html.

  2. Log into Dashboard with username admin and a blank password.

  3. Select Package Manager > Available, scroll down to find the package called "Shared Resources", and select the ⤓ (down-facing arrow) icon to download and install the Shared Resources package.

Set up oXygen

  1. Start oXygen XML Editor by clicking on its app icon in your Dock.

  2. From the External Tools toolbar menu (i.e., the green ▶ triangle-shaped icon), select these, one by one, waiting for each to complete (i.e. show BUILD SUCCESSFUL) before proceeding:

    • Pull updates from all repositories
    • 1. Apply Mac settings to hsg-project
    • 2. Clone all repositories & resources
    • Stop here! Don't select #3 or #4.

Long time users of hsg-project would expect the next step to be selecting commands #3 and #4 in this list. These commands are no longer applicable, since our Docker image contains a pre-built version of HSG. However, since the Docker image is not updated automatically with the latest commits to GitHub, new steps are needed to install the latest version of any particular repository for previewing the latest changes.

To install the latest version of any particular repository, open a file from that repository in oXygen and select Deploy current file's repository to localhost.

Or, to preview a file(s) that you're working on with, select Upload current file to localhost, and you will be able to preview your changes to the file(s) in your browser.

Note: These directions do not allow us to build and deploy the hsg-shell repository, so you will be limited to seeing the version of this repository that is included with the image.

Prepare for publishing

If you publish to history.state.gov, be sure you have completed the relevant steps in the Setup article for:

  • OpenVPN
  • oXygen's Data Source Explorer
  • Transmit

If any of these methods are not working as expected, contact Joe.

Wrap up

Congratulations! Your HSG Development Environment is now up to date!

Known issues

  1. These directions do not allow us to build and deploy the hsg-shell repository, so you will be limited to seeing the version of this repository that is included with the image. Contact Joe if you need a newer version of hsg-shell.

  2. The Generate troubleshooting report External Tools command may fail before completion. Any output from this report is still useful though.

  3. The 3. Apply hsg-project settings to eXist and 4. Deploy all repositories to localhost External Tools commands are no longer applicable in our environment. They will be removed when everyone has migrated to this new setup.

  4. When downloading XML documents from Docker eXist-db, you may see whitespace issues, caused by a missing configuration setting. This will be addressed in an update to our Docker image.

Clone this wiki locally