-
Notifications
You must be signed in to change notification settings - Fork 50
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #395 from kpeeples/kaas-container
KaaS Image Build and Push to Dockerhub
- Loading branch information
Showing
8 changed files
with
174 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
name: Docker image | ||
on: | ||
release: | ||
types: [published] | ||
workflow_dispatch: | ||
jobs: | ||
build: | ||
name: Build & push docker image | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v3.5.3 | ||
|
||
- name: Info | ||
run: echo "Parameters. ${{ github.event.base_ref }}, ${{ github.ref_type }}, ${{ github.ref }}" | ||
|
||
- name: Log in to Docker Hub | ||
uses: docker/login-action@v2 | ||
with: | ||
username: ${{ secrets.DOCKERHUB_USERNAME }} | ||
password: ${{ secrets.DOCKERHUB_PASSWORD }} | ||
|
||
- name: Build and push Docker image | ||
uses: docker/build-push-action@v4 | ||
with: | ||
context: "{{defaultContext}}:src/containers/docker" | ||
push: true | ||
tags: kpeeples/kaas-baseline:latest | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
| The jupyter hub docker images are at: https://github.com/jupyter/docker-stacks. We will extend them for Kestrel-as-a-Service and other use cases. At the moment the docker image is getting pushed to - https://hub.docker.com/repository/docker/kpeeples/kaas-baseline/general. The dockerfile in this repo includes kestrel-lang, kestrel-analytics, kestrel-huntbook and tutorials. The workflow file automatically builds the image upon commits. | ||
| | ||
| Login to dockerhub using one of the examples: | ||
| A. $ docker login | ||
| Username: | ||
| Password: | ||
| Login Succeeded | ||
| B. $ docker login --username demo --password example | ||
| C. $ cat password.txt | docker login --username demo --password-stdin | ||
| Note: you can use a credential helper - https://docs.docker.com/engine/reference/commandline/login/#credentials-store | ||
| | ||
| Manually Build image using one of the examples, the below url should change to the kestrel-lang url | ||
| A. $ docker build -t kpeeples/kaas-baseline:latest -t kpeeples/kaas-baseline:v1 | ||
| B. $ sudo docker build -t kpeeples/kaas-baseline:latest -t kpeeples/kaas-baseline:v2 https://raw.githubusercontent.com/kpeeples/kestrel-as-a-service/main/dockerhub/Dockerfile | ||
| | ||
| Push image, version should be incremental and will be standardized later | ||
| A. $ sudo docker push kpeeples/kaas-baseline:v1 | ||
| B. $ sudo docker push kpeeples/kaas-baseline:latest | ||
| |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
================================ | ||
Container Formats for Deployment | ||
================================ | ||
|
||
Docker Format for Dockerhub | ||
=========================== | ||
|
||
.. include:: dockerhub.rst | ||
|
||
OCI Format | ||
========== | ||
|
||
.. include:: oci.rst |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
|
||
Placeholder for future OCI (Open Container Initiative) - https://opencontainers.org/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
# Copyright (c) Jupyter Development Team. | ||
# Distributed under the terms of the Modified BSD License. | ||
ARG OWNER=jupyter | ||
ARG BASE_CONTAINER=$OWNER/base-notebook | ||
FROM $BASE_CONTAINER | ||
|
||
LABEL maintainer="Jupyter Project <jupyter@googlegroups.com>" | ||
|
||
# Fix: https://github.com/hadolint/hadolint/wiki/DL4006 | ||
# Fix: https://github.com/koalaman/shellcheck/wiki/SC3014 | ||
SHELL ["/bin/bash", "-o", "pipefail", "-c"] | ||
|
||
USER root | ||
|
||
# Install all OS dependencies for fully functional notebook server | ||
# added curl and gnupg for analytics | ||
RUN apt-get update --yes && \ | ||
apt-get install --yes --no-install-recommends \ | ||
# Common useful utilities | ||
git \ | ||
curl \ | ||
gnupg\ | ||
nano-tiny \ | ||
tzdata \ | ||
unzip \ | ||
vim-tiny \ | ||
# git-over-ssh | ||
openssh-client \ | ||
# less is needed to run help in R | ||
# see: https://github.com/jupyter/docker-stacks/issues/1588 | ||
less \ | ||
# nbconvert dependencies | ||
# https://nbconvert.readthedocs.io/en/latest/install.html#installing-tex | ||
texlive-xetex \ | ||
texlive-fonts-recommended \ | ||
texlive-plain-generic \ | ||
# Enable clipboard on Linux host systems | ||
xclip && \ | ||
apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
||
# analytics prereq | ||
RUN curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash | ||
|
||
RUN sudo apt-get install --yes git-lfs | ||
|
||
# Install kestrel-LANG | ||
RUN pip install kestrel-lang | ||
|
||
# Create alternative for nano -> nano-tiny | ||
RUN update-alternatives --install /usr/bin/nano nano /bin/nano-tiny 10 | ||
|
||
RUN mkdir -p /opt/kaas && chown ${NB_UID}:${NB_UID} /opt/kaas | ||
|
||
# Switch back to jovyan to avoid accidental container runs as root | ||
USER ${NB_UID} | ||
|
||
# Install kestrel-JUPYTER | ||
RUN pip install kestrel-jupyter | ||
|
||
# setup Jupyter | ||
RUN python -m kestrel_jupyter_kernel.setup | ||
|
||
RUN mv "${HOME}/.local/share/jupyter/kernels/kestrel"* "${CONDA_DIR}/share/jupyter/kernels/" && \ | ||
chmod -R go+rx "${CONDA_DIR}/share/jupyter" && \ | ||
rm -rf "${HOME}/.local" | ||
|
||
# Add R mimetype option to specify how the plot returns from R to the browser | ||
RUN curl https://raw.githubusercontent.com/kpeeples/kestrel-as-a-service/main/dockerhub/Rprofile.site -o ${HOME}/Rprofile.site | ||
RUN chown ${NB_UID}:${NB_GID} ${HOME}/Rprofile.site && mkdir -p /opt/conda/lib/R/etc && mv Rprofile.site /opt/conda/lib/R/etc/Rprofile.site | ||
#COPY --chown=${NB_UID}:${NB_GID} Rprofile.site /opt/conda/lib/R/etc/ | ||
|
||
# Install kestrel-ANALYTICS | ||
RUN git clone https://github.com/opencybersecurityalliance/data-bucket-kestrel.git /opt/kaas/data-bucket-kestrel | ||
|
||
RUN mkdir -p /opt/kaas/analytic-tutorials && \ | ||
mkdir -p /opt/kaas/huntbooks && \ | ||
mkdir -p /opt/kaas/tutorial && \ | ||
mkdir -p /opt/kaas/blackhat22 | ||
|
||
# for generic huntbooks like BH22 | ||
RUN cp -r /opt/kaas/data-bucket-kestrel/stix-bundles /opt/kaas/analytic-tutorials | ||
|
||
# for the tutorial | ||
RUN cp /opt/kaas/data-bucket-kestrel/stix-bundles/lab101.json /opt/kaas/analytic-tutorials | ||
|
||
RUN git clone https://github.com/opencybersecurityalliance/kestrel-analytics.git /opt/kaas/kestrel-analytics | ||
|
||
RUN cp /opt/kaas/data-bucket-kestrel/GeoLite2/GeoLite2-City.mmdb /opt/kaas/kestrel-analytics/analytics/piniponmap | ||
|
||
RUN rm -rf "/opt/kaas/data-bucket-kestrel" | ||
|
||
# install kestrel-HUNTBOOK | ||
RUN git clone https://github.com/opencybersecurityalliance/kestrel-huntbook.git /opt/kaas/kestrel-huntbook | ||
|
||
RUN mkdir -p /home/jovyan/.config/kestrel && \ | ||
mv /opt/kaas/kestrel-huntbook/config/stixshifter.yaml /home/jovyan/.config/kestrel/ && \ | ||
ln -s /home/jovyan/.config/kestrel/stixshifter.yaml /opt/kaas/kestrel-huntbook/huntbooks/stixshifter.yaml && \ | ||
ln -s /home/jovyan/.config/kestrel/stixshifter.yaml /opt/kaas/kestrel-huntbook/tutorial/stixshifter.yaml && \ | ||
ln -s /home/jovyan/.config/kestrel/stixshifter.yaml /opt/kaas/kestrel-huntbook/blackhat22/stixshifter.yaml | ||
|
||
RUN rmdir /opt/kaas/kestrel-huntbook/config | ||
|
||
RUN cp /opt/kaas/kestrel-analytics/pythonanalytics_sample.yaml /home/jovyan/.config/kestrel/pythonanalytics.yaml | ||
|
||
#RUN jupyter trust */*.ipynb |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Add R mimetype to specify how the plot returns from R to the browser. | ||
# https://notebook.community/andrie/jupyter-notebook-samples/Changing%20R%20plot%20options%20in%20Jupyter | ||
|
||
options(jupyter.plot_mimetypes = c('text/plain', 'image/png', 'image/jpeg', 'image/svg+xml', 'application/pdf')) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
This is a placeholder for any future oci formats |