Merge 1.7.0

* v1.7.0-dev-01

- Updated files to UTF format.
- Added installation requirements for Python3 to dockerfile
- Codebase updated for Python3
- Fixed cron.sh script
- Fixed web app ports report

* v1.7.0-dev-02

- Fixed wheel install
- Updated config tpl to main - integrate with SSH data source
- SSL Certificate Detail renamed to SSL Certificate
- Other Detail attributes tweaked in line with BMC TKU
- Reports incorporate "TLS Certificate" type

* v1.7.0-dev-03

- Migrated to SofwareInstance link. Will generate new SI if missing.

* v1.7.0-dev-04

- Fixed orphan certs report
- Added back mapping for devices (if not Host/SoftwareInstance)
- Added untested CMDB sync pattern (software only)

* v1.7.0-dev-05

- Reorganised file structure
- Added install script with API calls

* v1.7.0-dev-06

- Removed system credentials from config.ini
- Updated install file with api methods

* v1.7.0-dev-07

- Added TPL upload function

* v1.7.0-dev-08

- Added sshd user and port forward instructions
- Updated Dockerfile with tideway package

* v1.7.0-dev-09

- Updated config.ini, removed login details, updated query without quotes
- Updated getCert to use API calls for search

* v1.7.0-dev-10

- Added crontab and moved crond and sshd to supervisord
- Added cli parser

* v1.7.0-dev-11

- Made logfile optional argument
- Internalised xml file generation
- Updated crontab to specify config file
- Token now stored as persistent environment variable (multiple tokens supported)

* v1.7.0-dev-12

- Updated Dockerfile syntax
- getCert Discovery target is enforced through parameters
- New event to Discovery target to trigger a scan/collection
- Randomly generated gpg passphrase is sent with event details (not stored)
- API token stored in persistent environment variable (per Discovery instance)
- getCert TPL overhaul - triggering on ExternalEvent

* v1.7.0-dev-13

- Event Source working
- getCert TPL updated - fixed device associations and new custom cert relationship between Cert Detail and Printer, StorageDevice and SNMPManagedDevice (does not show up in UI).

* v1.7.0-dev-14

- Removed makefile code
- Relocated install directory
- Added debug options
- New logs directory
- IPv4 validation
- Fixed posted event response
- Removed config TPL - not required (variable info included in event)
- Fixed missing SHA1 value
- Added SHA256 fingerprint flag to openssl call
- Set removal groups as standard
- Fixed CMDB sync pattern missing terminator

* v1.7.0-dev-15

- makeself moved to Dockerfile installation
- standalone crontab update

* v1.7.0-dev-16

- Added coreutils to install (MD5 missing, unable to be found by makeself)?
- Updated all files to UTF-8 format
- Install script can take target argument (for standalone install)
- Updated build with makeself options - stores in a 'package' directory
- Removed prep script
- Setup script (run) now requires arguments for install/dashboards

* v1.7.0-dev-17

- Moved crontab deployment into installation and setup scripts in order to pass target parameter
- Fixed event log where mode is not set to 3
- Using python-crontab to set Docker crontab locally
- Fixed path creation for first time standalone install
- Fixed input for instance even if not doing install (used for crontab)

* v1.7.0-dev-20

- Updated docs
- Added archive docs.

.gitignore

@@ -14,7 +14,6 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
 lib64/
 parts/
 sdist/
@@ -133,3 +132,6 @@ licenses/
 makeself/*.py
 makeself/Traversys/
 *.zip
+*.gpg
+temp/
+standalone/package/

README.md

@@ -1,27 +1,49 @@
 # getCert CE - SSL Certificates for BMC Discovery
+Copyright 2021 Traversys Limited
 
 |     |     |
 | --- | --- |
 | License | Apache License 2.0 |
-| Version | 1.6 |
+| Version | 1.7.0 (ED-209) |
 
-Polls devices on a given IP range for SSL certificate details.
+# Overview
 
-## Installation
+Traversys getCert is an extension to BMC Discovery. It operates independently of Discovery scans in either a Docker container or a standalone script. getCert non-invasively collect information about externally facing SSL certificates (accessible via open web/application ports) against your data center infrastructure.
 
-### Requires
-* Docker
+getCet triggers an event to Discovery via the API which contains a temporary key to unlock the encrypted capture data stored with the getCert application files. The getCert pattern module will then trigger on the event and scan the getCert data source to retrieve and build a list of certificate Detail nodes. If any SoftwareInstances match the certificate IP or common name and port getCert will automatically attempt to map the Detail to the SI, alternatively, for other network devices - getCert will map directly to the device node.
+
+getCert comes with a beta CMDB sync pattern which follows the schema of BMC's TLS Certificate OOTB mappings - and will sync any SSL Certificate details mapped directly to a SoftwareInstance to BMC_Document.
+
+getCert is highly configurable, with the option to scan subnets, websites, DNS hostnames or IP addresses, and the discovered data can be added to the model in different ways with custom extensions.
+
+# Quickstart
 
-### Quickstart
 1. Install and configure Docker
-2. Run `docker build --tag getcert -f dockerbuild/Dockerfile .`
-3. To use run `docker run -it -v ${PWD}:/usr/dev getcert:latest`
-4. An installation file can be generated by running `prep.py`
+2. Run `docker build --tag getcert --progress=plain -f dockerbuild/Dockerfile .`
+3. Make a note of the password generated by the build script\
+   ![alt text](docs/images/screenshot.png "Get ssh password")
+4. Startup the Container
+```console
+docker run -t -d -p 2222:22 --name getCert -v ${PWD}:/opt/Traversys/getCert getcert:latest
+```
+5. Access the shell: `docker exec -it getCert /bin/bash`
+6. Navigate to `/opt/Traversys/getCert`
+7. Run the installation script `python3 install.py`
+8. Run getCert
+
+```console
+$ screen
+$ python3 /opt/Traversys/getCert/getcert.py --instance <Discovery URL/IP> --config /opt/Traversys/getCert/config.ini
+```
+
+## Scan Modes
+
+getCert will commence in the background and export details to an encrypted data file.
+
+There are 3 modes you can use and are set in the config file:
 
-### Deployment
-1. Copy the the **traversys_getcert.run** binary to the Discvory Appliance (recommended /usr/tideway/data/customer)
-2. Change permissions: `chomd 755 traversys_getcert.run`
-3. Run the installation: `./traversys_getcert.run`
-4. Follow on-screen instructions.
+1. Scan for SSL Certificates on the test ips/subnet string
+2. Scan for a list of ips/subnets in the specified input file
+3. Run the query (appliance login needed) to export a list of ips/subnets
 
-Documentation: https://traversys.github.io/getCert_CE/
+Full Documentation: https://traversys.github.io/getCert_CE/

VERSION

@@ -0,0 +1 @@
+1.7

source/config.ini → config.ini

@@ -1,9 +1,8 @@
 [ENV]
-root = /usr/tideway/data/customer/Traversys/getCert
+root = /opt/Traversys/getCert
 temp = %(root)s/temp
-log = %(temp)s/ssl-out.log
-capture = %(temp)s/ssl-out.xml
 iplist = %(temp)s/ips
+logs = %(root)s/logs
 
 [MODE]
 mode = 1
@@ -15,14 +14,7 @@ testsubnet = 192.168.1.0/24
 file = sample_input.txt
 
 [DISCO_QUERY]
-query = "search DiscoveryAccess where _last_marker show endpoint"
-
-[LOGIN]
-discouser = system
-discopass = system
-
-[CRON]
-cron = "0 1 * * 6"
+query = search DiscoveryAccess where _last_marker show endpoint
 
 [TIMEOUT]
 timeout = 3m

dockerbuild/Dockerfile

@@ -1,26 +1,36 @@
 # Pull from source
 FROM centos:7
-MAINTAINER team@traversys.io
+LABEL org.opencontainers.image.authors="team@traversys.io"
 
 # Setup the environment
 RUN yum clean all
 RUN yum -y --enablerepo=extras install epel-release && \
 yum -y install centos-release-scl && \
+yum install -y coreutils && \
 yum -y install nmap && \
-yum -y install python27 && \
+yum -y install python3 && \
 yum -y install net-tools && \
 yum -y install openssh-server openssh-clients && \
-yum -y install python-pip
-RUN pip install pyinstaller==3.6
+yum -y install python3-pip && \
+yum -y install makeself && \
+yum -y install zlib-devel && \
+yum -y install openssl && \
+yum -y install cronie && \
+yum -y install supervisor && \
+yum -y groupinstall "Development Tools"
+# Pip Packages
+RUN pip3 install pyinstaller==3.6
+RUN pip3 install wheel
+RUN pip3 install tideway
+RUN pip3 install python-dotenv
+RUN pip3 install python-crontab
+# Setup sshd
+RUN mkdir /var/run/sshd
+RUN /usr/bin/ssh-keygen -A
+RUN useradd getcert
+RUN SSHPASS="$(openssl rand -base64 20)" && echo "getcert:$SSHPASS" | chpasswd && echo "getcert user password: $SSHPASS"
+# Start running services
+COPY dockerbuild/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
+EXPOSE 22
+ENTRYPOINT ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
 
-CMD ["/bin/bash"]
-
-#
-# Build Docker image:
-#
-# docker build --tag getcert .   # creates image getcert
-#
-# Run Docker Container
-#
-# docker run -it -v <your git dir>:/usr/dev getcert:latest
-#

dockerbuild/supervisord.conf

@@ -0,0 +1,8 @@
+[supervisord]
+nodaemon=true
+
+[program:sshd]
+command=/usr/sbin/sshd -D
+
+[program:cron]
+command=/usr/sbin/crond