Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for split model files, restructured to allow import by other projects, improved command line syntax by use of cobra, and more... #57

Merged
merged 73 commits into from
Feb 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
28e6264
fixed typos and warnings
joreiche Dec 4, 2023
98882e0
fixed deprecaed and other warnings; fixed spelling
joreiche Dec 5, 2023
7f9f45d
made file paths cross-platform; eliminated IDE warnings; reformatted …
joreiche Dec 6, 2023
18585ce
implemented multi-yaml support
joreiche Dec 15, 2023
7a5366b
added support for directly pointing at the repo demo folders when cre…
joreiche Dec 15, 2023
a3baeea
add install mechanism
joreiche Dec 16, 2023
0233bc4
replace plugins with system calls to regular binaries
joreiche Dec 18, 2023
c7f4eba
modify builtin risks to use model as input parameter
joreiche Dec 18, 2023
ed97925
- made Dockerfile and Dockerfile.local consistent
joreiche Dec 20, 2023
8c2e7b0
converted parse test to go-style testing
joreiche Dec 20, 2023
0785567
moved to go-style folder structure
joreiche Dec 21, 2023
273a867
merged Yevhen's changes
joreiche Dec 22, 2023
2ffb181
Fix comment
Dec 22, 2023
2b634f1
Move create-stub-model, create-example-model, create-editing-support …
Dec 22, 2023
9619c24
Small cleanup
Dec 22, 2023
86a28df
Migrate print license and print 3rd party to cobra
Dec 22, 2023
7af8540
introduced config
joreiche Dec 23, 2023
4977664
Move parsing types closer to type definition
Dec 25, 2023
da977d6
Merge pull request #2 from joreiche/parse-types
ezavgorodniy Dec 25, 2023
b72a226
Move out parsing model code closer to model definition
Dec 25, 2023
4d8b398
Return error instead of panicking
Dec 25, 2023
f4b8aef
converting maps using iskCategory as index to use RiskCategory.Id ins…
joreiche Dec 26, 2023
5ce843a
committing mising file for last commit
joreiche Dec 26, 2023
01186a9
fixed marshaling/unmarshaling parsed model struct from/to json
joreiche Dec 26, 2023
324c639
Move server related files into package
Dec 27, 2023
13fda7c
Merge pull request #3 from joreiche/move-server
ezavgorodniy Dec 27, 2023
50023fc
resolved circular dependencies
joreiche Dec 27, 2023
8f86b3a
Merge branch 'master' of github.com:joreiche/threagile
joreiche Dec 27, 2023
dee5491
completed merge
joreiche Dec 27, 2023
bbe2aea
implemented config file loading and handling
joreiche Dec 27, 2023
d953aa9
Remove server related code from context.go
Jan 4, 2024
25268ff
Remove server related variables from context.go
Jan 4, 2024
aa558e0
Remove modelInput from context object because it's just a variable du…
Jan 4, 2024
7b81de0
Make drawSpaceLinesForLayoutUnfortunatelyFurtherSeparatesAllRanks as …
Jan 4, 2024
68e1e92
Allow to run application with cobra cli
Jan 4, 2024
2e91661
Move one more function out of context
Jan 4, 2024
f391430
Move out graphviz code into report package
Jan 4, 2024
5eb5b5b
Add forgotten file
Jan 5, 2024
027df78
Move parsed model out of context
Jan 5, 2024
6f5de2b
Refactor macros
Jan 5, 2024
c490c6b
Use parameters instead of context
Jan 5, 2024
e4857c2
Move logic to packages
Jan 5, 2024
a45ab19
Remove panic and global variables
Jan 5, 2024
6cdea6d
Remove accidentally uploaded file
Jan 5, 2024
87d8033
Adjust documentation according to changes
Jan 5, 2024
3713215
Inline calls to dot
Jan 5, 2024
87ab8e6
Restore reading config file
Jan 8, 2024
d6bb1c0
Move colors package to report package
Jan 8, 2024
f42969e
Reshuffle files to avoid too much logic inside parsed model, parsedMo…
Jan 8, 2024
3329cd5
Use go interfaces to specify built in risk rules, move custom risk ru…
Jan 9, 2024
77bc549
Tidy up go mod
Jan 9, 2024
8f26d96
- added list of contributors in addition to author to schema and inpu…
joreiche Jan 9, 2024
3e61f05
made sure to propagate build timestamp
joreiche Jan 10, 2024
8967685
made sure to omit empty fields what marshaling inpuyt structs
joreiche Jan 10, 2024
b35e4e9
fixed config file loading
joreiche Jan 24, 2024
dc8d4b0
improved model merging for split model yaml import
joreiche Jan 31, 2024
3d59ffc
improved string list and multi-line merging
joreiche Jan 31, 2024
ac5b4c4
fixed docker run for local docker file
joreiche Jan 31, 2024
6e23605
- added gosec workflow
joreiche Jan 31, 2024
a020fb4
trying to persuade Anchore to use the local code base rather than the…
joreiche Jan 31, 2024
b6bb2fb
upgraded codeql upload action to v2 according to deprecation notice
joreiche Jan 31, 2024
f03ea81
upgraded codeql upload action to v3 according to new deprecation noti…
joreiche Feb 1, 2024
eda8518
manually merged pull request #53
joreiche Feb 2, 2024
6af115d
resolved grype issues:
joreiche Feb 2, 2024
a7a61c6
ignoring grype build errors
joreiche Feb 2, 2024
033fc8a
Stored, sent and received data assets are always processed
Feb 2, 2024
16dfc6d
Merge pull request #4 from joreiche/pr-18-transfer
joreiche Feb 2, 2024
60bb614
manually merged pr #5: Infer CIA ratings of tech assets #19
joreiche Feb 2, 2024
be0dda0
made separate command 'analyze' to analyze a model
joreiche Feb 2, 2024
d45d4b6
added interactive mode
joreiche Feb 2, 2024
428119a
Add tests (red) to test previous commit
Feb 2, 2024
6c481d8
Merge branch 'master' of github.com:joreiche/threagile
joreiche Feb 2, 2024
5f76e14
fixed inference tests
joreiche Feb 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
**/.git
**/.gitignore
**/.DS_Store
**/*.tmp
**/*.tmp

.github
.dockerignore
Dockerfile
Dockerfile.*
35 changes: 21 additions & 14 deletions .github/workflows/anchore-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,24 @@ jobs:
Anchore-Build-Scan:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v4
- name: Build the Docker image
run: docker build . --file Dockerfile --tag localbuild/testimage:latest
- name: Run the local Anchore scan action itself with GitHub Advanced Security code scanning integration enabled
uses: anchore/scan-action@v3
with:
image-reference: "localbuild/testimage:latest"
dockerfile-path: "Dockerfile"
acs-report-enable: true
- name: Upload Anchore Scan Report
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: results.sarif
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build the Docker image
uses: docker/build-push-action@v5
with:
tags: localbuild/threagile:latest
file: Dockerfile.local
push: false
load: true

- name: Scan image
uses: anchore/scan-action@v3
with:
image: "localbuild/threagile:latest"
fail-build: false

- name: Upload Anchore Scan Report
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: results.sarif
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
uses: actions/checkout@v4
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
# a pull request then we can check out the head.
fetch-depth: 2

- name: Install Go
Expand All @@ -39,14 +39,14 @@ jobs:

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
Expand All @@ -60,4 +60,4 @@ jobs:
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
24 changes: 24 additions & 0 deletions .github/workflows/gosec-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Security Static Analysis

on:
push:
branches: [master]
pull_request:
# The branches below must be a subset of the branches above
branches: [master]
schedule:
- cron: '0 0 1 * *'

jobs:
analyze:
name: GoSec
runs-on: ubuntu-latest
env:
GO111MODULE: on
steps:
- name: Checkout Source
uses: actions/checkout@v4
- name: Run Gosec
uses: securego/gosec@master
with:
args: ./...
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,11 @@ stats.json
# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# build artifacts
/vendor/
/bin/
/out*/

# IDE stuff
.idea/
/.idea/
/config.json
47 changes: 47 additions & 0 deletions .grype.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
ignore:
- vulnerability: CVE-2023-42363
reason:
busybox(1.36.1-r15) - no upstream fix available
busybox-binsh(1.36.1-r15) - no upstream fix available
- vulnerability: CVE-2023-42364
reason:
busybox(1.36.1-r15) - no upstream fix available
busybox-binsh(1.36.1-r15) - no upstream fix available
- vulnerability: CVE-2023-42365
reason:
busybox(1.36.1-r15) - no upstream fix available
busybox-binsh(1.36.1-r15) - no upstream fix available
- vulnerability: CVE-2023-42366
reason:
busybox(1.36.1-r15) - no upstream fix available
busybox-binsh(1.36.1-r15) - no upstream fix available
- vulnerability: CVE-2014-9157
reason:
graphviz(8.0.5-r1) - no upstream fix available
graphviz-libs(8.0.5-r1) - no upstream fix available
- vulnerability: CVE-2023-37769
reason: pixman(0.42.2-r1) - no upstream fix available
- vulnerability: CVE-2023-43789
reason: libxpm(3.5.16-r1) - no upstream fix available
- vulnerability: CVE-2023-42363
reason: ssl_client(1.36.1-r5) - no upstream fix available
- vulnerability: CVE-2023-42364
reason: ssl_client(1.36.1-r5) - no upstream fix available
- vulnerability: CVE-2023-42365
reason: ssl_client(1.36.1-r5) - no upstream fix available
- vulnerability: CVE-2015-7313
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-3164
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-40745
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-41175
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-6228
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-6277
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-52355
reason: tiff( 4.5.1-r0) - no upstream fix available
- vulnerability: CVE-2023-52356
reason: tiff( 4.5.1-r0) - no upstream fix available
84 changes: 39 additions & 45 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@


######
## Stage 1: Clone the Git repository
######
FROM alpine/git as clone
WORKDIR /app

RUN git clone https://github.com/threagile/threagile.git


Expand All @@ -12,76 +15,67 @@ RUN git clone https://github.com/threagile/threagile.git
## Stage 2: Build application with Go's build tools
######
FROM golang as build
WORKDIR /app

ENV GO111MODULE=on

# https://stackoverflow.com/questions/36279253/go-compiled-binary-wont-run-in-an-alpine-docker-container-on-ubuntu-host
#ENV CGO_ENABLED=0 # cannot be set as otherwise plugins don't run
WORKDIR /app
COPY --from=clone /app/threagile /app

RUN go version
RUN go test ./...
RUN GOOS=linux go build -a -trimpath -ldflags="-s -w -X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -buildmode=plugin -o raa.so raa/raa/raa.go
RUN GOOS=linux go build -a -trimpath -ldflags="-s -w -X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -buildmode=plugin -o dummy.so raa/dummy/dummy.go
RUN GOOS=linux go build -a -trimpath -ldflags="-s -w -X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -buildmode=plugin -o demo-rule.so risks/custom/demo/demo-rule.go
RUN GOOS=linux go build -a -trimpath -ldflags="-s -w -X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -o threagile
RUN GOOS=linux go build -ldflags="-X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -o raa_calc cmd/raa/main.go
RUN GOOS=linux go build -ldflags="-X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -o raa_dummy cmd/raa_dummy/main.go
RUN GOOS=linux go build -ldflags="-X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -o risk_demo_rule cmd/risk_demo/main.go
RUN GOOS=linux go build -ldflags="-X main.buildTimestamp=$(date '+%Y%m%d%H%M%S')" -o threagile
# add the -race parameter to go build call in order to instrument with race condition detector: https://blog.golang.org/race-detector
# NOTE: copy files with final name to send to final build
RUN cp /app/demo/example/threagile.yaml /app/demo/example/threagile-example-model.yaml && \
cp /app/demo/stub/threagile.yaml /app/demo/stub/threagile-stub-model.yaml
RUN cp /app/demo/example/threagile.yaml /app/demo/example/threagile-example-model.yaml
RUN cp /app/demo/stub/threagile.yaml /app/demo/stub/threagile-stub-model.yaml

######
## Stage 3: Copy needed files into desired folder structure
######

FROM scratch AS files

COPY --from=build --chown=1000:1000 \
/app/threagile \
/app/raa.so \
/app/dummy.so \
/app/demo-rule.so \
/app/LICENSE.txt \
/app/report/template/background.pdf \
/app/support/openapi.yaml \
/app/support/schema.json \
/app/support/live-templates.txt \
/app/support/render-data-asset-diagram.sh \
/app/support/render-data-flow-diagram.sh \
/app/demo/example/threagile-example-model.yaml \
/app/demo/stub/threagile-stub-model.yaml \
\
/app/

COPY --from=build --chown=1000:1000 /app/server /app/server

######
## Stage 4: Make final small image
## Stage 3: Make final small image
######
FROM alpine
FROM alpine as deploy
WORKDIR /app

# label used in other scripts to filter
LABEL type="threagile"

# add certificates
RUN apk add --update --no-cache ca-certificates \
# add graphviz, fonts \
graphviz ttf-freefont \
# https://stackoverflow.com/questions/66963068/docker-alpine-executable-binary-not-found-even-if-in-path \
libc6-compat && \
RUN apk add --update --no-cache ca-certificates
# add graphviz, fonts
RUN apk add --update --no-cache graphviz ttf-freefont
# https://stackoverflow.com/questions/66963068/docker-alpine-executable-binary-not-found-even-if-in-path
RUN apk add libc6-compat
# https://stackoverflow.com/questions/34729748/installed-go-binary-not-found-in-path-on-alpine-linux-docker
# RUN mkdir -p /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2
# clean apk cache
rm -rf /var/cache/apk/* && \
# create application and data directories
mkdir -p /app /data && \
chown -R 1000:1000 /app /data
RUN rm -rf /var/cache/apk/*

COPY --from=files / /
RUN mkdir -p /app /data
RUN chown -R 1000:1000 /app /data

COPY --from=build --chown=1000:1000 /app/threagile /app/
COPY --from=build --chown=1000:1000 /app/raa_calc /app/
COPY --from=build --chown=1000:1000 /app/raa_dummy /app/
COPY --from=build --chown=1000:1000 /app/risk_demo_rule /app/
COPY --from=build --chown=1000:1000 /app/LICENSE.txt /app/
COPY --from=build --chown=1000:1000 /app/report/template/background.pdf /app/
COPY --from=build --chown=1000:1000 /app/support/openapi.yaml /app/
COPY --from=build --chown=1000:1000 /app/support/schema.json /app/
COPY --from=build --chown=1000:1000 /app/support/live-templates.txt /app/
COPY --from=build --chown=1000:1000 /app/demo/example/threagile-example-model.yaml /app/
COPY --from=build --chown=1000:1000 /app/demo/stub/threagile-stub-model.yaml /app/
COPY --from=build --chown=1000:1000 /app/server /app/server

USER 1000:1000
WORKDIR /app

ENV PATH=/app:$PATH \
GIN_MODE=release
ENV PATH=/app:$PATH GIN_MODE=release

ENTRYPOINT ["/app/threagile"]
CMD ["-help"]
CMD ["help"]
Loading
Loading