Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: CPLYTM-224 producing OSCAL Profiles by level #419

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

feat: CPLYTM-224 producing OSCAL Profiles by level #419

wants to merge 2 commits into from

Conversation

hbraswelrh
Copy link
Contributor

@hbraswelrh hbraswelrh commented Jan 15, 2025 •
edited
Loading

Description

The sync_cac_content_profile command allows for user input to interact with both the SyncCacContentProfile task, and the AuthoredProfile task for creation of OSCAL Profiles organized by baseline level. The CLI will take user input of policy_id and filter_by_level and update the title of the produced OSCAL Profile. This functionality organizes OSCAL Profiles into a manageable separation of controls by baseline level. The oscal_catalog CLI option is specific to the Catalog that will be referenced for extracting the necessary data to produce an OSCAL Profile.

Note to reviewers:

The major changes were completed in these files:

  • trestlebot/cli/commands/sync_cac_content.py: the sync_cac_content_profile command was introduced here.
  • tests/trestlebot/cli/testd/test_sync_cac_content.py: the tests associated with sync_cac_content_profile are included here.
  • trestlebot/tasks/sync_cac_content_profile_task.py: changes made to include a task for syncing content of CaC repository for interaction with CLI and AuthoredProfile task for creation of OSCAL Profiles.
  • tests/trestlebot/tasks/test_sync_cac_content_profile_task.py: changes made to include a test for task which syncs content of CaC repository for interaction with CLI and AuthoredProfile task for creation of OSCAL Profiles.
  • trestlebot/tasks/authored/profile.py`: changes made to functionality of authoring profiles in trestle-bot and leveraging compliance-trestle resource to interact with the SyncCacContentProfile task.
  • tests/trestlebot/tasks/authored/test_profile.py: changes made to test functionality of authoring profiles in trestle-bot and leveraging compliance-trestle resource to interact with the SyncCacContentProfile task.

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this been tested?

  • GitHub CI was used for testing the creation of a temporary trestle-bot repository and storing the relevant input data. The unit tests for the sync_cac_content_profile command check if the profile path and expected OSCAL Profile is created. The GitHub CI tests have passed with all checks.
  • Test B

Test Configuration:

  • Firmware version: N40ET47W (1.29 )
  • Hardware: Lenovo ThinkPad P1 Gen 4i
  • Toolchain:
  • SDK:

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@hbraswelrh hbraswelrh changed the title CPLYTM 224 producing OSCAL Profiles by level (WIP #2) CPLYTM-224 producing OSCAL Profiles by level (WIP #2) Jan 16, 2025
@qduanmu qduanmu mentioned this pull request Jan 17, 2025
Draft
12 tasks
@hbraswelrh
feat: sync_cac_content command and task for OSCAL Profiles sorted by …
6bdd537 
…level

docs: applying suggestions from code review for authoring CI workflows

Co-authored-by: Jennifer Power <jpower@redhat.com>

fix: run the paths-filter step in its own job (complytime#370)

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

build(deps-dev): bump mkdocs-material from 9.5.37 to 9.5.43 (complytime#377)

Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.37 to 9.5.43.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](squidfunk/mkdocs-material@9.5.37...9.5.43)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

build(deps-dev): bump virtualenv from 20.26.5 to 20.27.1 (complytime#379)

Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.26.5 to 20.27.1.
- [Release notes](https://github.com/pypa/virtualenv/releases)
- [Changelog](https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst)
- [Commits](pypa/virtualenv@20.26.5...20.27.1)

---
updated-dependencies:
- dependency-name: virtualenv
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

build(deps): bump argcomplete from 3.5.0 to 3.5.1 (complytime#371)

Bumps [argcomplete](https://github.com/kislyuk/argcomplete) from 3.5.0 to 3.5.1.
- [Release notes](https://github.com/kislyuk/argcomplete/releases)
- [Changelog](https://github.com/kislyuk/argcomplete/blob/develop/Changes.rst)
- [Commits](kislyuk/argcomplete@v3.5.0...v3.5.1)

---
updated-dependencies:
- dependency-name: argcomplete
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

build(deps): bump httpcore from 1.0.5 to 1.0.6 (complytime#373)

Bumps [httpcore](https://github.com/encode/httpcore) from 1.0.5 to 1.0.6.
- [Release notes](https://github.com/encode/httpcore/releases)
- [Changelog](https://github.com/encode/httpcore/blob/master/CHANGELOG.md)
- [Commits](encode/httpcore@1.0.5...1.0.6)

---
updated-dependencies:
- dependency-name: httpcore
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

docs: add authoring tutorial (complytime#375)

* docs: add authoring tutorial

* docs: fix typos and add statement around pull request best practices

chore: add openssf scorecard workflow (complytime#359)

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

build(deps): bump compliance-trestle from 3.4.0 to 3.5.0 (complytime#380)

Bumps [compliance-trestle](https://github.com/oscal-compass/compliance-trestle) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/oscal-compass/compliance-trestle/releases)
- [Changelog](https://github.com/oscal-compass/compliance-trestle/blob/develop/CHANGELOG.md)
- [Commits](oscal-compass/compliance-trestle@v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: compliance-trestle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

docs: adr-001 cli implementation (complytime#347)

* docs: adding draft of CLI decision record

* docs: adding details around config file

* docs: refactor wording for clarity

* docs: update config example

* expand content for default behaviors around oscal-model

feat: 295 monorepo directory structure design proposal (complytime#389)

* initial directory structure organization

* feat: initial work on config and common options

* chore: add openssf scorecard workflow (complytime#359)

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

* build(deps): bump compliance-trestle from 3.4.0 to 3.5.0 (complytime#380)

Bumps [compliance-trestle](https://github.com/oscal-compass/compliance-trestle) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/oscal-compass/compliance-trestle/releases)
- [Changelog](https://github.com/oscal-compass/compliance-trestle/blob/develop/CHANGELOG.md)
- [Commits](oscal-compass/compliance-trestle@v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: compliance-trestle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: adds logic to load yaml config into click context to set defautl values.  improves config error handling.

* feat: adds debug logging statements

* feat: add markdown directory creation and call to compliance trestle init

* feat: simplify directory creation and better error handling for invalid configs

* feat: initial work on autosync

* Initial create command for click cli

* Initial create command for click cli

* adding unit test for config module

* adding unit test for config module

* Update autosync command

* Update autosync command

* feat: add ssp index option

* feat: add ssp index option

* add unit tests for init command

* add unit tests for init command

* feat: root call create and logging replacement

* feat: root call create and logging replacement

* feat: add upstream commands, fix common options decorators, expand config

* feat: add upstream commands, fix common options decorators, expand config

* Update autosync options and add tests

* Update autosync options and add tests

* docs: adr-001 cli implementation (complytime#347)

* docs: adding draft of CLI decision record

* docs: adding details around config file

* docs: refactor wording for clarity

* docs: update config example

* expand content for default behaviors around oscal-model

* feat: add logic to make_config for nested upstream model and update related tests

* feat: add logic to make_config for nested upstream model and update related tests

* feat: create command logic for compdef and ssp

* feat: create command logic for compdef and ssp

* feat: create command updates to prompts and logger messages

* feat: create command updates to prompts and logger messages

* feat: add default git info to init prompts and config

* feat: add default git info to init prompts and config

* fix hidden keep file creation

* fix hidden keep file creation

* Add rule-transform command and unit test

* Add rule-transform command and unit test

* feat: create command logic and adding unit tests

* feat: create command logic and adding unit tests

* Fix AttributeError, some misc updates

AttributeError: 'NoneType' object has no attribute 'encode'

* Fix AttributeError, some misc updates

AttributeError: 'NoneType' object has no attribute 'encode'

* feat: unit tests added for create command

* feat: unit tests added for create command

* refactor sync upstreams and autosync to match existing entrypoint syntax

* refactor sync upstreams and autosync to match existing entrypoint syntax

* Fix AttributeError, some misc updates

AttributeError: 'NoneType' object has no attribute 'encode'

* Fix AttributeError, some misc updates

AttributeError: 'NoneType' object has no attribute 'encode'

* feat: unit tests added for create command

* feat: unit tests added for create command

* fix: docstrings added for create command unit tests

* fix: docstrings added for create command unit tests

* add file pattern filter

* add file pattern filter

* fix: updated headers with license and copyright

* fix: updated headers with license and copyright

* fix: updated logger statements

* fix: updated logger statements

* fix: logger statements shortened

* fix: logger statements shortened

* fix: yaml default deletion

* fix: yaml default deletion

* docs: updates to reference the CLI commands in the README.md

* docs: updates to reference the CLI commands in the README.md

* feat: update for required ssp name

* feat: update for required ssp name

* Update trestlebot/cli/commands/init.py

Co-authored-by: Jennifer Power <barnabei.jennifer@gmail.com>

* Update trestlebot/cli/commands/init.py

Co-authored-by: Jennifer Power <barnabei.jennifer@gmail.com>

* Update trestlebot/cli/commands/init.py

Co-authored-by: Jennifer Power <barnabei.jennifer@gmail.com>

* Update trestlebot/cli/commands/init.py

Co-authored-by: Jennifer Power <barnabei.jennifer@gmail.com>

* fix typo in error msg

* fix typo in error msg

* fix help text for sync upstreams

* fix help text for sync upstreams

* fix: update for help text and testing location errors

* fix: update for help text and testing location errors

* fix: update for clarity on profile name for trestle workspace

* fix: update for clarity on profile name for trestle workspace

* Fix AssertionError, add missing register

* Fix AssertionError, add missing register

* fix: profile name prompting update

* fix: profile name prompting update

* feat: updating compdef list to required

* feat: updating compdef list to required

* docs: change of verbiage for readability

* docs: change of verbiage for readability

* docs: change to indicate trestle-bot as a cli tool

* docs: change to indicate trestle-bot as a cli tool

* feat: change to help description of create command

* feat: change to help description of create command

* docs: added high level folder structure for cli

* docs: added high level folder structure for cli

* fix: default value returned if no key in dictionary

* fix: default value returned if no key in dictionary

* feat: align skip-item option to skip-items

* feat: align skip-item option to skip-items

* fix: add missing git options in create command

* fix: add missing git options in create command

* fix: refactor testt and remove prompts

* fix: refactor testt and remove prompts

* fix: formatting issues and typos

* fix: formatting issues and typos

* chore: update poetry lock with latest dependencies

Signed-off-by: George Vauter <gvauter@redhat.com>

* chore: update poetry lock with latest dependencies

Signed-off-by: George Vauter <gvauter@redhat.com>

* fix: do not overwrite config path if set

Signed-off-by: George Vauter <gvauter@redhat.com>

* fix: do not overwrite config path if set

Signed-off-by: George Vauter <gvauter@redhat.com>

* fix: do not overwrite config path if set

Signed-off-by: George Vauter <gvauter@redhat.com>

* fix: do not overwrite config path if set

Signed-off-by: George Vauter <gvauter@redhat.com>

---------

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: George Vauter <gvauter@redhat.com>
Co-authored-by: Hannah Braswell <hbraswel@redhat.com>
Co-authored-by: Jennifer Power <barnabei.jennifer@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Qingmin Duanmu <qduanmu@redhat.com>

feat: initialize command for cac to oscal transformation

chore: update pyproject.toml entrypoints to cli root command

chore: update actions with new cli design

docs: update CONTRIBUTING.md

chore: update actions for debug and config options

chore: rename rule-transform to rules-transform

fix: update e2e test to use new commands

fix: update create command for e2e testing

Changes made to  use compdef_name reference instead of profile_name for model filter

fix: a typo in autosync command

fix: sys.exit with errorcode when exceptions

fix: fix a typo in cli root

feat: populate cac content product name as component title

Signed-off-by: Sophia Wang <huiwang@redhat.com>

Add unitest for populate cac product nameto component title

Signed-off-by: Sophia Wang <huiwang@redhat.com>

Move the ssg connections from utils to cac transformer

Signed-off-by: Sophia Wang <huiwang@redhat.com>

Add function to only update not recreate component definition if it exists

Signed-off-by: Sophia Wang <huiwang@redhat.com>

Update the component description as the product full name

Signed-off-by: Sophia Wang <huiwang@redhat.com>

add the sync cac content task to push local change to remote

Signed-off-by: Sophia Wang <huiwang@redhat.com>

chore: allow lower case in PR subject (complytime#406)

Signed-off-by: George Vauter <gvauter@redhat.com>

feat: add cac content rules transformation

feat: CPLYTM-421 create validation component from rules

Signed-off-by: Sophia Wang <huiwang@redhat.com>

chore: add notice regarding repo org move (complytime#413)

Signed-off-by: George Vauter <gvauter@redhat.com>

chore: update SyncCacContentTask

feat: add unit test for validation component

Signed-off-by: Sophia Wang <huiwang@redhat.com>

:arrow_up: bump actions/download-artifact from 4.1.7 to 4.1.8

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.7 to 4.1.8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@65a9edc...fa0a91b)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump actions/upload-artifact from 4.3.4 to 4.6.0

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.4 to 4.6.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@0b2256b...65c4c4a)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

feat: sync cac content profile task and subcommand

Add create_or_update to AuthoredProfile

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

test: adds tests and updates to sync_cac_profile_task

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

feat: sync cac content profile task and subcommand

fix: updating the cac_content_root type to pass a string

fix: fixes linting errors for sync_cac_content_profile task

Signed-off-by: Jennifer Power <barnabei.jennifer@gmail.com>

feat: sync cac content profile task and subcommand

feat: sync cac content profile task and subcommand

feat: testing repo passed before content dir

feat: testing repo passed before content dir

fix: unit tests in pycharm

PyCharm sets the CWD of unit tests to the tests directory by default,
but the unit tests assume a CWD of the repo root. This updates the unit
tests not to rely on the assumption of CWD

Signed-off-by: d10n <d10n@redhat.com>

feat: testing with catalog path extension

feat: testing for metadata checks

feat: testing for oscal profile imports

fix: testing oscal profile populated in correct path

fix: testing oscal profile populated in correct path

fix: testing oscal profile populated in correct path

fix: testing oscal profile populated in correct path

fix: testing oscal profile populated in correct path

fix: testing oscal profile populated in correct path

fix: pinpointing existing error 2

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: pinpointing error by not including a default for filter_by_level

fix: adding command to root

fix: adding modelutils handler for getting the correct catalog path

fix: adding modelutils handler for getting the correct catalog path

fix: taking catalog kwargs as part of path for profile data

fix: taking catalog kwargs as part of path for profile data

fix: taking catalog kwargs as part of path for profile data

fix: taking catalog kwargs as part of path for profile data

feat: updates for unit testing for sync_cac_content_profile command

feat: add parameter transformation

feat: update poetry.lock and add jinja macros

fix: improve the validation components with parameters

Signed-off-by: Sophia Wang <huiwang@redhat.com>

fix: fix test failure in validation component

feat: update rule description value with rule title

chore: create a minimalist macro file for unit tests

This file contains only the necessary macros used by rules in the
content_dir directory.

Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

chore: remove macros not relevant for current tests

This keeps the test content simpler and smaller.

Signed-off-by: Marcus Burghardt <maburgha@redhat.com>
hbraswelrh
hbraswelrh commented Jan 18, 2025
View reviewed changes
trestlebot/cli/commands/sync_cac_content.py
from typing import Any, List

import click
import trestle.oscal.catalog as cat
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loaded properly

hbraswelrh
hbraswelrh commented Jan 18, 2025
View reviewed changes
trestlebot/cli/commands/sync_cac_content.py
import click
import trestle.oscal.catalog as cat
from trestle.common.model_utils import ModelUtils
from trestle.core.models.file_content_type import FileContentType
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 13 and 14 are from most recent commit

hbraswelrh
hbraswelrh commented Jan 18, 2025
View reviewed changes
trestlebot/cli/commands/sync_cac_content.py
filter_by_level = kwargs.get("filter_by_level", list())

# get path here
oscal_catalog_path = ModelUtils.get_model_path_for_name_and_class(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correctly included recent commit oscal_catalog_path for the use of ModelUtils for the oscal_catalog as input

hbraswelrh
hbraswelrh commented Jan 18, 2025
View reviewed changes
trestlebot/tasks/sync_cac_content_profile_task.py
# accessing control file within content/controls
# the instance can use the methods within the ControlsManager() class

# TODO Ask Marcus to address use of get_policy in PR notes
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hbraswelrh Must delete these comments

@hbraswelrh hbraswelrh changed the title CPLYTM-224 producing OSCAL Profiles by level (WIP #2) feat: CPLYTM-224 producing OSCAL Profiles by level Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@hbraswelrh