Skip to content

Commit

Permalink
feat: support private repository configuration (#265)
Browse files Browse the repository at this point in the history 
* feat: support private repository configuration

* feat: add tests to dependabot_file.py

* fix: remove prettier from Makefile

* fix: README lint

* fix: README lint

* fix: add suggestions

---------

Co-authored-by: JM (Jason Meridth) <jmeridth@github.com>
  • Loading branch information
@ricardojdsilva87 @jmeridth
ricardojdsilva87 and jmeridth authored Nov 16, 2024
1 parent 35040bd commit d5af164
Show file tree
Hide file tree
Showing 10 changed files with 586 additions and 187 deletions.
1 change: 1 addition & 0 deletions .env-example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ BATCH_SIZE = ""
BODY = ""
COMMIT_MESSAGE = ""
CREATED_AFTER_DATE = ""
DEPENDABOT_CONFIG_FILE = ""
DRY_RUN = ""
ENABLE_SECURITY_UPDATES = ""
EXEMPT_ECOSYSTEMS = ""
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -153,4 +153,7 @@ devenv.local.nix
# devenv
.envrc
devenv.*
.devenv*
.devenv*

# Local testing files
dependabot-output.yaml
56 changes: 54 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ This action can be configured to authenticate with GitHub App Installation or Pe
| `GH_APP_PRIVATE_KEY` | True | `""` | GitHub Application Private Key. See [documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app) for more details. |
| `GITHUB_APP_ENTERPRISE_ONLY` | False | false | Set this input to `true` if your app is created in GHE and communicates with GHE. |

The needed GitHub app permissions are the following:
The needed GitHub app permissions are the following under `Repository permissions`:

- `Administration` - Read and Write (Needed to activate the [automated security updates](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/configuring-dependabot-security-updates#managing-dependabot-security-updates-for-your-repositories) )
- `Pull Requests` - Read and Write (If `TYPE` input is set to `pull`)
Expand Down Expand Up @@ -125,6 +125,58 @@ The needed GitHub app permissions are the following:
| `SCHEDULE` | False | `weekly` | Schedule interval by which to check for dependency updates via Dependabot. Allowed values are `daily`, `weekly`, or `monthly` |
| `SCHEDULE_DAY` | False | '' | Scheduled day by which to check for dependency updates via Dependabot. Allowed values are days of the week full names (i.e., `monday`) |
| `LABELS` | False | "" | A comma separated list of labels that should be added to pull requests opened by dependabot. |
| `DEPENDABOT_CONFIG_FILE` | False | "" | Location of the configuration file for `dependabot.yml` configurations. If the file is present locally it takes precedence over the one in the repository. |

### Private repositories configuration

Dependabot allows the configuration of [private registries](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#configuration-options-for-private-registries) for dependabot to use.
To add a private registry configuration to the dependabot file the `DEPENDABOT_CONFIG_FILE` needs to be set with the path of the configuration file.

This configuration file needs to exist on the repository where the action runs. It can also be created locally to test some configurations (if created locally it takes precedence over the file on the repository).

#### Usage

Set the input variable:

```yaml
DEPENDABOT_CONFIG_FILE = "dependabot-config.yaml"
```

Create a file on your repository in the same path:

```yaml
npm:
type: "npm"
url: "https://yourprivateregistry/npm/"
username: "${{secrets.username}}"
password: "${{secrets.password}}"
key: <used if necessary>
token: <used if necessary>
replaces-base: <used if necessary>
maven:
type: "maven"
url: "https://yourprivateregistry/maven/"
username: "${{secrets.username}}"
password: "${{secrets.password}}"
```
The principal key of each configuration need to match the package managers that the [script is looking for](https://github.com/github/evergreen/blob/main/dependabot_file.py#L78).
The `dependabot.yaml` created file will look like the following with the `registries:` key added:

```yaml
updates:
- package-ecosystem: "npm"
directory: "/"
registries: --> added configuration
- 'npm' --> added configuration
schedule:
interval: "weekly"
labels:
- "test"
- "dependabot"
- "new"
```

### Example workflows

Expand Down Expand Up @@ -225,7 +277,7 @@ jobs:
GH_APP_PRIVATE_KEY: ${{ secrets.GH_APP_PRIVATE_KEY }}
# GITHUB_APP_ENTERPRISE_ONLY: True --> Set to true when created GHE App needs to communicate with GHE api
GH_ENTERPRISE_URL: ${{ github.server_url }}
# GH_TOKEN: ${{ steps.app-token.outputs.token }} --> the token input is not used if the github app inputs are set
# GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} --> the token input is not used if the github app inputs are set
ORGANIZATION: your_organization
UPDATE_EXISTING: True
GROUP_DEPENDENCIES: True
Expand Down
152 changes: 96 additions & 56 deletions dependabot_file.py
View file
Original file line number Diff line number Diff line change
@@ -1,52 +1,103 @@
"""This module contains the function to build the dependabot.yml file for a repo"""

import base64
import copy
import io

import github3
import yaml
import ruamel.yaml
from ruamel.yaml.scalarstring import SingleQuotedScalarString

# Define data structure for dependabot.yaml
data = {
"version": 2,
"registries": {},
"updates": [],
}

yaml = ruamel.yaml.YAML()
stream = io.StringIO()


def make_dependabot_config(
ecosystem, group_dependencies, indent, schedule, schedule_day, labels
ecosystem,
group_dependencies,
schedule,
schedule_day,
labels,
dependabot_config,
extra_dependabot_config,
) -> str:
"""
Make the dependabot configuration for a specific package ecosystem
Args:
ecosystem: the package ecosystem to make the dependabot configuration for
group_dependencies: whether to group dependencies in the dependabot.yml file
indent: the number of spaces to indent the dependabot configuration ex: " "
schedule: the schedule to run dependabot ex: "daily"
schedule_day: the day of the week to run dependabot ex: "monday" if schedule is "weekly"
labels: the list of labels to be added to dependabot configuration
dependabot_config: extra dependabot configs
extra_dependabot_config: File with the configuration to add dependabot configs (ex: private registries)
Returns:
str: the dependabot configuration for the package ecosystem
"""
schedule_day_line = ""
if schedule_day:
schedule_day_line += f"""
{indent}{indent}{indent}day: '{schedule_day}'"""

dependabot_config = f"""{indent}- package-ecosystem: '{ecosystem}'
{indent}{indent}directory: '/'
{indent}{indent}schedule:
{indent}{indent}{indent}interval: '{schedule}'{schedule_day_line}
"""
dependabot_config["updates"].append(
{
"package-ecosystem": SingleQuotedScalarString(ecosystem),
"directory": SingleQuotedScalarString("/"),
}
)

if extra_dependabot_config:
ecosystem_config = extra_dependabot_config.get(ecosystem)
if ecosystem_config:
dependabot_config["registries"][ecosystem] = ecosystem_config
dependabot_config["updates"][-1].update(
{"registries": [SingleQuotedScalarString(ecosystem)]}
)
else:
dependabot_config.pop("registries", None)

if schedule_day:
dependabot_config["updates"][-1].update(
{
"schedule": {
"interval": SingleQuotedScalarString(schedule),
"day": SingleQuotedScalarString(schedule_day),
},
}
)
else:
dependabot_config["updates"][-1].update(
{
"schedule": {"interval": SingleQuotedScalarString(schedule)},
}
)

if labels:
dependabot_config += f"""{indent}{indent}labels:
"""
quoted_labels = []
for label in labels:
dependabot_config += f"""{indent}{indent}{indent}- \"{label}\"
"""
quoted_labels.append(SingleQuotedScalarString(label))
dependabot_config["updates"][-1].update({"labels": quoted_labels})

if group_dependencies:
dependabot_config += f"""{indent}{indent}groups:
{indent}{indent}{indent}production-dependencies:
{indent}{indent}{indent}{indent}dependency-type: 'production'
{indent}{indent}{indent}development-dependencies:
{indent}{indent}{indent}{indent}dependency-type: 'development'
"""
return dependabot_config
dependabot_config["updates"][-1].update(
{
"groups": {
"production-dependencies": {
"dependency-type": SingleQuotedScalarString("production")
},
"development-dependencies": {
"dependency-type": SingleQuotedScalarString("development")
},
}
}
)

return yaml.dump(dependabot_config, stream)


def build_dependabot_file(
Expand All @@ -58,6 +109,7 @@ def build_dependabot_file(
schedule,
schedule_day,
labels,
extra_dependabot_config,
) -> str | None:
"""
Build the dependabot.yml file for a repo based on the repo contents
Expand All @@ -71,6 +123,7 @@ def build_dependabot_file(
schedule: the schedule to run dependabot ex: "daily"
schedule_day: the day of the week to run dependabot ex: "monday" if schedule is "daily"
labels: the list of labels to be added to dependabot configuration
extra_dependabot_config: File with the configuration to add dependabot configs (ex: private registries)
Returns:
str: the dependabot.yml file for the repo
Expand All @@ -89,30 +142,20 @@ def build_dependabot_file(
"github-actions": False,
"maven": False,
}
DEFAULT_INDENT = 2 # pylint: disable=invalid-name

# create a local copy in order to avoid overwriting the global exemption list
exempt_ecosystems_list = exempt_ecosystems.copy()
if existing_config:
dependabot_file = existing_config.decoded.decode("utf-8")
ecosystem_line = next(
line
for line in dependabot_file.splitlines()
if "- package-ecosystem:" in line
)
indent = " " * (len(ecosystem_line) - len(ecosystem_line.lstrip()))
if len(indent) < DEFAULT_INDENT:
print(
f"Invalid dependabot.yml file. No indentation found. Skipping {repo.full_name}"
)
return None
yaml.preserve_quotes = True
try:
dependabot_file = yaml.load(base64.b64decode(existing_config.content))
except ruamel.yaml.YAMLError as e:
print(f"YAML indentation error: {e}")
raise
else:
indent = " " * DEFAULT_INDENT
dependabot_file = """---
version: 2
updates:
"""
dependabot_file = copy.deepcopy(data)

add_existing_ecosystem_to_exempt_list(exempt_ecosystems_list, existing_config)
add_existing_ecosystem_to_exempt_list(exempt_ecosystems_list, dependabot_file)

# If there are repository specific exemptions,
# overwrite the global exemptions for this repo only
Expand Down Expand Up @@ -151,17 +194,14 @@ def build_dependabot_file(
try:
if repo.file_contents(file):
package_managers_found[manager] = True
# If the last thing in the file is not a newline,
# add one before adding a new language config to the file
if dependabot_file and dependabot_file[-1] != "\n":
dependabot_file += "\n"
dependabot_file += make_dependabot_config(
make_dependabot_config(
manager,
group_dependencies,
indent,
schedule,
schedule_day,
labels,
dependabot_file,
extra_dependabot_config,
)
break
except github3.exceptions.NotFoundError:
Expand All @@ -173,13 +213,14 @@ def build_dependabot_file(
for file in repo.directory_contents("/"):
if file[0].endswith(".tf"):
package_managers_found["terraform"] = True
dependabot_file += make_dependabot_config(
make_dependabot_config(
"terraform",
group_dependencies,
indent,
schedule,
schedule_day,
labels,
dependabot_file,
extra_dependabot_config,
)
break
except github3.exceptions.NotFoundError:
Expand All @@ -189,13 +230,14 @@ def build_dependabot_file(
for file in repo.directory_contents(".github/workflows"):
if file[0].endswith(".yml") or file[0].endswith(".yaml"):
package_managers_found["github-actions"] = True
dependabot_file += make_dependabot_config(
make_dependabot_config(
"github-actions",
group_dependencies,
indent,
schedule,
schedule_day,
labels,
dependabot_file,
extra_dependabot_config,
)
break
except github3.exceptions.NotFoundError:
Expand All @@ -212,7 +254,5 @@ def add_existing_ecosystem_to_exempt_list(exempt_ecosystems, existing_config):
to the exempt list so we don't get duplicate entries and maintain configuration settings
"""
if existing_config:
existing_config_obj = yaml.safe_load(existing_config.decoded)
if existing_config_obj:
for entry in existing_config_obj.get("updates", []):
exempt_ecosystems.append(entry["package-ecosystem"])
for entry in existing_config.get("updates", []):
exempt_ecosystems.append(entry["package-ecosystem"])
Loading

0 comments on commit d5af164

Please sign in to comment.