Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use terraform to maintain the organizational structure #14

Closed
wants to merge 39 commits into from
Closed

Conversation

cunla
Copy link
Contributor

@cunla cunla commented Jul 12, 2024

Hi,

This is a terraform org definition (P).
I tried various solutions, but I think this is the most flexible long term.

The idea is to have the organization defined in production.tfvars and to add users/repos/... - simply make changes to the one file.

I will create a github-action where on a PR it will list the changes and on merge it will apply them.

It is based on https://github.com/osinfra-io/github-organization-management.

Please review that this base-settings makes sense. I will add in the comment the initial TF plan.

terraform/main.tf Outdated Show resolved Hide resolved
@williln
Copy link

williln commented Jul 15, 2024

@cunla Would you mind adding something to the README to document this process?

Copy link
Contributor

@tim-schilling tim-schilling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cunla I'd be curious to get your thoughts on having a yaml membership file that would form the makeup of the teams and their memberships. It would mean we'd need to have a step that would translate that data into the production.tfvars structure.

yaml example:

django-commons-playground:
  members:
    - cunla
    - tim-schilling
    - priyapahwa
  teams:
    admins:
       members:
         - cunla
         - tim-schilling
    committers:
        members:
          - priyapahwa

The purpose behind this approach would be that it should be easier to understand what the actual team structure is.

terraform/README.md Outdated Show resolved Hide resolved
terraform/locals.tf Outdated Show resolved Hide resolved
terraform/tfvars/production.tfvars Outdated Show resolved Hide resolved
terraform/main.tf Show resolved Hide resolved
terraform/tfvars/production.tfvars Outdated Show resolved Hide resolved
@cunla
Copy link
Contributor Author

cunla commented Jul 20, 2024

@cunla I'd be curious to get your thoughts on having a yaml membership file that would form the makeup of the teams and their memberships. It would mean we'd need to have a step that would translate that data into the production.tfvars structure.

yaml example:

django-commons-playground:
  members:
    - cunla
    - tim-schilling
    - priyapahwa
  teams:
    admins:
       members:
         - cunla
         - tim-schilling
    committers:
        members:
          - priyapahwa

The purpose behind this approach would be that it should be easier to understand what the actual team structure is.

I understand terraform having its own way of representing a model sucks (Not sure why they did not go YAML), however, there are advantages to keeping production.tfvars in tf format since is the object structure is defined in locals.tf, and a good IDE can tell you (to an extent) if you are doing something wrong in production.tfvars

Copy link

github-actions bot commented Jul 20, 2024

Terraform plan in terraform
With var files: terraform/production/org.tfvars terraform/production/repositories.tfvars terraform/production/teams.tfvars
With variables: github_token = (sensitive value)

Plan: 3 to add, 12 to change, 1 to destroy.
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create
!~  update in-place
-   destroy

Terraform will perform the following actions:

  # github_organization_security_manager.this will be created
+   resource "github_organization_security_manager" "this" {
+       id        = (known after apply)
+       team_slug = (known after apply)
    }

  # github_repository.this[".github"] will be updated in-place
!~  resource "github_repository" "this" {
+       archive_on_destroy          = true
!~      delete_branch_on_merge      = false -> true
+       description                 = "A Special Repository."
!~      has_discussions             = false -> true
!~      has_wiki                    = true -> false
        id                          = ".github"
+       merge_commit_message        = "PR_TITLE"
+       merge_commit_title          = "MERGE_MESSAGE"
        name                        = ".github"
+       squash_merge_commit_message = "BLANK"
+       squash_merge_commit_title   = "PR_TITLE"
!~      vulnerability_alerts        = false -> true
#        (27 unchanged attributes hidden)

#        (1 unchanged block hidden)
    }

  # github_repository.this["controls"] will be updated in-place
!~  resource "github_repository" "this" {
+       archive_on_destroy          = true
!~      delete_branch_on_merge      = false -> true
!~      has_discussions             = false -> true
!~      has_wiki                    = true -> false
        id                          = "controls"
+       merge_commit_message        = "PR_TITLE"
+       merge_commit_title          = "MERGE_MESSAGE"
        name                        = "controls"
+       squash_merge_commit_message = "BLANK"
+       squash_merge_commit_title   = "PR_TITLE"
!~      vulnerability_alerts        = false -> true
#        (28 unchanged attributes hidden)

#        (1 unchanged block hidden)
    }

  # github_repository.this["django-commons-playground"] will be updated in-place
!~  resource "github_repository" "this" {
+       archive_on_destroy          = true
!~      delete_branch_on_merge      = false -> true
!~      has_discussions             = false -> true
!~      has_wiki                    = true -> false
        id                          = "django-commons-playground"
+       merge_commit_message        = "PR_TITLE"
+       merge_commit_title          = "MERGE_MESSAGE"
        name                        = "django-commons-playground"
+       squash_merge_commit_message = "BLANK"
+       squash_merge_commit_title   = "PR_TITLE"
!~      vulnerability_alerts        = false -> true
#        (28 unchanged attributes hidden)

#        (1 unchanged block hidden)
    }

  # github_repository.this["membership"] will be updated in-place
!~  resource "github_repository" "this" {
+       archive_on_destroy          = true
!~      delete_branch_on_merge      = false -> true
+       description                 = "Membership repository for the django-commons organization."
!~      has_wiki                    = true -> false
        id                          = "membership"
+       merge_commit_message        = "PR_TITLE"
+       merge_commit_title          = "MERGE_MESSAGE"
        name                        = "membership"
+       squash_merge_commit_message = "BLANK"
+       squash_merge_commit_title   = "PR_TITLE"
!~      vulnerability_alerts        = false -> true
#        (28 unchanged attributes hidden)

#        (1 unchanged block hidden)
    }

  # github_team.children["django-community-playground-admins"] will be updated in-place
!~  resource "github_team" "children" {
+       description               = "django-community-playground administrators"
        id                        = "9757650"
        name                      = "django-community-playground-admins"
#        (10 unchanged attributes hidden)
    }

  # github_team.children["django-community-playground-committers"] will be updated in-place
!~  resource "github_team" "children" {
+       description               = "django-community-playground committers"
        id                        = "9757668"
        name                      = "django-community-playground-committers"
#        (10 unchanged attributes hidden)
    }

  # github_team.parents["Admins"] will be updated in-place
!~  resource "github_team" "parents" {
+       description               = "django-commons administrators"
        id                        = "9763562"
        name                      = "Admins"
#        (10 unchanged attributes hidden)
    }

  # github_team.parents["django-community-playground"] will be updated in-place
!~  resource "github_team" "parents" {
+       description               = "django-community-playground team"
        id                        = "9757678"
        name                      = "django-community-playground"
#        (10 unchanged attributes hidden)
    }

  # github_team.parents["security-team"] will be created
+   resource "github_team" "parents" {
+       create_default_maintainer = false
+       description               = "django-commons security team"
+       etag                      = (known after apply)
+       id                        = (known after apply)
+       members_count             = (known after apply)
+       name                      = "security-team"
+       node_id                   = (known after apply)
+       parent_team_read_id       = (known after apply)
+       parent_team_read_slug     = (known after apply)
+       privacy                   = "closed"
+       slug                      = (known after apply)
    }

  # github_team_members.children["django-community-playground-admins"] will be updated in-place
!~  resource "github_team_members" "children" {
        id      = "9757650"
#        (1 unchanged attribute hidden)

-       members {
-           role     = "maintainer" -> null
-           username = "Stormheg" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "cunla" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "ryancheley" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "tim-schilling" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "williln" -> null
        }
+       members {
+           role     = "member"
+           username = "Stormheg"
        }
+       members {
+           role     = "member"
+           username = "cunla"
        }
+       members {
+           role     = "member"
+           username = "ryancheley"
        }
+       members {
+           role     = "member"
+           username = "tim-schilling"
        }
+       members {
+           role     = "member"
+           username = "williln"
        }
    }

  # github_team_members.children["django-community-playground-committers"] will be updated in-place
!~  resource "github_team_members" "children" {
        id      = "9757668"
#        (1 unchanged attribute hidden)

-       members {
-           role     = "maintainer" -> null
-           username = "Stormheg" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "cunla" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "ryancheley" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "tim-schilling" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "williln" -> null
        }

#        (1 unchanged block hidden)
    }

  # github_team_members.parents["Admins"] will be destroyed
  # (because key ["Admins"] is not in for_each map)
-   resource "github_team_members" "parents" {
-       id      = "9763562" -> null
-       team_id = "9763562" -> null

-       members {
-           role     = "maintainer" -> null
-           username = "Stormheg" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "cunla" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "ryancheley" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "tim-schilling" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "williln" -> null
        }
    }

  # github_team_members.parents["django-community-playground"] will be updated in-place
!~  resource "github_team_members" "parents" {
        id      = "9757678"
#        (1 unchanged attribute hidden)

-       members {
-           role     = "maintainer" -> null
-           username = "Stormheg" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "cunla" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "ryancheley" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "tim-schilling" -> null
        }
-       members {
-           role     = "maintainer" -> null
-           username = "williln" -> null
        }
+       members {
+           role     = "member"
+           username = "Stormheg"
        }
+       members {
+           role     = "member"
+           username = "cunla"
        }
+       members {
+           role     = "member"
+           username = "ryancheley"
        }
+       members {
+           role     = "member"
+           username = "tim-schilling"
        }
+       members {
+           role     = "member"
+           username = "williln"
        }

#        (1 unchanged block hidden)
    }

  # github_team_settings.this["django-community-playground"] will be updated in-place
!~  resource "github_team_settings" "this" {
        id        = "T_kwDOCaaRBM4AlOPu"
#        (3 unchanged attributes hidden)

+       review_request_delegation {
+           algorithm    = "LOAD_BALANCE"
+           member_count = 2
+           notify       = false
        }
    }

  # time_rotating.this will be created
+   resource "time_rotating" "this" {
+       day              = (known after apply)
+       hour             = (known after apply)
+       id               = (known after apply)
+       minute           = (known after apply)
+       month            = (known after apply)
+       rfc3339          = (known after apply)
+       rotation_days    = 5
+       rotation_rfc3339 = (known after apply)
+       second           = (known after apply)
+       unix             = (known after apply)
+       year             = (known after apply)
    }

Plan: 3 to add, 12 to change, 1 to destroy.

📝 Plan generated in Plan org changes and list them in a PR #25

…n does.

Removes the security template and other sections
that aren't 100% obviously critical.

We can always re-add these later, but teams can also
configure this manually.
This should make it easier to determine what a parent team
relates to versus what a child team relates to.
I didn't see it was used for the playground repo.
These need to be merged rather than concatenated.
Adds a comment to hopefully reduce confusion.
The Django Commons admin team will have access to all
teams by being owners in the organization. That isn't
controlled by the terraform plan.
@tim-schilling tim-schilling changed the title terraform org mgmt Use terraform to maintain the organizational structure Jul 24, 2024
@tim-schilling
Copy link
Contributor

@cunla I'm done making changes. Let me know what you think. My hope is that the naming scheme better reflects the purpose of the objects and collections rather than how they are utilized in GitHub's data model. This should make it easier for a person to come in and make a change without knowing much about terraform or GitHub's data model.

@tim-schilling
Copy link
Contributor

@Stormheg @ryancheley @williln The specific ask I'd have of you is to review the README changes (rendered page). It's been updated to reflect what Daniel's efforts would mean for maintaining repositories and memberships in GitHub.

I'd like to know if you're in one of the following categories:

  • You feel that this is reasonable enough to start with
  • You feel there are too many rough edges (please tell us what concerns you too)
  • You'd benefit from further clarification before joining one of the above camps (please tell us where we can help)


If they aren't a real human or reasonably trustworthy, close the issue.
2. If they are not a real human or not reasonably trustworthy, close the issue, asking for more information they are a human and not a spam bot. You can explain that by being a member, they can impact repositories immediately.
3. Add the user to the `members` collection in the [`terraform/production/org.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/org.tfvars) file.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. Add the user to the `members` collection in the [`terraform/production/org.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/org.tfvars) file.
3. Add the user's GitHub username to the `members` collection in the [`terraform/production/org.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/org.tfvars) file.

"new_user"
]
```
5. If they requested to be on specific repository team(s), in the [`terraform/production/teams.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/teams.tfvars) file, for the repository's key under `teams_repositories`, add them to the `members` collection.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
5. If they requested to be on specific repository team(s), in the [`terraform/production/teams.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/teams.tfvars) file, for the repository's key under `teams_repositories`, add them to the `members` collection.
5. If they requested to be on specific repository team(s), find that team's repository in the [`terraform/production/teams.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/teams.tfvars) file under `teams_repositories`. Add the new user to the `members` collection.

}
}
```
6. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
6. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
6. Create a pull request to the `main` branch. This will trigger terraform to plan the changes in the organization to be

```
6. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
executed. Review the changes and make sure they align with the request.
7. Merge the pull-request, it will trigger terraform to apply the changes in the organization.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
7. Merge the pull-request, it will trigger terraform to apply the changes in the organization.
7. Merge the pull request. This will trigger terraform to apply the changes to the organization.


## Team Change Playbook

1. If they are a real human and are reasonably trustworthy, comment "Approved" and close the issue manually
2. Add the member to requested team(s)
1. If they are not a real human or not reasonably trustworthy, close the issue, asking for more information they are a human and not a spam bot. You can explain that by being a member, they can impact repositories immediately.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically duplicated from the section "New Member Playbook." Should the New Member Playbook maybe refer to this section, instead of reproducing it? Something like, "If they requested to be on specific repository team(s), follow the instructions below in Team Change Playbook."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking here is that after X successful PRs, the existing admins/committers invite a person to have commit or admin rights on a repo. That's what flow this is attempting to cover which is separate from a new member.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tim-schilling Sounds good, thank you for clarifying!


1. Confirm with all existing admins that they are okay with the prospective admin
1. Confirm with all existing admins that they are okay with the change
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Confirm with all existing admins that they are okay with the change
1. Confirm with all existing admins that they approve changes to the repository admins or committers.


1. Confirm with all existing admins that they are okay with the prospective admin
1. Confirm with all existing admins that they are okay with the change
2. If there's disagreement, close the issue and ask for the admins to come to a consensus
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. If there's disagreement, close the issue and ask for the admins to come to a consensus
2. If there's disagreement, close the issue and ask for the admins to come to a consensus.

}
}
```
4. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
4. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
4. Create a pull request to `main` branch. This will trigger terraform to plan the changes in the organization to be

```
4. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
executed. Review the changes and make sure they align with the request.
5. Merge the pull-request, it will trigger terraform to apply the changes in the organization.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
5. Merge the pull-request, it will trigger terraform to apply the changes in the organization.
5. Merge the pull request. This will trigger terraform to apply the changes in the organization.


## New Project Playbook

1. Check if repository meets [inbound requirements](https://github.com/django-commons/membership/blob/main/incoming_repo_requirements.md)
1. Check if repository
meets [inbound requirements](https://github.com/django-commons/membership/blob/main/incoming_repo_requirements.md)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
meets [inbound requirements](https://github.com/django-commons/membership/blob/main/incoming_repo_requirements.md)
meets [inbound requirements](https://github.com/django-commons/membership/blob/main/incoming_repo_requirements.md).


Assuming repository name is `repo-name`:

1. In [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars), Add the new repository to the `repositories` section:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. In [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars), Add the new repository to the `repositories` section:
1. In [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars), add the new repository to the `repositories` section:

}
}
```
4. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
4. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
4. Create a pull request to the `main` branch. This will trigger terraform to plan the changes in the organization to be

1. Remove the repository from the `repositories` section in [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars)
2. Remove the parent team and child teams for the repository from the `teams_repositories` and `teams_repositories_privileged` sections in
[`terraform/production/teams.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/teams.tfvars)
3. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. Create a pull-request to `main` branch, it will trigger terraform to plan the changes in the organization to be
3. Create a pull request to the `main` branch. This will trigger terraform to plan the changes in the organization to be

@williln
Copy link

williln commented Jul 24, 2024

@cunla @tim-schilling I made some minor copyediting suggestions for the README for clarity, but nothing that should hold up the PR.

With the caveat that this is still pretty new territory for me, this all seems good enough to start with and I am +1 on merging and getting started. edit And thank you @cunla for all of this work!

@tim-schilling
Copy link
Contributor

Closing in favor of #15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants