Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding the Azure sync module functions along with new cloud client functionality #50366

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
a1c4bab
Protobuf and configuration for Access Graph Azure Discovery
mvbrock Dec 17, 2024
e09de37
Adding the Azure sync module functions along with new cloud client fu…
mvbrock Dec 17, 2024
25cff35
Forgot to decouple role definitions fetching function from the fetcher
mvbrock Dec 17, 2024
5f618ae
Moving reconciliation to the upstream azure sync PR
mvbrock Dec 17, 2024
4e018a0
Moving reconciliation test to the upstream azure sync PR
mvbrock Dec 17, 2024
e567074
Updating go.sum
mvbrock Dec 17, 2024
bffd6fd
Fixing rebase after protobuf gen
mvbrock Dec 18, 2024
158d893
Nolinting until upstream PRs
mvbrock Dec 19, 2024
bceb074
Updating to use existing msgraph client
mvbrock Dec 19, 2024
2b5b510
Adding protection around nil values
mvbrock Dec 20, 2024
c255c97
PR feedback
mvbrock Dec 20, 2024
3c2144e
Updating principal fetching to incorporate metadata from principal su…
mvbrock Dec 20, 2024
a282aff
Updating opts to not leak URL parameters
mvbrock Dec 30, 2024
b1ef8a8
Conformant package name
mvbrock Dec 31, 2024
41a57b4
Using variadic options
mvbrock Jan 6, 2025
91532ef
PR feedback
mvbrock Jan 6, 2025
be28854
Removing memberOf expansion
mvbrock Jan 6, 2025
a1dc8d4
Expanding memberships by calling memberOf on each user
mvbrock Jan 7, 2025
80c0f7a
Also returning expanded principals for improved readability
mvbrock Jan 7, 2025
8fa56d5
Removing ptrToList
mvbrock Jan 7, 2025
0d149ce
PR feedback
mvbrock Jan 9, 2025
801ef67
Rebase go.sum stuff
mvbrock Jan 9, 2025
948e4f7
Go mod tidy
mvbrock Jan 9, 2025
10f5018
Linting
mvbrock Jan 9, 2025
24c1f9d
Linting
mvbrock Jan 9, 2025
301278b
Collecting errors from fetching memberships and using a WithContext e…
mvbrock Jan 9, 2025
00b6191
Fixing go.mod
mvbrock Jan 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
connectrpc.com/connect v1.18.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v6 v6.3.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/msi/armmsi v1.2.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -668,6 +668,8 @@ github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.0/go.mod h1:eWRD7oawr1Mu1sLC
github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.1/go.mod h1:eWRD7oawr1Mu1sLCawqVc0CUiF43ia3qQMxLscsKQ9w=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 h1:Hp+EScFOu9HeCbeW8WU2yQPJd4gGwhMgKxWe+G6jNzw=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0/go.mod h1:/pz8dyNQe+Ey3yBp/XuYz7oqX8YDNWVpPB0hH3XWfbc=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0 h1:JAebRMoc3vL+Nd97GBprHYHucO4+wlW+tNbBIumqJlk=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0/go.mod h1:zflC9v4VfViJrSvcvplqws/yGXVbUEMZi/iHpZdSPWA=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5 v5.0.0 h1:5n7dPVqsWfVKw+ZiEKSd3Kzu7gwBkbEBkeXb8rgaE9Q=
Expand Down
1 change: 1 addition & 0 deletions integrations/event-handler/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v6 v6.3.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/msi/armmsi v1.2.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions integrations/event-handler/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,8 @@ github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0 h1:+m0M/LFxN43KvUL
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0/go.mod h1:PwOyop78lveYMRs6oCxjiVyBdyCgIYH6XHIVZO9/SFQ=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 h1:Hp+EScFOu9HeCbeW8WU2yQPJd4gGwhMgKxWe+G6jNzw=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0/go.mod h1:/pz8dyNQe+Ey3yBp/XuYz7oqX8YDNWVpPB0hH3XWfbc=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0 h1:JAebRMoc3vL+Nd97GBprHYHucO4+wlW+tNbBIumqJlk=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0/go.mod h1:zflC9v4VfViJrSvcvplqws/yGXVbUEMZi/iHpZdSPWA=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5 v5.0.0 h1:5n7dPVqsWfVKw+ZiEKSd3Kzu7gwBkbEBkeXb8rgaE9Q=
Expand Down
1 change: 1 addition & 0 deletions integrations/terraform/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v6 v6.3.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/msi/armmsi v1.2.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions integrations/terraform/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -644,6 +644,8 @@ github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0 h1:+m0M/LFxN43KvUL
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0/go.mod h1:PwOyop78lveYMRs6oCxjiVyBdyCgIYH6XHIVZO9/SFQ=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 h1:Hp+EScFOu9HeCbeW8WU2yQPJd4gGwhMgKxWe+G6jNzw=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0/go.mod h1:/pz8dyNQe+Ey3yBp/XuYz7oqX8YDNWVpPB0hH3XWfbc=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0 h1:JAebRMoc3vL+Nd97GBprHYHucO4+wlW+tNbBIumqJlk=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v6 v6.2.0/go.mod h1:zflC9v4VfViJrSvcvplqws/yGXVbUEMZi/iHpZdSPWA=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5 v5.0.0 h1:5n7dPVqsWfVKw+ZiEKSd3Kzu7gwBkbEBkeXb8rgaE9Q=
Expand Down
57 changes: 57 additions & 0 deletions lib/cloud/azure/roleassignments.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Teleport
* Copyright (C) 2024 Gravitational, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

package azure

import (
"context"

"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/arm"
"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2"
"github.com/gravitational/trace"
)

// RoleAssignmentsClient wraps the Azure API to provide a high level subset of functionality
type RoleAssignmentsClient struct {
cli *armauthorization.RoleAssignmentsClient
}

// NewRoleAssignmentsClient creates a new client for a given subscription and credentials
func NewRoleAssignmentsClient(subscription string, cred azcore.TokenCredential, options *arm.ClientOptions) (*RoleAssignmentsClient, error) {
clientFactory, err := armauthorization.NewClientFactory(subscription, cred, options)
if err != nil {
return nil, trace.Wrap(err)
}
roleDefCli := clientFactory.NewRoleAssignmentsClient()
return &RoleAssignmentsClient{cli: roleDefCli}, nil
}

// ListRoleAssignments returns role assignments for a given scope
func (c *RoleAssignmentsClient) ListRoleAssignments(ctx context.Context, scope string) ([]*armauthorization.RoleAssignment, error) {
pager := c.cli.NewListForScopePager(scope, nil)
var roleDefs []*armauthorization.RoleAssignment
for pager.More() {
page, err := pager.NextPage(ctx)
if err != nil {
return nil, trace.Wrap(err)
}
roleDefs = append(roleDefs, page.Value...)
}
return roleDefs, nil
}
57 changes: 57 additions & 0 deletions lib/cloud/azure/roledefinitions.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Teleport
* Copyright (C) 2024 Gravitational, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

package azure

import (
"context"

"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/arm"
"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2"
"github.com/gravitational/trace"
)

// RoleDefinitionsClient wraps the Azure API to provide a high level subset of functionality
type RoleDefinitionsClient struct {
cli *armauthorization.RoleDefinitionsClient
}

// NewRoleDefinitionsClient creates a new client for a given subscription and credentials
func NewRoleDefinitionsClient(subscription string, cred azcore.TokenCredential, options *arm.ClientOptions) (*RoleDefinitionsClient, error) {
clientFactory, err := armauthorization.NewClientFactory(subscription, cred, options)
if err != nil {
return nil, trace.Wrap(err)
}
roleDefCli := clientFactory.NewRoleDefinitionsClient()
return &RoleDefinitionsClient{cli: roleDefCli}, nil
}

// ListRoleDefinitions returns role definitions for a given scope
func (c *RoleDefinitionsClient) ListRoleDefinitions(ctx context.Context, scope string) ([]*armauthorization.RoleDefinition, error) {
mvbrock marked this conversation as resolved.
Show resolved Hide resolved
pager := c.cli.NewListPager(scope, nil)
var roleDefs []*armauthorization.RoleDefinition
for pager.More() {
page, err := pager.NextPage(ctx)
if err != nil {
return nil, trace.Wrap(err)
}
roleDefs = append(roleDefs, page.Value...)
}
return roleDefs, nil
}
28 changes: 27 additions & 1 deletion lib/cloud/clients.go
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,10 @@ type azureClients struct {
azurePostgresFlexServersClients azure.ClientMap[azure.PostgresFlexServersClient]
// azureRunCommandClients contains the cached Azure Run Command clients.
azureRunCommandClients azure.ClientMap[azure.RunCommandClient]
// azureRoleDefinitionsClients contains the cached Azure Role Definitions clients.
azureRoleDefinitionsClients azure.ClientMap[azure.RoleDefinitionsClient]
// azureRoleAssignmentsClients contains the cached Azure Role Assignments clients.
azureRoleAssignmentsClients azure.ClientMap[azure.RoleAssignmentsClient]
}

// credentialsSource defines where the credentials must come from.
Expand Down Expand Up @@ -743,6 +747,16 @@ func (c *cloudClients) GetAzureRunCommandClient(subscription string) (azure.RunC
return c.azureRunCommandClients.Get(subscription, c.GetAzureCredential)
}

// GetAzureRoleDefinitionsClient returns an Azure Role Definitions client
func (c *cloudClients) GetAzureRoleDefinitionsClient(subscription string) (azure.RoleDefinitionsClient, error) {
return c.azureRoleDefinitionsClients.Get(subscription, c.GetAzureCredential)
}

// GetAzureRoleAssignmentsClient returns an Azure Role Assignments client
func (c *cloudClients) GetAzureRoleAssignmentsClient(subscription string) (azure.RoleAssignmentsClient, error) {
return c.azureRoleAssignmentsClients.Get(subscription, c.GetAzureCredential)
}

// Close closes all initialized clients.
func (c *cloudClients) Close() (err error) {
c.mtx.Lock()
Expand Down Expand Up @@ -1050,6 +1064,8 @@ type TestCloudClients struct {
AzureMySQLFlex azure.MySQLFlexServersClient
AzurePostgresFlex azure.PostgresFlexServersClient
AzureRunCommand azure.RunCommandClient
AzureRoleDefinitions azure.RoleDefinitionsClient
AzureRoleAssignments azure.RoleAssignmentsClient
}

// GetAWSSession returns AWS session for the specified region, optionally
Expand Down Expand Up @@ -1294,11 +1310,21 @@ func (c *TestCloudClients) GetAzurePostgresFlexServersClient(subscription string
return c.AzurePostgresFlex, nil
}

// GetAzureRunCommand returns an Azure Run Command client for the given subscription.
// GetAzureRunCommandClient returns an Azure Run Command client for the given subscription.
func (c *TestCloudClients) GetAzureRunCommandClient(subscription string) (azure.RunCommandClient, error) {
return c.AzureRunCommand, nil
}

// GetAzureRoleDefinitionsClient returns an Azure Role Definitions client for the given subscription.
func (c *TestCloudClients) GetAzureRoleDefinitionsClient(subscription string) (azure.RoleDefinitionsClient, error) {
return c.AzureRoleDefinitions, nil
}

// GetAzureRoleAssignmentsClient returns an Azure Role Assignments client for the given subscription.
func (c *TestCloudClients) GetAzureRoleAssignmentsClient(subscription string) (azure.RoleAssignmentsClient, error) {
return c.AzureRoleAssignments, nil
}

// Close closes all initialized clients.
func (c *TestCloudClients) Close() error {
return nil
Expand Down
5 changes: 5 additions & 0 deletions lib/msgraph/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ type GroupMember interface {
isGroupMember()
}

type Membership struct {
Type string `json:"@odata.type"`
ID string `json:"id"`
}

type DirectoryObject struct {
ID *string `json:"id,omitempty"`
DisplayName *string `json:"displayName,omitempty"`
Expand Down
16 changes: 16 additions & 0 deletions lib/msgraph/paginated.go
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,22 @@ func (c *Client) IterateUsers(ctx context.Context, f func(*User) bool) error {
return iterateSimple(c, ctx, "users", f)
}

// IterateServicePrincipals lists all service principals in the Entra ID directory using pagination.
// `f` will be called for each object in the result set.
// if `f` returns `false`, the iteration is stopped (equivalent to `break` in a normal loop).
// Ref: [https://learn.microsoft.com/en-us/graph/api/serviceprincipal-list].
func (c *Client) IterateServicePrincipals(ctx context.Context, f func(principal *ServicePrincipal) bool) error {
return iterateSimple(c, ctx, "servicePrincipals", f)
}

// IterateUserMembership lists all group memberships for a given user ID as directory objects.
// `f` will be called for each directory object in the result set.
// if `f` returns `false`, the iteration is stopped (equivalent to `break` in a normal loop).
// Ref: [https://learn.microsoft.com/en-us/graph/api/group-list-memberof].
func (c *Client) IterateUserMembership(ctx context.Context, userID string, f func(object *DirectoryObject) bool) error {
return iterateSimple(c, ctx, path.Join("users", userID, "memberOf"), f)
}

// IterateGroupMembers lists all members for the given Entra ID group using pagination.
// `f` will be called for each object in the result set.
// if `f` returns `false`, the iteration is stopped (equivalent to `break` in a normal loop).
Expand Down
58 changes: 58 additions & 0 deletions lib/srv/discovery/fetchers/azure-sync/memberships.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
* Teleport
* Copyright (C) 2025 Gravitational, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

package azuresync

import (
"context"

"github.com/gravitational/trace"
"golang.org/x/sync/errgroup"

accessgraphv1alpha "github.com/gravitational/teleport/gen/proto/go/accessgraph/v1alpha"
"github.com/gravitational/teleport/lib/msgraph"
)

const parallelism = 10 //nolint:unused // invoked in a dependent PR

func expandMemberships(ctx context.Context, cli *msgraph.Client, principals []*accessgraphv1alpha.AzurePrincipal) ([]*accessgraphv1alpha.AzurePrincipal, error) { //nolint:unused // invoked in a dependent PR
eg, _ := errgroup.WithContext(ctx)
eg.SetLimit(parallelism)
errCh := make(chan error, len(principals))
for _, principal := range principals {
eg.Go(func() error {
err := cli.IterateUserMembership(ctx, principal.Id, func(obj *msgraph.DirectoryObject) bool {
principal.MemberOf = append(principal.MemberOf, *obj.ID)
return true
})
if err != nil {
errCh <- err
}
return nil
})
}
_ = eg.Wait()
var errs []error
for chErr := range errCh {
errs = append(errs, chErr)
}
if len(errs) > 0 {
return nil, trace.NewAggregate(errs...)
}
return principals, nil
}
Loading
Loading