Skip to content

Commit

Permalink
Add logs and describe for pgo (#96)
Browse files Browse the repository at this point in the history
* Add logs and describe for pgo
* Release notes

Issues: [PGO-1175]
  • Loading branch information
benjaminjb authored May 9, 2024
1 parent 56a0c82 commit 0c7a778
Show file tree
Hide file tree
Showing 10 changed files with 116 additions and 13 deletions.
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,10 @@ The `pgo` CLI supports all actively maintained versions of PGO v5+.

For more about PGO, please see the
[PGO Documentation](https://access.crunchydata.com/documentation/postgres-operator/).

## Testing the Client

The PGO client has several KUTTL tests that run through some common scenarios.
In order to run these tests, we need a PGO operator or the equivalent running, i.e., `make deploy-dev`.

Note: the `support export` test requires a PGO operator running in the `postgres-operator` namespace.
2 changes: 1 addition & 1 deletion docs/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ defaultContentLanguageInSubdir= false
enableMissingTranslationPlaceholders = false

[params]
clientVersion = "0.4.1"
clientVersion = "0.4.2"

# crunchy-hugo-theme params
editURL = "https://github.com/CrunchyData/postgres-operator/edit/master/docs/content/"
Expand Down
9 changes: 9 additions & 0 deletions docs/content/reference/pgo_support_export.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ PostgresCluster.
jobs.batch [list]
limitranges [list]
namespaces [get]
networkpolicies.networking.k8s.io [list]
nodes [list]
persistentvolumeclaims [list]
poddisruptionbudgets.policy [list]
Expand Down Expand Up @@ -61,6 +62,11 @@ kubectl pgo support export daisy --output . --pg-logs-count 2
# This is only required when monitoring is not deployed in the PostgresCluster's namespace.
kubectl pgo support export daisy --monitoring-namespace another-namespace --output .
# Operator namespace override
# This is only required when the Operator is not deployed in the PostgresCluster's namespace.
# This is used for getting the logs and specs for the operator pod(s).
kubectl pgo support export daisy --operator-namespace another-namespace --output .
```
### Example output
```
Expand Down Expand Up @@ -91,11 +97,13 @@ Collecting services...
Collecting endpoints...
Collecting serviceaccounts...
Collecting ingresses...
Collecting networkpolicies...
Collecting limitranges...
Collecting events...
Collecting Postgres logs...
Collecting PostgresCluster pod logs...
Collecting monitoring pod logs...
Collecting operator pod logs...
Collecting Patroni info...
Collecting pgBackRest info...
Collecting processes...
Expand All @@ -114,6 +122,7 @@ Collecting PGO CLI logs...
```
-h, --help help for export
--monitoring-namespace string Monitoring namespace override
--operator-namespace string Operator namespace override
-o, --output string Path to save export tarball
-l, --pg-logs-count int Number of pg_log files to save (default 2)
```
Expand Down
2 changes: 1 addition & 1 deletion docs/content/reference/pgo_version.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pgo version
```
### Example output
```
Client Version: v0.4.1
Client Version: v0.4.2
Operator Version: v5.5.0
```

Expand Down
3 changes: 2 additions & 1 deletion docs/content/releases/0.4.1.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "0.4.1"
draft: false
weight: 996
weight: 995
---

[Crunchy Postgres for Kubernetes]: https://www.crunchydata.com/products/crunchy-postgresql-for-kubernetes
Expand All @@ -16,4 +16,5 @@ For more information about using the CLI and the various commands available, ple
Additionally, please see the [CPK documentation](https://access.crunchydata.com/documentation/postgres-operator/latest) for information about [getting started](https://access.crunchydata.com/documentation/postgres-operator/latest/quickstart/) with Crunchy Postgres for Kubernetes.

## Bug

- The `support export` command would fail fatally if the user didn't have permissions to list postgresclusters at the cluster-scope. This release fixes that so our postgrescluster list fails gracefully (like our other atttempts to gather Kubernetes resources).
20 changes: 20 additions & 0 deletions docs/content/releases/0.4.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "0.4.2"
draft: false
weight: 994
---

[Crunchy Postgres for Kubernetes]: https://www.crunchydata.com/products/crunchy-postgresql-for-kubernetes
[`pgo` CLI documentation]: https://access.crunchydata.com/documentation/postgres-operator-client/latest

Crunchy Data announces the release of `pgo`, Postgres Operator Client from Crunchy Data 0.4.2.

Built as a `kubectl` plugin, the `pgo` CLI facilitates the creation and management of PostgreSQL clusters created using [Crunchy Postgres for Kubernetes][].

For more information about using the CLI and the various commands available, please see the [`pgo` CLI documentation][].

Additionally, please see the [CPK documentation](https://access.crunchydata.com/documentation/postgres-operator/latest) for information about [getting started](https://access.crunchydata.com/documentation/postgres-operator/latest/quickstart/) with Crunchy Postgres for Kubernetes.

## Changes

- The `support export` command now captures logs from and descriptions for the PGO pod.
2 changes: 1 addition & 1 deletion internal/cmd/client_version.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@
package cmd

// store the current PGO CLI version
const clientVersion = "v0.4.1"
const clientVersion = "v0.4.2"
57 changes: 57 additions & 0 deletions internal/cmd/export.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ import (
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/selection"
"k8s.io/apimachinery/pkg/util/duration"
"k8s.io/cli-runtime/pkg/printers"
"k8s.io/client-go/discovery"
Expand Down Expand Up @@ -126,6 +128,22 @@ var clusterNamespacedResources = []schema.GroupVersionResource{{
Resource: "serviceaccounts",
}}

// Resources specifically for the operator;
// currently only pods, but leaving as is to allow expansion as requested.
var operatorNamespacedResources = []schema.GroupVersionResource{{
Group: appsv1.SchemeGroupVersion.Group,
Version: appsv1.SchemeGroupVersion.Version,
Resource: "deployments",
}, {
Group: appsv1.SchemeGroupVersion.Group,
Version: appsv1.SchemeGroupVersion.Version,
Resource: "replicasets",
}, {
Group: corev1.SchemeGroupVersion.Group,
Version: corev1.SchemeGroupVersion.Version,
Resource: "pods",
}}

// These "removed" GVRs are for making our CLI backwards compatible with older PGO versions.
var removedNamespacedResources = []schema.GroupVersionResource{{
Group: batchv1beta1.SchemeGroupVersion.Group,
Expand All @@ -142,6 +160,10 @@ var otherNamespacedResources = []schema.GroupVersionResource{{
Group: networkingv1.SchemeGroupVersion.Group,
Version: networkingv1.SchemeGroupVersion.Version,
Resource: "ingresses",
}, {
Group: networkingv1.SchemeGroupVersion.Group,
Version: networkingv1.SchemeGroupVersion.Version,
Resource: "networkpolicies",
}, {
Group: corev1.SchemeGroupVersion.Group,
Version: corev1.SchemeGroupVersion.Version,
Expand All @@ -168,6 +190,7 @@ PostgresCluster.
jobs.batch [list]
limitranges [list]
namespaces [get]
networkpolicies.networking.k8s.io [list]
nodes [list]
persistentvolumeclaims [list]
poddisruptionbudgets.policy [list]
Expand Down Expand Up @@ -215,6 +238,9 @@ PostgresCluster.
var monitoringNamespace string
cmd.Flags().StringVarP(&monitoringNamespace, "monitoring-namespace", "", "", "Monitoring namespace override")

var operatorNamespace string
cmd.Flags().StringVarP(&operatorNamespace, "operator-namespace", "", "", "Operator namespace override")

cmd.Args = cobra.ExactArgs(1)

cmd.Example = internal.FormatExample(`# Short Flags
Expand All @@ -227,6 +253,11 @@ kubectl pgo support export daisy --output . --pg-logs-count 2
# This is only required when monitoring is not deployed in the PostgresCluster's namespace.
kubectl pgo support export daisy --monitoring-namespace another-namespace --output .
# Operator namespace override
# This is only required when the Operator is not deployed in the PostgresCluster's namespace.
# This is used for getting the logs and specs for the operator pod(s).
kubectl pgo support export daisy --operator-namespace another-namespace --output .
### Example output
┌────────────────────────────────────────────────────────────────
| PGO CLI Support Export Tool
Expand Down Expand Up @@ -255,11 +286,13 @@ Collecting services...
Collecting endpoints...
Collecting serviceaccounts...
Collecting ingresses...
Collecting networkpolicies...
Collecting limitranges...
Collecting events...
Collecting Postgres logs...
Collecting PostgresCluster pod logs...
Collecting monitoring pod logs...
Collecting operator pod logs...
Collecting Patroni info...
Collecting pgBackRest info...
Collecting processes...
Expand Down Expand Up @@ -287,6 +320,7 @@ Collecting PGO CLI logs...
writeDebug(cmd, fmt.Sprintf("Flag - Output Directory: %s\n", outputDir))
writeDebug(cmd, fmt.Sprintf("Flag - Num Logs: %d\n", numLogs))
writeDebug(cmd, fmt.Sprintf("Flag - Monitoring Namespace: %s\n", monitoringNamespace))
writeDebug(cmd, fmt.Sprintf("Flag - Operator Namespace: %s\n", operatorNamespace))

namespace, err := config.Namespace()
if err != nil {
Expand Down Expand Up @@ -433,6 +467,29 @@ Collecting PGO CLI logs...
err = gatherPodLogs(ctx, clientset, monitoringNamespace, util.LabelMonitoring, "monitoring", tw, cmd)
}

// get operator Pod logs and descriptions
if operatorNamespace == "" {
operatorNamespace = namespace
}
// Operator and Operator upgrade pods should have
// "postgres-operator.crunchydata.com/control-plane" label
// but with different values
if err == nil {
req, _ := labels.NewRequirement(util.LabelOperator,
selection.Exists, []string{},
)
nsListOpts := metav1.ListOptions{
LabelSelector: req.String(),
}
err = gatherNamespacedAPIResources(ctx, dynamicClient,
operatorNamespace, "operator", operatorNamespacedResources,
nsListOpts, tw, cmd)
}
if err == nil {
writeInfo(cmd, "Collecting operator pod logs...")
err = gatherPodLogs(ctx, clientset, operatorNamespace, util.LabelOperator, "operator", tw, cmd)
}

// Exec resources
if err == nil {
err = gatherPatroniInfo(ctx, clientset, restConfig, namespace, clusterName, tw, cmd)
Expand Down
3 changes: 3 additions & 0 deletions internal/util/naming.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ const (

// LabelMonitoring is used to identify monitoring Pods
LabelMonitoring = "app.kubernetes.io/name=postgres-operator-monitoring"

// LabelOperator is used to identify operator Pods
LabelOperator = "postgres-operator.crunchydata.com/control-plane"
)

const (
Expand Down
24 changes: 15 additions & 9 deletions testing/kuttl/e2e/support-export/01--support_export.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- script: kubectl-pgo --namespace $NAMESPACE support export kuttl-support-cluster -o .
- script: kubectl-pgo --namespace $NAMESPACE --operator-namespace postgres-operator support export kuttl-support-cluster -o .
- script: tar -xzf ./crunchy_k8s_support_export_*.tar.gz
- script: |
CLEANUP="rm -r ./kuttl-support-cluster ./crunchy_k8s_support_export_*.tar.gz"
check_file() {
if [ ! -s ./kuttl-support-cluster/"${1}" ]
if [ ! -s ./"${1}" ]
then
echo "Expected ${1} file to not be empty"
eval "$CLEANUP"
Expand All @@ -29,23 +29,29 @@ commands:
}
# check that the cluster-names file exists and is not empty
check_file "cluster-names"
check_file "kuttl-support-cluster/cluster-names"
# check that the system-time file exists and is not empty
check_file "system-time"
check_file "kuttl-support-cluster/system-time"
# check that the context file exists and is not empty
check_file "current-context"
check_file "kuttl-support-cluster/current-context"
# check that the patroni info file exists and is not empty
check_file "patroni-info"
check_file "kuttl-support-cluster/patroni-info"
# check that the pgbackrest info file exists and is not empty
check_file "pgbackrest-info"
check_file "kuttl-support-cluster/pgbackrest-info"
# check that the plugin list file exists and is not empty
# the file will at least include kubectl-pgo
check_file "plugin-list"
check_file "kuttl-support-cluster/plugin-list"
# check that the operator file exists and is not empty
# the list file will not be empty for the requested Kubernetes types
check_file "operator/deployments/list"
check_file "operator/replicasets/list"
check_file "operator/pods/list"
# check for expected gzip compression level
FILE_INFO=$(file ./crunchy_k8s_support_export_*.tar.gz)
Expand Down Expand Up @@ -85,7 +91,7 @@ commands:
fi
# check that the events file exist and is not empty
check_file "events"
check_file "kuttl-support-cluster/events"
EVENTS="./kuttl-support-cluster/events"
# check that the events file contains the expected string
Expand Down

0 comments on commit 0c7a778

Please sign in to comment.