Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: generate coverage profiles for e2e #1631

Merged
merged 3 commits into from
Apr 8, 2024
Merged

feat: generate coverage profiles for e2e #1631

merged 3 commits into from
Apr 8, 2024

Conversation

FrankYang0529
Copy link
Contributor

@FrankYang0529 FrankYang0529 marked this pull request as ready for review December 20, 2023 09:05
@FrankYang0529 FrankYang0529 requested a review from a team as a code owner December 20, 2023 09:05
@FrankYang0529
Copy link
Contributor Author

Sample pipeline: https://ci.longhorn.io/job/public/job/v1.2.x/job/v1.2.x-longhorn-tests-sles-amd64/340/

yangchiu
yangchiu requested changes Dec 22, 2023
View reviewed changes
test_framework/scripts/longhorn-coverage.sh
uninstall_longhorn_by_chart

# install longhorn again to copy coverage data from pods, so we don't need scp files from host
install_longhorn_by_chart
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help to elaborate more on this part? Like when will the coverage data be generated? Upon Longhorn uninstallation? And where will be the coverage data (or the go-cover-dir folder) located? On the worker nodes?

Since I can find the uninstall_longhorn_by_chart and install_longhorn_by_chart are failed in https://ci.longhorn.io/job/public/job/v1.2.x/job/v1.2.x-longhorn-tests-sles-amd64/340/:

test_framework/scripts/longhorn-coverage.sh: line 6: pipelines/utilities/longhorn-setup.sh: No such file or directory
+ main
+ set_kubeconfig
+ [[ amd64 == \a\m\d\6\4 ]]
+ [[ k3s == \r\k\e ]]
+ [[ k3s == \r\k\e\2 ]]
+ export KUBECONFIG=/src/longhorn-tests/test_framework/terraform/aws/sles/k3s.yaml
+ KUBECONFIG=/src/longhorn-tests/test_framework/terraform/aws/sles/k3s.yaml
+ uninstall_longhorn_by_chart
test_framework/scripts/longhorn-coverage.sh: line 63: uninstall_longhorn_by_chart: command not found
+ install_longhorn_by_chart
test_framework/scripts/longhorn-coverage.sh: line 66: install_longhorn_by_chart: command not found
+ copy_go-cover-dir_from_all_longhorn-manager_pods

But the coverage data are still copied and published successfully.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Thanks.

I added more content for it. The program write coverage data when it invokes os.Exit() or returns normally from main.main. If there is any go programs exit, we can get coverage data.

The data is located on each node path /go-cover-dir. Also, the data will be mounted to each LH pod on /go-cover-dir path. I don't want to use scp to collect data, so I install LH again to use kubectl cp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last build fix the issue. Thanks.

https://ci.longhorn.io/job/public/job/v1.2.x/job/v1.2.x-longhorn-tests-sles-amd64/342/

Copy link
Member

@yangchiu yangchiu Dec 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why Longhorn isn't uninstalled before but we can still get coverage data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Installing LH in a new cluster and found that there is some crashloop, but it can be fixed finally. In this situation, we have some program exits, so we can get some coverage data. Also, we run longhorn-manager-test:master-head. I'm not sure the test case in it, but if it creates, mounts, and destroys volume, we will get longhorn-engine coverage data.

Following is watching pod result when we install a LH in a new cluster:

> kgpo -n longhorn-system -w
NAME                     READY   STATUS    RESTARTS   AGE
longhorn-manager-crzcg   0/1     Pending   0          0s
longhorn-ui-5d95cff4fd-lcqml   0/1     Pending   0          0s
longhorn-driver-deployer-5c4cb88bf-d2gkj   0/1     Pending   0          0s
longhorn-manager-fpc29                     0/1     Pending   0          0s
longhorn-ui-5d95cff4fd-l674b               0/1     Pending   0          0s
longhorn-manager-crzcg                     0/1     Pending   0          0s
longhorn-ui-5d95cff4fd-lcqml               0/1     Pending   0          0s
longhorn-driver-deployer-5c4cb88bf-d2gkj   0/1     Pending   0          0s
longhorn-ui-5d95cff4fd-l674b               0/1     Pending   0          0s
longhorn-manager-fpc29                     0/1     Pending   0          0s
longhorn-ui-5d95cff4fd-lcqml               0/1     ContainerCreating   0          0s
longhorn-manager-crzcg                     0/1     ContainerCreating   0          0s
longhorn-ui-5d95cff4fd-l674b               0/1     ContainerCreating   0          0s
longhorn-driver-deployer-5c4cb88bf-d2gkj   0/1     Init:0/1            0          0s
longhorn-manager-fpc29                     0/1     ContainerCreating   0          0s
longhorn-ui-5d95cff4fd-l674b               0/1     ContainerCreating   0          2s
longhorn-manager-crzcg                     0/1     ContainerCreating   0          2s
longhorn-manager-fpc29                     0/1     ContainerCreating   0          2s
longhorn-ui-5d95cff4fd-lcqml               0/1     ContainerCreating   0          2s
longhorn-driver-deployer-5c4cb88bf-d2gkj   0/1     Init:0/1            0          2s
longhorn-manager-crzcg                     0/1     Running             0          24s
longhorn-manager-crzcg                     1/1     Running             0          25s
longhorn-manager-crzcg                     0/1     Error               0          29s
longhorn-ui-5d95cff4fd-lcqml               1/1     Running             0          30s
longhorn-manager-crzcg                     0/1     Error               1 (2s ago)   30s
longhorn-manager-crzcg                     0/1     CrashLoopBackOff    1 (2s ago)   32s
longhorn-ui-5d95cff4fd-l674b               1/1     Running             0            46s
longhorn-manager-crzcg                     0/1     Error               2 (17s ago)   47s
longhorn-driver-deployer-5c4cb88bf-d2gkj   0/1     Init:0/1            0             51s
longhorn-manager-crzcg                     0/1     CrashLoopBackOff    2 (5s ago)    52s
longhorn-manager-fpc29                     0/1     Running             0             53s
longhorn-manager-fpc29                     1/1     Running             0             53s
engine-image-ei-b907910b-g2n4k             0/1     Pending             0             0s
engine-image-ei-b907910b-g2n4k             0/1     Pending             0             0s
engine-image-ei-b907910b-9gwmw             0/1     Pending             0             0s
engine-image-ei-b907910b-9gwmw             0/1     Pending             0             0s
engine-image-ei-b907910b-g2n4k             0/1     ContainerCreating   0             0s
engine-image-ei-b907910b-9gwmw             0/1     ContainerCreating   0             0s
instance-manager-2dc99d1ff3ea6bb41b72639d4ffb61f9   0/1     Pending             0             0s
instance-manager-2dc99d1ff3ea6bb41b72639d4ffb61f9   0/1     ContainerCreating   0             0s
engine-image-ei-b907910b-g2n4k                      0/1     ContainerCreating   0             0s
engine-image-ei-b907910b-9gwmw                      0/1     ContainerCreating   0             0s
instance-manager-2dc99d1ff3ea6bb41b72639d4ffb61f9   0/1     ContainerCreating   0             0s
longhorn-driver-deployer-5c4cb88bf-d2gkj            0/1     PodInitializing     0             57s
discover-proc-kubelet-cmdline                       0/1     Pending             0             0s
discover-proc-kubelet-cmdline                       0/1     Pending             0             0s
discover-proc-kubelet-cmdline                       0/1     ContainerCreating   0             0s
discover-proc-kubelet-cmdline                       0/1     ContainerCreating   0             1s
longhorn-driver-deployer-5c4cb88bf-d2gkj            1/1     Running             0             58s
discover-proc-kubelet-cmdline                       0/1     Completed           0             1s
discover-proc-kubelet-cmdline                       0/1     Completed           0             2s
discover-proc-kubelet-cmdline                       0/1     Completed           0             3s
discover-proc-kubelet-cmdline                       0/1     Terminating         0             3s
discover-proc-kubelet-cmdline                       0/1     Terminating         0             3s
csi-attacher-5c4bfdcf59-m77g4                       0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-7fzkf                       0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-m77g4                       0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-vhrhm                       0/1     Pending             0             0s
csi-provisioner-667796df57-mlfd2                    0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-7fzkf                       0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-vhrhm                       0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-m77g4                       0/1     ContainerCreating   0             0s
csi-provisioner-667796df57-6xl8d                    0/1     Pending             0             0s
csi-provisioner-667796df57-8bgrd                    0/1     Pending             0             0s
csi-provisioner-667796df57-mlfd2                    0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-7fzkf                       0/1     ContainerCreating   0             0s
csi-provisioner-667796df57-8bgrd                    0/1     Pending             0             0s
csi-attacher-5c4bfdcf59-vhrhm                       0/1     ContainerCreating   0             0s
csi-provisioner-667796df57-6xl8d                    0/1     Pending             0             0s
csi-resizer-694f8f5f64-q2nnj                        0/1     Pending             0             0s
csi-resizer-694f8f5f64-q2nnj                        0/1     Pending             0             0s
csi-resizer-694f8f5f64-d9fnx                        0/1     Pending             0             0s
csi-resizer-694f8f5f64-xt955                        0/1     Pending             0             0s
csi-provisioner-667796df57-8bgrd                    0/1     ContainerCreating   0             0s
csi-snapshotter-959b69d4b-qzp8c                     0/1     Pending             0             0s
csi-resizer-694f8f5f64-xt955                        0/1     Pending             0             0s
csi-provisioner-667796df57-6xl8d                    0/1     ContainerCreating   0             0s
csi-resizer-694f8f5f64-d9fnx                        0/1     Pending             0             0s
csi-snapshotter-959b69d4b-btkb9                     0/1     Pending             0             0s
csi-snapshotter-959b69d4b-crj5g                     0/1     Pending             0             0s
csi-snapshotter-959b69d4b-qzp8c                     0/1     Pending             0             0s
csi-provisioner-667796df57-mlfd2                    0/1     ContainerCreating   0             0s
csi-snapshotter-959b69d4b-btkb9                     0/1     Pending             0             0s
csi-snapshotter-959b69d4b-crj5g                     0/1     Pending             0             0s
csi-resizer-694f8f5f64-d9fnx                        0/1     ContainerCreating   0             0s
longhorn-csi-plugin-2bhzb                           0/3     Pending             0             0s
csi-resizer-694f8f5f64-xt955                        0/1     ContainerCreating   0             0s
csi-snapshotter-959b69d4b-btkb9                     0/1     ContainerCreating   0             0s
longhorn-csi-plugin-9l48p                           0/3     Pending             0             0s
longhorn-csi-plugin-2bhzb                           0/3     Pending             0             0s
longhorn-csi-plugin-9l48p                           0/3     Pending             0             0s
csi-snapshotter-959b69d4b-qzp8c                     0/1     ContainerCreating   0             0s
csi-resizer-694f8f5f64-q2nnj                        0/1     ContainerCreating   0             0s
longhorn-csi-plugin-9l48p                           0/3     ContainerCreating   0             0s
longhorn-csi-plugin-2bhzb                           0/3     ContainerCreating   0             0s
csi-snapshotter-959b69d4b-crj5g                     0/1     ContainerCreating   0             0s
csi-attacher-5c4bfdcf59-7fzkf                       0/1     ContainerCreating   0             1s
csi-provisioner-667796df57-6xl8d                    0/1     ContainerCreating   0             1s
csi-resizer-694f8f5f64-d9fnx                        0/1     ContainerCreating   0             1s
csi-attacher-5c4bfdcf59-m77g4                       0/1     ContainerCreating   0             1s
csi-snapshotter-959b69d4b-btkb9                     0/1     ContainerCreating   0             1s
csi-attacher-5c4bfdcf59-vhrhm                       0/1     ContainerCreating   0             1s
csi-resizer-694f8f5f64-q2nnj                        0/1     ContainerCreating   0             1s
longhorn-csi-plugin-9l48p                           0/3     ContainerCreating   0             1s
csi-provisioner-667796df57-mlfd2                    0/1     ContainerCreating   0             1s
csi-provisioner-667796df57-8bgrd                    0/1     ContainerCreating   0             1s
csi-resizer-694f8f5f64-xt955                        0/1     ContainerCreating   0             1s
longhorn-csi-plugin-2bhzb                           0/3     ContainerCreating   0             1s
csi-snapshotter-959b69d4b-qzp8c                     0/1     ContainerCreating   0             1s
csi-snapshotter-959b69d4b-crj5g                     0/1     ContainerCreating   0             1s
longhorn-manager-crzcg                              0/1     Running             3 (28s ago)   75s
longhorn-manager-crzcg                              1/1     Running             3 (28s ago)   75s
instance-manager-843e3dde57d65c0384c81f20f0480779   0/1     Pending             0             0s
instance-manager-843e3dde57d65c0384c81f20f0480779   0/1     ContainerCreating   0             0s
instance-manager-843e3dde57d65c0384c81f20f0480779   0/1     ContainerCreating   0             1s
engine-image-ei-b907910b-9gwmw                      0/1     Running             0             46s
engine-image-ei-b907910b-g2n4k                      0/1     Running             0             47s
engine-image-ei-b907910b-g2n4k                      1/1     Running             0             49s
engine-image-ei-b907910b-9gwmw                      1/1     Running             0             50s
csi-resizer-694f8f5f64-q2nnj                        1/1     Running             0             61s
-instance-manager-2dc99d1ff3ea6bb41b72639d4ffb61f9   1/1     Running             0             72s
csi-attacher-5c4bfdcf59-m77g4                       1/1     Running             0             71s
csi-provisioner-667796df57-8bgrd                    1/1     Running             0             82s
csi-provisioner-667796df57-6xl8d                    1/1     Running             0             83s
csi-attacher-5c4bfdcf59-vhrhm                       1/1     Running             0             84s
csi-resizer-694f8f5f64-xt955                        1/1     Running             0             86s
csi-resizer-694f8f5f64-q2nnj                        0/1     Error               0             91s
csi-resizer-694f8f5f64-q2nnj                        1/1     Running             1 (2s ago)    92s
csi-attacher-5c4bfdcf59-7fzkf                       1/1     Running             0             92s
csi-provisioner-667796df57-mlfd2                    1/1     Running             0             97s
csi-resizer-694f8f5f64-d9fnx                        1/1     Running             0             100s
csi-attacher-5c4bfdcf59-m77g4                       0/1     Error               0             101s
csi-attacher-5c4bfdcf59-m77g4                       1/1     Running             1 (2s ago)    102s
csi-snapshotter-959b69d4b-crj5g                     1/1     Running             0             107s
csi-snapshotter-959b69d4b-qzp8c                     1/1     Running             0             110s
csi-provisioner-667796df57-8bgrd                    0/1     Error               0             112s
csi-provisioner-667796df57-8bgrd                    1/1     Running             1 (2s ago)    113s
csi-provisioner-667796df57-6xl8d                    0/1     Error               0             113s
csi-attacher-5c4bfdcf59-vhrhm                       0/1     Error               0             114s
csi-provisioner-667796df57-6xl8d                    1/1     Running             1 (1s ago)    114s
csi-attacher-5c4bfdcf59-vhrhm                       1/1     Running             1 (2s ago)    115s
csi-resizer-694f8f5f64-xt955                        0/1     Error               0             116s
csi-resizer-694f8f5f64-xt955                        1/1     Running             1 (1s ago)    117s
csi-resizer-694f8f5f64-q2nnj                        0/1     Error               1 (32s ago)   2m2s
csi-attacher-5c4bfdcf59-7fzkf                       0/1     Error               0             2m2s
csi-attacher-5c4bfdcf59-7fzkf                       1/1     Running             1 (2s ago)    2m3s
csi-snapshotter-959b69d4b-btkb9                     1/1     Running             0             2m5s
csi-provisioner-667796df57-mlfd2                    0/1     Error               0             2m7s
csi-provisioner-667796df57-mlfd2                    1/1     Running             1 (1s ago)    2m8s
instance-manager-843e3dde57d65c0384c81f20f0480779   1/1     Running             0             113s
csi-resizer-694f8f5f64-d9fnx                        0/1     Error               0             2m10s
csi-resizer-694f8f5f64-d9fnx                        1/1     Running             1 (1s ago)    2m11s
csi-attacher-5c4bfdcf59-m77g4                       0/1     Error               1 (32s ago)   2m12s
longhorn-csi-plugin-9l48p                           2/3     Running             0             2m13s
longhorn-csi-plugin-9l48p                           2/3     Running             0             2m16s
csi-resizer-694f8f5f64-q2nnj                        0/1     CrashLoopBackOff    1 (15s ago)   2m16s
longhorn-csi-plugin-9l48p                           3/3     Running             0             2m16s
csi-snapshotter-959b69d4b-crj5g                     0/1     Error               0             2m17s
csi-resizer-694f8f5f64-q2nnj                        1/1     Running             2 (16s ago)   2m17s
longhorn-csi-plugin-2bhzb                           1/3     Error               0             2m17s
longhorn-csi-plugin-2bhzb                           2/3     Running             1 (14s ago)   2m18s
csi-snapshotter-959b69d4b-crj5g                     1/1     Running             1 (1s ago)    2m18s
longhorn-csi-plugin-2bhzb                           2/3     Running             1 (17s ago)   2m21s
longhorn-csi-plugin-2bhzb                           3/3     Running             1 (17s ago)   2m21s
csi-attacher-5c4bfdcf59-m77g4                       0/1     CrashLoopBackOff    1 (13s ago)   2m24s
csi-attacher-5c4bfdcf59-m77g4                       1/1     Running             2 (14s ago)   2m25s

@FrankYang0529
feat: generate coverage profiles for e2e
2de8af6 
Signed-off-by: PoAn Yang <poan.yang@suse.com>
@github-actions github-actions bot added the stale label Feb 10, 2024
@github-actions github-actions bot closed this Feb 21, 2024
@FrankYang0529 FrankYang0529 reopened this Mar 1, 2024
@longhorn longhorn deleted a comment from github-actions bot Mar 1, 2024
@longhorn longhorn deleted a comment from github-actions bot Mar 1, 2024
@innobead innobead removed the stale label Mar 1, 2024
@yangchiu
Copy link
Member

yangchiu commented Mar 1, 2024

Not merging this PR to prevent unexpected changes from corrupting daily regression process. Hope we can merge this after v1.6.1 release.

@FrankYang0529
Copy link
Contributor Author

Not merging this PR to prevent unexpected changes from corrupting daily regression process. Hope we can merge this after v1.6.1 release.

Hi @yangchiu, v1.6.1 is released. Could we merge this PR?

@yangchiu yangchiu self-requested a review April 8, 2024 01:36
@yangchiu yangchiu merged commit 3f74d3f into longhorn:master Apr 8, 2024
4 of 5 checks passed
@FrankYang0529 FrankYang0529 deleted the LH-7166 branch April 8, 2024 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yangchiu yangchiu yangchiu approved these changes

@innobead innobead Awaiting requested review from innobead

Assignees

@FrankYang0529 FrankYang0529

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@FrankYang0529 @yangchiu @innobead