RHCEPHQE-17401: Refactor with context in parallel module.

[psathyan]

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Unit Testing
Logs

(venv) psathyan@magna002:~/src/cephci$ cat utest_parallel.py
from ceph.parallel import parallel

def trigger(param):
    print(f"I got this varlue {param}")
    return param


with parallel() as p:
    for x in range(5):
        p.spawn(trigger, x)

    for r in p:
        print(f"The retrived result is {r}")

print("The number threads spawned is", p.count)
print("-" * 20)
for r in p:
    print(f"Outside results retrival value is {r}")

(venv) psathyan@magna002:~/src/cephci$ python utest_parallel.py
I got this varlue 0
I got this varlue 1
I got this varlue 2
I got this varlue 3
The retrived result is 0
I got this varlue 4
The retrived result is 1
The retrived result is 2
The retrived result is 3
The retrived result is 4
The number threads spawned is 5
--------------------
Outside results retrival value is 0
Outside results retrival value is 1
Outside results retrival value is 2
Outside results retrival value is 3
Outside results retrival value is 4

[openshift-ci-robot]

@psathyan: This pull request references RHCEPHQE-17401 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@psathyan: This pull request references RHCEPHQE-17401 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Unit Testing
Logs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@psathyan: This pull request references RHCEPHQE-17401 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Unit Testing
Logs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@psathyan: This pull request references RHCEPHQE-17401 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Unit Testing
Logs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@psathyan: This pull request references RHCEPHQE-17401 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

The initial design populated the results only when it had completed successfully. This could be a impact when qe is refering the values during the context.

In this PR, the futures.as_completed is not used instead the timeout is controlled via datetime methods. Also, results are continued to be appended at the end of the method.

The iteration is modified to use _futures instead of _results. This way we can capture details on the cancelled threads also.

• Review the automation design
• Implement the test script and perform test runs

Unit Testing
Logs

(venv) psathyan@magna002:~/src/cephci$ cat utest_parallel.py
from ceph.parallel import parallel

def trigger(param):
   print(f"I got this varlue {param}")
   return param


with parallel() as p:
   for x in range(5):
       p.spawn(trigger, x)

   for r in p:
       print(f"The retrived result is {r}")

print("The number threads spawned is", p.count)
print("-" * 20)
for r in p:
   print(f"Outside results retrival value is {r}")

(venv) psathyan@magna002:~/src/cephci$ python utest_parallel.py
I got this varlue 0
I got this varlue 1
I got this varlue 2
I got this varlue 3
The retrived result is 0
I got this varlue 4
The retrived result is 1
The retrived result is 2
The retrived result is 3
The retrived result is 4
The number threads spawned is 5
--------------------
Outside results retrival value is 0
Outside results retrival value is 1
Outside results retrival value is 2
Outside results retrival value is 3
Outside results retrival value is 4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[AmarnatReddy]

I have tested these changes on the failing suite, and they are working as expected.
Logs : http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-8FMKOI/
/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AmarnatReddy, psathyan, vamahaja

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mergify]

This pull request has been removed from the queue for the following reason: pull request dequeued.

Pull request #4357 has been dequeued. The pull request rule doesn't match anymore. The following conditions don't match anymore:

• branch-protection-review-decision=approved
• any of: [🔀 queue conditions]
  • all of: [📌 queue conditions of queue default]
    • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.

If you want to requeue this pull request, you need to post a comment with the text: @mergifyio requeue

[openshift-ci]

New changes are detected. LGTM label has been removed.