Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Override test metada from plan using adjust-tests #2865

Merged
merged 12 commits into from
Oct 18, 2024
Merged

Override test metada from plan using adjust-tests #2865

merged 12 commits into from
Oct 18, 2024

Conversation

lukaszachy
Copy link
Collaborator

@lukaszachy lukaszachy commented Apr 16, 2024 •
edited
Loading

New option for discover -h fmf to adjust discovered tests

I went with 'adjust-tests' keyword proposed in #2430 (reply in thread)

Pull Request Checklist

  • implement the feature
  • write the documentation
  • extend the test coverage
  • update the specification
  • adjust plugin docstring
  • modify the json schema
  • include a release note

I'm not able to set the versionadded into the discover/fmf.py so I'll omit that.

@lukaszachy lukaszachy linked an issue Apr 16, 2024 that may be closed by this pull request
Override test duration from plan #1843
Closed
@lukaszachy lukaszachy added the status | blocked The merging of PR is blocked on some other issue label Apr 16, 2024
@lukaszachy
Copy link
Collaborator Author

Note that you need teemtee/fmf#230 to try this PR

@lukaszachy lukaszachy mentioned this pull request Apr 16, 2024
Closed
happz
happz reviewed Apr 16, 2024
View reviewed changes
happz
happz reviewed Apr 16, 2024
View reviewed changes
@lukaszachy lukaszachy added this to the 1.34 milestone Apr 30, 2024
@lukaszachy lukaszachy modified the milestones: 1.34, 1.35 Jun 5, 2024
@lukaszachy
Copy link
Collaborator Author

fmf released today, but it will take some time it reaches stable repos, thus aiming for next tmt version with this.

@lukaszachy lukaszachy removed the status | blocked The merging of PR is blocked on some other issue label Jun 5, 2024
@LecrisUT LecrisUT mentioned this pull request Jun 21, 2024
Open
@martinhoyer martinhoyer modified the milestones: 1.35, 1.36 Jul 30, 2024
@lukaszachy lukaszachy modified the milestones: 1.36, 1.37 Aug 27, 2024
@lukaszachy lukaszachy added the priority | must high priority, must be included in the next release label Aug 27, 2024
@lukaszachy
Copy link
Collaborator Author

Adding the 'must' label as I really need this to get rid of "virtual" test for 'mutually exclusive packages' as foo and foo-ng where plan level change of 'require/recommend' will be really good to have.

@lukaszachy lukaszachy force-pushed the edit-discovered-tests branch from 8c719b9 to d95f2f9 Compare August 27, 2024 16:49
@psss psss modified the milestones: 1.37, 1.38 Sep 10, 2024
@thrix thrix removed the priority | must high priority, must be included in the next release label Oct 1, 2024
@lukaszachy
Copy link
Collaborator Author

lukaszachy commented Oct 2, 2024 •
edited
Loading

Once corner case to solve: Test without explicit duration (so the default 5m is used), but 'adjust-test' specifies 'duration+: *2'.

One would expect this leads to the 10m, but how to do that?
Problem is that input data have 'duration: *2' so the default is not used.
And tmt.utils.duration_to_seconds('*2') == 0

Should duration_to_seconds be aware of the default value?
tmt comes to play after fmf data are processed so I don't see good place where to put the tmt's default value :/

I modified the 'duration_to_seconds' but it doesn't feel right

@lukaszachy lukaszachy force-pushed the edit-discovered-tests branch from d95f2f9 to 91d5900 Compare October 2, 2024 20:54
@lukaszachy
Copy link
Collaborator Author

To speed this up I'll ignore for now the CLI way to specify this parameters, no idea how could one add dictionary from CLI in a nice wau.

@lukaszachy lukaszachy marked this pull request as ready for review October 3, 2024 08:36
@happz
Copy link
Collaborator

happz commented Oct 17, 2024

testing-farm:fedora-40-x86_64:provision in /tests/prepare/recommend fails during provision with " fail: no such table: image" (https://artifacts.osci.redhat.com/testing-farm/66c67184-2ae8-47cf-a607-b7485e05c659/)

I'll restart but any idea what was it about?

testcloud switched to using sqlite3 internally, could be related. @frantisekz any chance it rings a bell?

@psss psss force-pushed the edit-discovered-tests branch from d984bce to 9030398 Compare October 17, 2024 15:59
@psss
Copy link
Collaborator

psss commented Oct 17, 2024 •
edited
Loading

Proposing a couple of docs improvements in 9030398.

psss
psss requested changes Oct 17, 2024
View reviewed changes
Copy link
Collaborator

@psss psss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice, thanks! Just a few comments.

@psss psss changed the title Override test attribute from plan Override test metada from plan using adjust-tests Oct 17, 2024
@psss psss added the plugin | fmf The fmf discover plugin label Oct 17, 2024
@lukaszachy lukaszachy force-pushed the edit-discovered-tests branch from 9030398 to edcaa2f Compare October 17, 2024 17:03
@lukaszachy
Copy link
Collaborator Author

Feedback addressed. Rebased to solve the confict in docs/releases ... Is there anything we can to about that file? it's a little pain :/

@lukaszachy lukaszachy requested review from psss and happz October 17, 2024 17:04
@psss psss force-pushed the edit-discovered-tests branch from edcaa2f to 7bb31ac Compare October 17, 2024 22:06
@psss
Copy link
Collaborator

psss commented Oct 17, 2024

I just removed the \b chars in 7bb31ac as they break the online docs and @happz already fixed the issue in #3301.

psss
psss approved these changes Oct 17, 2024
View reviewed changes
Copy link
Collaborator

@psss psss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now looks good, thanks!

@psss psss merged commit 2f2c4b4 into main Oct 18, 2024
21 of 22 checks passed
@psss psss deleted the edit-discovered-tests branch October 18, 2024 05:48
@psss psss self-assigned this Oct 18, 2024
@psss psss mentioned this pull request Oct 23, 2024
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KwisatzHaderach KwisatzHaderach KwisatzHaderach approved these changes

@psss psss psss approved these changes

@martinhoyer martinhoyer martinhoyer approved these changes

@thrix thrix Awaiting requested review from thrix thrix is a code owner

@janhavlin janhavlin Awaiting requested review from janhavlin janhavlin is a code owner

@happz happz Awaiting requested review from happz happz is a code owner

Assignees

@psss psss

Labels
ci | full test Pull request is ready for the full test execution plugin | fmf The fmf discover plugin priority | must high priority, must be included in the next release step | discover Stuff related to the discover step
Projects
None yet
Milestone
1.38
Development

Successfully merging this pull request may close these issues.

Override test duration from plan
6 participants
@lukaszachy @KwisatzHaderach @happz @psss @martinhoyer @thrix