[RFC] Run bsim tests with twister #85610
Open
+1,768
−3,245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zephyrbot
added
platform: nRF BSIM
zephyrbot
requested review from
aescolar,
alwa-nordic,
cvinayak,
gchwier,
golowanow,
hakehuang,
hermabe,
jhedberg,
LukaszMrugala,
nashif,
PerMac,
rugeGerritsen,
sjanc,
Thalley and
theob-pro
|
I'm quite surprised by this PR. Not just because it appeared quite out of the blue for me, but also because there has been quite related work by others (CC @gchwier ) in these area this overlaps with:
We have had this want for very long, but so far we have not succeeded. So instead of doing the same we have done in previous PRs where we discussed some details and ended up spending time polishing something which was rejected on principal issues, I would suggest we have a talk about it between the more involved and relevant people, try to reach an agreement on the principal, and if we do, then look at the particular details. |
ardo-nordic
force-pushed
the
run_bsim_with_twister
branch
2 times, most recently
from
87f98dd to
8f1a143
Compare
|
I just added the 4.2 milestone, as freeze for 4.1 is today, and this is neither ready not urgent. So to make it clear this is not trying to get into this release. |
|
@ardo-nordic next time consider opening such PRs as draft until they pass CI to avoid unnecessary noise for reviewers :) |
ardo-nordic
force-pushed
the
run_bsim_with_twister
branch
2 times, most recently
from
f747a95 to
dcd499b
Compare
ardo-nordic
force-pushed
the
run_bsim_with_twister
branch
from
dcd499b to
7c288b6
Compare
Bsim harness now can also be used to run tests. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
BSIM tests might be reusing the same binary files, so it makes sense to be able to split tests into build-only, which will only build and copy executable files, and run only, which will run them whenever needed. This requires more sequential pipeline to avoid race conitions. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Adding functionality to split harness_config into common and test-case specific part, which are joined together for any given test. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Adding Bsim phy-specific options to harness config. Clean up leftover log files. Adding option to expand extra args with individual values for different simulated devices. Adding random seed value to every sim device. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Changing bsim_exe_name values to a list due to respective schema changes. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Adding necessary changes to testcase.yaml files in all Bsim host tests to make them directly runnable by twister's bsim harness. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Shell scripts are no longer needed since twister can be used to build and run host bsim tests. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Increasing test timeouts for better margins. Removing missed obsolete shell running files. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Chaning timeouts in testcase.yaml files to be the same as they were in the shell running scripts. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
Fixing minor compliance errors and issues popping up after rebase. Fixing unit tests. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
ardo-nordic
force-pushed
the
run_bsim_with_twister
branch
3 times, most recently
from
ed01c65 to
2b022cb
Compare
With added support of running bsim tests with Twister, separate building command is no longer need as tests can be build and run with single command. Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
ardo-nordic
force-pushed
the
run_bsim_with_twister
branch
from
2b022cb to
b1be69e
Compare
ardo-nordic
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bsim harness in twister has been modified to be able to run tests. The main changes it entailed are as following:
Atm, only host is extensively using bsim harness, and thus is the only one affected.