Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SE050 test #423

Merged
merged 14 commits into from
Oct 27, 2023
Merged

Add SE050 test #423

merged 14 commits into from
Oct 27, 2023

Conversation

sosthene-nitrokey
Copy link
Contributor

This PR adds a test that is introduced in the admin app with Nitrokey/admin-app#5

Note that it takes around 30seconds to complete, it's not crashing.

Copy link
Member

@szszszsz szszszsz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that this is generally a caller for the in-firmware test, and interpreter of the results gathered there.

  1. Why the control of the tests is placed in the firmware instead of the test calling code?
    Was it easier that way? Or is this a quick POC?
  2. I think pytest would be better to use here, but its done already.

@sosthene-nitrokey
Copy link
Contributor Author

Why the control of the tests is placed in the firmware instead of the test calling code?

What do you mean by "the control of the test"?

I think pytest would be better to use here, but its done already.

The goal here is to get info from beta testers that the SE050 driver works as expected with NK3 that are already deployed. If doing so through pytest we likely wouldn't get feedback since users are unlikely to use it.

@szszszsz
Copy link
Member

What do you mean by "the control of the test"?

You are essentially calling a fixed test routine in the firmware, and waiting for the results. Instead you could construct test cases on the calling side, and use a proper direct communication channel to se050.

If doing so through pytest we likely wouldn't get feedback since users are unlikely to use it.

That explains things. I do not think though running pytest itself is that much trouble for a beta tester, but that's my opinion.

@sosthene-nitrokey
Copy link
Contributor Author

You are essentially calling a fixed test routine in the firmware, and waiting for the results. Instead you could construct test cases on the calling side, and use a proper direct communication channel to se050.

I wanted to test the actual APDU generation from the driver to test it, so all commands to the SE050 needs to be run from rust code. This is meant to test the driver in real conditions. Communicating to the SE050 directly from nitropy would make that much less useful, even if it were to test the T=1 over I2C part of the driver.

I do not think though running pytest itself is that much trouble for a beta tester, but that's my opinion.

I agree, but running pytest is much less discoverable while nk3 test is documented and used.

pynitrokey/cli/nk3/test.py Outdated Show resolved Hide resolved
@daringer
Copy link
Collaborator

this does not interfere with anything else, should we merge this ?

@robin-nitrokey
Copy link
Member

Running this with the se050-test-app feature disabled now gives me:

[4/5]	se050    	SE050                   	FAILURE  	object of type 'type' has no len()

@sosthene-nitrokey
Copy link
Contributor Author

Running this with the se050-test-app feature disabled now gives me:

[4/5]	se050    	SE050                   	FAILURE  	object of type 'type' has no len()

Can you try again? This should be fixed I believe.

@robin-nitrokey
Copy link
Member

Yes, fixed.

try:
q.put(AdminApp(device).se050_tests())
except CtapError as e:
if e.code == CtapError.ERR.INVALID_LENGTH:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also check for INVALID_COMMAND instead of whitelisting firmware versions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. We can also update admin-app to return it when the command is disabled instead of INVALID_LENGTH.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning UnsupportedCommand seems to lead to pynitrokey not receiving the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea was to make it easier to handle the case that the firmware does not (yet) support a command. _call always returns bytes, so None always indicates that the command is not supported. And checking for None is easier than catching the exception and checking the error code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so I can just skip if None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relevant PR in the admin app: Nitrokey/admin-app#12

@robin-nitrokey robin-nitrokey merged commit 876fda9 into master Oct 27, 2023
10 checks passed
@robin-nitrokey robin-nitrokey deleted the se050-test branch October 27, 2023 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants