Check deprecated iobundle type #4634

christoph-zededa · 2025-02-20T13:06:59Z

pillar/types: check for deprecated IoBundle Types

so far only for IoUSB is checked

IoUsb should not be used anymore in favor of IoUsbController and
IoUsbDevice

also make spdx checker ignore 'do-not-edit' files.

will be used in the following commits Signed-off-by: Christoph Ostarek <christoph@zededa.com>

rene · 2025-02-20T15:43:06Z

@christoph-zededa , although IoUSB is deprecated, is there any use case where IoUSB can still be used? I'm wondering if these changes can break systems using old device models + EVE with IoUsbController?

christoph-zededa · 2025-02-20T15:59:03Z

@christoph-zededa , although IoUSB is deprecated, is there any use case where IoUSB can still be used? I'm wondering if these changes can break systems using old device models + EVE with IoUsbController?

I added be643f3 to remove all remaining uses of types.IoUSB.
It should not break it, but starting the edge app will fail: https://github.com/lf-edge/eve/blob/master/pkg/pillar/cmd/domainmgr/domainmgr.go#L1481

OhmSpectator · 2025-02-21T09:05:23Z

tools/spdx-check.sh

@@ -111,6 +111,10 @@ file_to_be_checked() {
      return 1
    fi
  done
+  if grep -qi "do not edit" "$file";


Which files exactly are you referring to?

Even if a file is generated by a tool, I still think it should contain the header—it's recommended so we know the consequences of using that generating tool. While I agree that we can (and already do) ignore some generated files, we can still add headers to files that come from tools we maintain ourselves.

Could you clarify which specific files you have in mind here?

iotype_string.go

COuld we change it to "Code generated by.*stringer"

OhmSpectator · 2025-02-21T09:12:03Z

pkg/pillar/types/iotype_string.go

@@ -0,0 +1,55 @@
+// Code generated by "stringer --type IoType"; DO NOT EDIT.


Interesting, I didn't know about that tool—I've always handled it manually. While it looks useful, I'm wondering how maintainable it is. If we change an enum, will the tool be run automatically? If not, then we shouldn't really treat the output as entirely auto-generated, and we’d need to maintain it ourselves. If it is run automatically, then we can skip the header.

So, I see two options:

Automate the tool run so we don't need a header, or

Treat the file as manually maintained and add the header.

I'm wondering how maintainable it is

it throws a compile error if one makes incompatible changes

And after that, one has to go and fix it manually. So. it's not rerun automatically. So, it's manually maintained file =)

We also have lots of other auto-generated files checked in.

Like under vendors and API? Or something else?

Yes, eve-api is probably the biggest chunk (another case are all these draw.io diagrams).

Perhaps a small argument for checking in the generated code is that if you use an IDE outside of the build container then some source files are missing and your IDE may become unhappy; also people running go test on their system directly have to create these files first.

I didn't see this custom String() function earlier. Doesn't the golang built-in String() for an enum produce the correct output?

Unfortunately not, that's why they created https://pkg.go.dev/golang.org/x/tools/cmd/stringer

OhmSpectator · 2025-02-21T09:23:47Z

pkg/pillar/types/assignableadapters.go

+		ioBundle := &aa.IoBundleList[i]
+		ioBundle.Error.removeByType(ErrIOBundleDeprecatedType{})
+
+		if ioBundle.Type == IoUSB {


Where exactly does this type originate? Could it come from the controller or from persistent storage? If it can be stored and then a user updates EVE to a newer version, might they encounter this error? Or is it only ever set by our internal code, making it impossible for an older or external value to appear?

If it can come from an older source, is there any way we could safely handle unknown types — perhaps trying to guess the correct type and logging a warning—rather than breaking the system that was previously running without issues?

It is derived from an eve-api type: PhyIoUSB = 2;

Nobody is reading these warnings ;-) - see also #4634 (comment)

Afaik the commercial users got informed about the change.

Afaik the commercial users got informed about the change.

Unfortunately, it does not prevent them from making mistakes and then requesting fixes from the engineering... We know such cases =(

I must have missed something in the past reviews/discussions and I don't recall us communicating to users that all existing models which published in the controllers (including the eden model files which we can change) need to be changed from IoTypeUsb to IoTypeUsbController.

I thought we introduced IoTypeUsbController as a more clear name, but that existing uses of IoTypeUSb would be interpreted to mean the same thing as IoTypeUsbController. If that is not the case I think we will break existing usage.
Let's set up some time on Monday to talk about this. Perhaps I'm missing something.

christoph-zededa · 2025-02-21T10:48:46Z

Alternatives:

Introduce warnings so that https://github.com/lf-edge/eve/blob/master/pkg/pillar/cmd/domainmgr/domainmgr.go#L1481 will not return an error but nevertheless will set the error and report it to the controller
Try to guess the correct type, i.e. if PciLong is set and Usbproduct and Usbaddr are not set, then it is a IoUsbController, otherwise it is a IoUsbDevice. But I am not sure if that is 100% correct.

OhmSpectator · 2025-02-21T10:58:20Z

I would like @eriknordmark to share his experience in handling such situations. I don't think it's the first time we have to introduce a potentially breaking change.

OhmSpectator · 2025-02-21T10:59:13Z

@christoph-zededa, in general, how did you come to the idea of this change? Is it related to some issue?

christoph-zededa · 2025-02-21T11:13:15Z

@christoph-zededa, in general, how did you come to the idea of this change? Is it related to some issue?

It is because of two issues I saw this week:

Somebody had an edge application that expects IoUsb and another one that expects IoUsbDevice and they were wondering what they should do in the model manifest to be able to run both.
[12.0-stable] Revert "domainmgr: do pci-reserve on start of edge app" #4633 - and they were using IoUsb and were wondering why the behavior is different on EVE 12.x vs 13.x and I think with IoUsbController the behavior would not have been different - see https://github.com/lf-edge/eve/blob/master/pkg/pillar/cmd/domainmgr/domainmgr.go#L1399

these files have been auto-generated and IMHO do not pass the threshold of originality and so do not need a copyright moreover it is a hassle to manually add the copyright every time again Signed-off-by: Christoph Ostarek <christoph@zededa.com>

so far only for IoUSB is checked IoUsb should not be used anymore in favor of IoUsbController and IoUsbDevice Signed-off-by: Christoph Ostarek <christoph@zededa.com>

that is either IoUSBController or IoUSBDevice Signed-off-by: Christoph Ostarek <christoph@zededa.com>

to hopefully fix flaky test in the future Signed-off-by: Christoph Ostarek <christoph@zededa.com>

pillar: add String method for IoType

27560a8

will be used in the following commits Signed-off-by: Christoph Ostarek <christoph@zededa.com>

github-actions bot requested review from deitch, eriknordmark, jsfakian, milan-zededa, OhmSpectator and uncleDecart February 20, 2025 13:07

christoph-zededa force-pushed the check_deprecated_iobundle_type branch 2 times, most recently from 49e5d68 to 8cc307e Compare February 20, 2025 13:18

christoph-zededa marked this pull request as ready for review February 20, 2025 14:00

christoph-zededa requested review from rene, rouming and shjala as code owners February 20, 2025 15:53

github-actions bot requested a review from rucoder February 20, 2025 15:54

OhmSpectator reviewed Feb 21, 2025

View reviewed changes

christoph-zededa added 4 commits February 21, 2025 18:37

spdx-check: ignore 'do-not-edit' files

e34ec6f

these files have been auto-generated and IMHO do not pass the threshold of originality and so do not need a copyright moreover it is a hassle to manually add the copyright every time again Signed-off-by: Christoph Ostarek <christoph@zededa.com>

pillar/types: check for deprecated IoBundle Types

08284e8

so far only for IoUSB is checked IoUsb should not be used anymore in favor of IoUsbController and IoUsbDevice Signed-off-by: Christoph Ostarek <christoph@zededa.com>

pillar: replace IoUSB device type with correct one

366a814

that is either IoUSBController or IoUSBDevice Signed-off-by: Christoph Ostarek <christoph@zededa.com>

pillar/watcher: enhance test error output

4691a87

to hopefully fix flaky test in the future Signed-off-by: Christoph Ostarek <christoph@zededa.com>

christoph-zededa force-pushed the check_deprecated_iobundle_type branch from dedad09 to 4691a87 Compare February 21, 2025 17:37

github-actions bot requested a review from OhmSpectator February 21, 2025 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check deprecated iobundle type #4634

Check deprecated iobundle type #4634

christoph-zededa commented Feb 20, 2025 •

edited

Loading

rene commented Feb 20, 2025

christoph-zededa commented Feb 20, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

christoph-zededa Feb 21, 2025

eriknordmark Feb 21, 2025

christoph-zededa Feb 21, 2025

OhmSpectator Feb 21, 2025

christoph-zededa Feb 21, 2025

christoph-zededa Feb 21, 2025 •

edited

Loading

OhmSpectator Feb 21, 2025

eriknordmark Feb 21, 2025

christoph-zededa commented Feb 21, 2025

OhmSpectator commented Feb 21, 2025

OhmSpectator commented Feb 21, 2025

christoph-zededa commented Feb 21, 2025

		@@ -0,0 +1,55 @@
		// Code generated by "stringer --type IoType"; DO NOT EDIT.

Check deprecated iobundle type #4634

Are you sure you want to change the base?

Check deprecated iobundle type #4634

Conversation

christoph-zededa commented Feb 20, 2025 • edited Loading

rene commented Feb 20, 2025

christoph-zededa commented Feb 20, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christoph-zededa Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christoph-zededa commented Feb 21, 2025

OhmSpectator commented Feb 21, 2025

OhmSpectator commented Feb 21, 2025

christoph-zededa commented Feb 21, 2025

christoph-zededa commented Feb 20, 2025 •

edited

Loading

christoph-zededa Feb 21, 2025 •

edited

Loading