Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(RHEL-27512) test-execute debug #245

Closed
wants to merge 36 commits into from

Conversation

mrc0mmand
Copy link
Member

@mrc0mmand mrc0mmand commented Mar 6, 2024

No description provided.

lnykryn and others added 30 commits February 29, 2024 16:14
RHEL-only

Resolves: RHEL-27512
When running tests in a container, /sys might not be mounted, so
let's make sure we skip tests that depend on /sys in this case.

(cherry picked from commit a412a1b)

Related: RHEL-27512
(cherry picked from commit 0ce8870)

Related: RHEL-27512
- use SD_ID128_STRING_MAX or friend,
- use sizeof(sd_id128_t),
- use newly introduced ascii_ishex().

(cherry picked from commit 28bf2de)

Related: RHEL-27512
…ontents is "uninitialized"

Then, this drops ID128_PLAIN_OR_UNINIT. Also, this renames
Id128Format -> Id128FormatFlag, and make it bitfield.

Fixes #25634.

(cherry picked from commit 057bf78)

Related: RHEL-27512
(cherry picked from commit a237c6e)

Related: RHEL-27512
(cherry picked from commit 9be90c4)

Related: RHEL-27512
…oc/ is not mounted

(cherry picked from commit e272034)

Related: RHEL-27512
And drop to mention sd_id128_get_boot_app_specific() may return -ENOENT
or -ENOMEDIUM. The function does not read /etc/machine-id. But reads a
file in the procfs, which is a kind of the kernel API. Hence the
failures are caused only when the system has wrong setup.

(cherry picked from commit c576920)

Related: RHEL-27512
(cherry picked from commit b40c8eb)

Related: RHEL-27512
… an ID is in an invalid format

EINVAL suggests that the caller passes an invalid argument. EIO is
for "input/output error", i.e. the error you'd get if the disk or
file system is borked, and this error code could be returned by the
underlying read/write functions.

Let's make the functions return an unambiguous error code.

(cherry picked from commit e8a6625)

Related: RHEL-27512
It may be useful to check if the machine ID or friends is set or not.

(cherry picked from commit 786b652)

Related: RHEL-27512
(cherry picked from commit 75fa1f2)

Related: RHEL-27512
(cherry picked from commit f0d8358)

Related: RHEL-27512
This also ensures that the test is skipped when /etc/machine-id exists,
but is not initialized.

(cherry picked from commit 415eb50)

Related: RHEL-27512
…nitialized

The part of test_chase_symlink in test-fs-util that calls
sd_id128_get_machine will fail if /etc/machine-id is empty, so skip this
block if the machine-id is not initialized.

(cherry picked from commit 079fcdd)

Related: RHEL-27512
(cherry picked from commit a635b62)

Related: RHEL-27512
(cherry picked from commit 2c6b738)

Related: RHEL-27512
When executed on a systemd with an empty /etc/machine-id,
test-journal-interleaving fails in test_sequence_numbers_one() when
re-opening the existing "two.journal". This is because opening the
existing journal file with managed_journal_file_open() causes
journal_file_verify_header() to be called. This function tries to
compare the current machine-id to the machine-id in the journal file
header, but does not handle the case where the machine-id is empty or
non-existent.

Check if we have an initialized machine-id before executing this portion
of the test.

(cherry picked from commit 3a9ca23)

Related: RHEL-27512
The journal stuff in RHEL 9 can't handle existing but uninitialized
/etc/machine-id and backporting all the necessary changes would mean
pulling in another 50+ commits (I stopped after 50, so it's probably
much more). And to make matters worse, upstream renamed
chase_symlinks*() stuff to chase_*(), which makes the backports even
more painful (and risky).

Related PRs:
  - https://github.com/systemd/systemd/pull/25734/commits
  - https://github.com/systemd/systemd/pull/27137/commits
  - https://github.com/systemd/systemd/pull/27122/commits

RHEL-only

Related: RHEL-27512
We have a test where we compare the results from nftw() and our own
resurce_dit_at(). nftw() skips a dangling symlink when running under mkosi and
the test fails. I don't understand why nftw() does that, but in our code we
don't need to test and care about the details of nftw(), which we don't use,
outside of the one test, so let's just skip symlinks in the test.

Closes #29603.

(cherry picked from commit 974959e)

Related: RHEL-27512
(cherry picked from commit 31cfcf5)

Related: RHEL-27512
If /etc/machine-id contains "uninitialized", specifier_printf() with
%m will fail with ENOPKG, so ignore that error as well.

(cherry picked from commit 7429c8f)

Related: RHEL-27512
The test depends on /sys being writable, so let's skip it when /sys
is read-only.

(cherry picked from commit 34b5977)

Related: RHEL-27512
Upstream mkosi sets $KERNEL_INSTALL_BYPASS to 1 by default [0] which
then trip over check-help tests, as the message about
$KERNEL_INSTALL_BYPASS is not printed out to stderr. Upstream systemd
doesn't have this issue, since kernel-install there was rewritten in C,
so the code base is completely different.

 1068/1073 systemd:dist-check / check-help-kernel-install                       FAIL             0.02s   exit status 4
>>> MALLOC_PERTURB_=212 /work/src/tools/check-help.sh /work/build/kernel-install
――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
kernel-install with an unknown parameter does not print to stderr
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

[0] systemd/mkosi@deaaa83

RHEL-only

Related: RHEL-27512
Before this, tests are split into two categories, system and user, but
both are running in fully privileged environment. Hence, unprivileged
user scope was mostly not covered by the test.

Let's run all tests in both system and user scopes, and drop capabilities
when Manager is running in user scope.

This also makes the host environment protected more from the test run.

(cherry picked from commit 4e032f6)

Related: RHEL-27512
If '+' is specified with 'C', let's merge the tree with any existing
tree.

(cherry picked from commit 1fd5ec5)

Related: RHEL-27512
…ary units

This function is like `generator_open_unit_file`, but if `ret_temp_path` is
passed, a temporary unit is created instead.

(cherry picked from commit 8a84e0d)

Related: RHEL-27512
aafeijoo-suse and others added 5 commits February 29, 2024 16:14
…changed

When the `systemd-network-generator` is included in the initrd and runs from
there first, the next times it runs after switching to real root it
thinks there is a duplicate entry on the kernel command line.

This patch rewrites the unit file if the content has changed, instead of
displaying an error message.

(cherry picked from commit f3e4d04)

Related: RHEL-27512
It's been a while since we introduced Differential ShellCheck and it
proved to be quite useful (and in some ways even better than the shellcheck
run by super-linter). So, to have only one linter scream at us for not
knowing how to write bash properly, let's drop the super-linter's one in
favor of Differential ShellCheck.

Follow-up for systemd/systemd#24328 (review)

(cherry picked from commit c4b167f)

Related: RHEL-27512
- drop Hyperscale SIG repos, so we don't pull in btrfs stuff
- use XFS for rootfs
- install gnu-efi headers, since RHEL 9 sd-boot still requires it

RHEL-only

Related: RHEL-27512
Notes:
  - on RHEL 9 we don't have vsock support in systemd, so we have to fall
    back to some "older" checks (i.e. pre-6aca147f82).
  - our systemd-repart doesn't automatically fall back to Format=swap
    for Type=swap partitions, so we have to do that explicitely
  - don't pull in sd-resolved and sd-networkd, since the former needs
    some extra configuration to work alongsideNM, and we don't
    build the latter at all on RHEL 9

RHEL-only

Related: RHEL-27512
The automagic enrollment works only with systemd v253 onwards, on v252
we need to do this one extra step.

RHEL-only

Related: RHEL-27512
@github-actions github-actions bot changed the title test-execute debug (RHEL-27512) test-execute debug Mar 6, 2024
@github-actions github-actions bot added pr/needs-ci Formerly needs-ci pr/needs-review Formerly needs-review labels Mar 6, 2024
Copy link

github-actions bot commented Mar 6, 2024

Commit validation

Tracker - Missing issue tracker ✋

The following commits meet all requirements

commit upstream
58ccb7d - backport new mkosi RHEL-only
2ac3874 - test: Skip various tests when /sys is not mounted systemd/systemd@a412a1b
dc76198 - string-util: introduce ascii_ishex() systemd/systemd@0ce8870
0de5a59 - sd-id128: several cleanups systemd/systemd@28bf2de
0b2fe13 - sd-id128: make id128_read() or friends return -ENOPKG when the file co… systemd/systemd@057bf78
b64576c - test: add tests for "uninitialized" string handling by id128_read_fd()… systemd/systemd@66c7949
8f2f475 - man: mention sd_id128_get_machine() or friend may return -ENOPKG systemd/systemd@a237c6e
1c58069 - sd-id128: make sd_id128_get_boot() and friend return -ENOMEDIUM systemd/systemd@9be90c4
fa1bd8f - sd-id128: make sd_id128_get_boot() and friend return -ENOSYS when /pro… systemd/systemd@e272034
c325843 - man: mention that sd_id128_get_boot() and friend may return -ENOSYS systemd/systemd@c576920
9a03fb5 - sd-id128: fold do_sync flag into Id128FormatFlag systemd/systemd@b40c8eb
5dd96bb - sd-id128: make sd_id128_get_machine() or friends return -EUCLEAN when … systemd/systemd@e8a6625
2a4a4d0 - sd-id128: allow sd_id128_get_machine() and friend to be called with NU… systemd/systemd@786b652
72e8b9e - sd-id128: also refuse an empty invocation ID systemd/systemd@75fa1f2
f8a00c6 - man: update documents for sd_id128_get_invocation() systemd/systemd@f0d8358
ed29e57 - test-id128: simplify machine-id check systemd/systemd@415eb50
5f9f871 - test-fs-util: skip part of test_chase_symlinks if machine-id is not in… systemd/systemd@079fcdd
60d2cff - test-unit-name: simplify machine-id check systemd/systemd@a635b62
8f0be53 - test-load-fragment: simplify machine-id check systemd/systemd@2c6b738
60dcdd1 - journal: skip part of test-journal-interleaving if no machine-id exist… systemd/systemd@3a9ca23
db6e53a - test: skip journal tests without valid /etc/machine-id RHEL-only
a8ffed7 - test-recurse-dir: work around nftw() ignoring symlinks() systemd/systemd@974959e
dc0d9f6 - test: Skip test-recurse-dir on overlayfs systemd/systemd@31cfcf5
349c36f - test-specifier: Ignore -ENOPKG from specifier_printf() systemd/systemd@7429c8f
6336105 - test-execute: Skip when /sys is read-only systemd/systemd@34b5977
c68b6f8 - kernel-install: Make sure KERNEL_INSTALL_BYPASS is disabled in tests systemd/systemd@4435da1
f4a2c83 - tools: make sure $KERNEL_INSTALL_BYPASS is disabled when checking help… RHEL-only
f211057 - test-execute: drop capabilities when testing with user manager systemd/systemd@4e032f6
196ffa6 - tmpfiles: Add merge support for copy files action systemd/systemd@1fd5ec5
34b4993 - generator: add generator_open_unit_file_full to allow creating tempora… systemd/systemd@8a84e0d
5bc8393 - network-generator: rewrite unit if it already exists and its content c… systemd/systemd@f3e4d04
01072de - ci: drop super-linter's shellcheck systemd/systemd@c4b167f
553d5a1 - mkosi: make sure we build & use RHEL 9 stuff RHEL-only
4b7e039 - ci: backport mkosi CI configuration from upstream RHEL-only
df9719b - mkosi: explicitly enroll SecureBoot keys RHEL-only

The following commits need an inspection

commit note
97b85ea - debug-only Missing issue tracker
Missing upstream reference ‼️

Follow-up detection

Failed

🔴 Some follow-up commits for this Pull Request were detected in upstream

Follow-ups

commit follow-up
f211057 - test-execute: drop capabilities when testing with user manager systemd/systemd@d51e31a

Tracker validation

🔴 Missing tracker or Unknown tracker type; type: 'unknown'


Pull Request validation

Failed

🔴 Failed or pending checks - build (GCC, openssl)[failure],build (GCC, auto)[failure],build (CLANG_RELEASE, auto)[failure],build (CLANG, gcrypt)[failure],build (CLANG, auto)[failure],build (gcc, 12, gold, openssl)[in_progress],build (gcc, 11, bfd, gcrypt)[in_progress],PR (undefined)[in_progress],build (clang, 15, bfd, auto)[in_progress],PR (memory)[in_progress],build (clang, 14, lld, openssl)[in_progress],PR (address)[in_progress],build (clang, 13, mold, gcrypt)[in_progress] Failed or pending statuses - CentOS CI (CentOS Stream 9 + sanitizers)[pending],CentOS CI (CentOS Stream 9)[pending]
🔴 Review - Missing review from a member.

@github-actions github-actions bot added the tracker/missing Formerly needs-bz label Mar 6, 2024
@mrc0mmand mrc0mmand closed this Mar 6, 2024
@mrc0mmand mrc0mmand deleted the new_mkosi branch March 6, 2024 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/follow-up pr/needs-ci Formerly needs-ci pr/needs-review Formerly needs-review tracker/missing Formerly needs-bz
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants