Skip to content

[ntuple] Initial minimal support for SoA I/O#21647

Open
jblomer wants to merge 14 commits intoroot-project:masterfrom
jblomer:ntuple-soa-v2
Open

[ntuple] Initial minimal support for SoA I/O#21647
jblomer wants to merge 14 commits intoroot-project:masterfrom
jblomer:ntuple-soa-v2

Conversation

@jblomer
Copy link
Contributor

@jblomer jblomer commented Mar 20, 2026

First step towards SoA I/O in RNTuple. This PR lays the groundwork:

  • Dictionary annotation of classes that represent an on-disk collection of an underlying record type T in SoA layout
  • On-disk field marker for collections that were written in SoA layout
  • A first implementation of a SoA collection field. Currently, only writing is supported. To read, users need to impose a model or use views. Read support for the SoA field will follow.

Use of the SoA field is experimental at this stage and RNTuple will warn.

Part of #19230

@jblomer jblomer self-assigned this Mar 20, 2026
@jblomer jblomer requested a review from bellenot as a code owner March 20, 2026 15:00
@jblomer jblomer requested review from dpiparo and pcanal as code owners March 20, 2026 15:00
@github-actions
Copy link

github-actions bot commented Mar 20, 2026

Test Results

    22 files      22 suites   3d 11h 59m 12s ⏱️
 3 836 tests  3 834 ✅ 1 💤 1 ❌
75 707 runs  75 697 ✅ 9 💤 1 ❌

For more details on these failures, see this check.

Results for commit 5985bf0.

♻️ This comment has been updated with latest results.

Copy link
Member

@hahnjo hahnjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I made a first pass over the PR and left some comments. Two "random" thoughts:

  1. Because RSoAField doesn't support reading at the moment, the user is required to impose a model (if a top-level field), or even required to use views (if inside a member). This will be lifted eventually.
  2. It is probably possible (I didn't see a check against it) to have a class inherit from a base class that is annotated with rntupleSoARecord. At first I thought this might cause problems, but maybe it's fine? Do we want to allow this?

jblomer added 9 commits March 24, 2026 13:35
Add a new dict option to the LinkDef class pragma parser and the
genreflex XML parser. When this option is set, the dictionary of the
class has the rntuple.SoARecord attribute set to the type name of
the record that underlies the SoA collection.

This option and enforcing RNTuple streamer mode are mutually exclusive.

fixup
Add EnsureValidUserClass() internal utility function that checks if a
TClass represents a valid RNTuple user class. These checks are used by
the RClassField and will be used later by the RSoAField.
Add SoA flag to field descriptor flags as well as a description of the
treatment of SoA collections.
jblomer added 5 commits March 24, 2026 14:01
First implementation of a SoA field. This implementation supports only
writing, no reading. Written data can be read though as collection of the
underlying record type.

Other limitations that will be lifted in follow-up commits:
  - support for inheritance
  - support for automatic schema evolution
  - support for class rename rules
  - support for SoA layout of inner structs
@jblomer
Copy link
Contributor Author

jblomer commented Mar 24, 2026

Good point on the inheritance. I didn't think about it but I don't think we need to forbid it, given that we already have classes that inherit from std::vector in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants