Version 1.9.0

These are differences from 1.8.0 to 1.9.0. Most only affect Awkward Array version 2, in the awkward._v2 submodule of 1.9.0. The exceptions are called out with "v1" in the subject line.

Features

• feat: C++ refactoring: ak.to_pandas by @ioanaif in #1369
• feat: C++ refactoring: ak.nan_to_num by @ioanaif in #1352
• feat: C++ refactoring: ak.run_lengths by @ioanaif in #1347
• feat: add is_tuple describe operation by @agoose77 in #1351
• feat: C++ refactoring: ak.unzip by @ioanaif in #1354
• feat: pure Cling demo and improvements to C++ JIT infrastructure by @jpivarski in #1359
• feat: C++ refactoring: ak.broadcast_arrays by @ioanaif in #1368
• feat: irst version of ak._v2.from_parquet by @jpivarski in #1338
• feat: C++ refactoring: ak.copy by @ioanaif in #1367
• feat: C++ refactoring: ak.unflatten by @ioanaif in #1360
• feat: add depth_limit to ak.broadcast_arrays by @agoose77 in #1373
• feat: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in #1376
• feat: This PR sets up the architecture to call CuPy Raw Kernels from Awkward. by @swishdiff in #1355
• feat: Reducers with axis=None and typetracers. by @jpivarski in #1380
• feat: This PR adds the generated kernels and simplifies the template specialization generation process. by @swishdiff in #1381
• feat: High-level ak._v2.Array clean-ups. by @jpivarski in #1392
• feat: This PR adds all the remaining kernels in the studies directory by @swishdiff in #1390
• feat: This PR adds JAX as a new nplike by @swishdiff in #1399
• feat: Passing behaviour in ak._v2 functions by @ioanaif in #1415
• feat: Enable broadcasting of string equality. by @jpivarski in #1427
• feat: Enabled string/categorical behavior by @ioanaif in #1421
• feat: Implements ak.nan_to_none and all of the ak.nan* functions to override NumPy's. by @jpivarski in #1428
• feat: Pretty-printing types by @jpivarski in #1430
• feat: Register both v1 and v2 Arrays in Numba entry_points. by @jpivarski in #1432
• feat: Add Array and Record.delitem. And fix show(type=True). by @jpivarski in #1433
• feat: Enable categorical behavior - testing by @ioanaif in #1434
• feat: Enable mixins behavior by @ioanaif in #1437
• feat: Implementing ak._v2.to_parquet. by @jpivarski in #1440
• feat: awkward to rdataframe by @ianna in #1374
• feat: Enable ak.firsts by @ioanaif in #1443
• feat: Enable ak.singletons by @ioanaif in #1444
• feat: Revamping the to_json/from_json interface. by @jpivarski in #1449
• feat: This PR attempts to add autodifferentiation support for Awkward Arrays using JAX pytrees. by @swishdiff in #1447
• feat: Adding a Forth Based Avro Reader by @aryan26roy in #1491
• feat: Adding repr overriden behavior by @ioanaif in #1487
• feat: rdataframe to awkward by @ianna in #1474
• feat: add missing _like methods to TypeTracer by @agoose77 in #1505
• feat: from rdataframe for awkward arrays by @ianna in #1508
• feat: Add typeparser to v2. by @jpivarski in #1513
• feat: add C++ headers-only distribution configuration by @ianna in #1523
• feat: Growable Buffer header by @ManasviGoyal in #1535
• feat: parquet redux by @martindurant in #1476
• feat: Templated LayoutBuilder by @ManasviGoyal in #1494
• feat: raise AttributeError for public Array attributes by @agoose77 in #1573
• feat: Public interface for layout.recursively_apply and broadcast_and_apply. by @jpivarski in #1610
• feat: prevent reducers like ak.sum on records (v2) by @ioanaif in #1607
• feat: Drop ak.behavior['.', 'Name'] = cls, which isn't working/isn't tested. by @jpivarski in #1651
• feat: retrieve multiple columns from RDataFrame in a single event loop by @ianna in #1625

Bug-fixes

• fix: Fixed ak.num with axis=0 in typetracer. by @jpivarski in #1329
• fix: ak.flatten and ak.ravel should test for nplike.ndarray, not np.ndarray. by @jpivarski in #1340
• fix: Straighten out error handling via a thread-local (but otherwise global) context by @jpivarski in #1327
• fix: Fix PR #788: avoid materializing VirtualArrays in ak.with_name. by @jpivarski in #1346
• fix: fix docstring line in is_tuple (v1) by @agoose77 in #1356
• fix: fixes #1363 by ensuring that arguments documented as 'iterable of X' aren't used in 'len(X)'. by @jpivarski in #1364
• fix: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in #1372
• fix: Fixes nonlocal reducers in which the first list is empty. by @jpivarski in #1378
• fix: Fix _prettyprint after 'for i in range' changed to 'for i, val in enumerate'. by @jpivarski in #1384
• fix: bump black to 22.3.0 due to click 8.1 release by @henryiii in #1385
• fix: No zero-length shortcuts for ak.argsort (v1 & v2). by @jpivarski in #1387
• fix: ErrorContexts should only contain strings. by @jpivarski in #1393
• fix: ROOT doesn't recognize for-each iterators without operator== by @jpivarski in #1398
• fix: Implement recursively_apply for Record by @agoose77 in #1401
• fix: from_numpy references ListArray64 by @agoose77 in #1404
• fix: Windows builds stopped working; be looser about directory name. by @jpivarski in #1407
• fix: pypy 3.9 by @henryiii in #1412
• fix: replace llvmlite.ir instead of llvmlite.llvmpy.core by @Ahmad-AlSubaie in #1413
• fix: Fix performance issue in v2 tolist. by @jpivarski in #1418
• fix: Fix iteration over NumpyArray type. by @jpivarski in #1419
• fix: Removed bytemask() in favour of mask_as_bool() by @ioanaif in #1410
• fix: Fix ak._v2.to_arrow for sliced ListOffsetArray. by @jpivarski in #1425
• fix: ListOffsetArray._reduce_next is not implemented for 32-bit. by @jpivarski in #1426
• fix: Fixed miscellaneous optiontype-in-Parquet bugs by @jpivarski in #1431
• fix: Remove duplicated import of to/from-parquet by @douglasdavis in #1435
• fix: is_unique() for IndexedArray by @ioanaif in #1429
• fix: Fixes to_layout with allow_records=False and allows single-record writing to Arrow and Parquet by @jpivarski in #1456
• fix: Fix RDataFrame GetColumnNames order in test. by @jpivarski in #1457
• fix: Streamline recursively_apply for small slices of big arrays. by @jpivarski in #1458
• fix: getitem* functions must consistently set the slicer in handle_error. by @jpivarski in #1469
• fix: to_rdataframe extensive tests and bug fixes by @ianna in #1478
• fix: Fix selecting columns from Parquet. by @jpivarski in #1486
• fix: Fix categorical equality handling (bad copy-paste from v1). by @jpivarski in #1492
• fix: _to_numpy method should return a numpy array by @swishdi...

Version 1.9.0rc14

No changes; re-triggering deployment because GitHub Actions failed to start for 1.9.0rc13.

Full Changelog: v1.9.0rc13...v1.9.0rc14

Version 1.9.0rc13

Features

• feat: prevent reducers like ak.sum on records (v2) by @ioanaif in #1607
• feat: Drop ak.behavior['.', 'Name'] = cls, which isn't working/isn't tested. by @jpivarski in #1651
• feat: retrieve multiple columns from RDataFrame in a single event loop by @ianna in #1625

Bug-fixes

• fix: Use dtype=np.int64, not int, for platform independence (Windows). by @jpivarski in #1636
• fix: support root empty field in Parquet file by @agoose77 in #1619
• fix: empty slice lists of record arrays (#1593) by @agoose77 in #1597
• fix: ak.from_iter should interpret top-level tuples as ak.Array (v1 and v2). by @jpivarski in #1642
• fix: Ensure that ak._v2.to_json raises errors when appropriate. by @jpivarski in #1649
• fix: pass a copy of RecordArray's internal fields in HL API by @Saransh-cpp in #1650
• fix: include self._length in RegularArray.mergemany by @agoose77 in #1645
• fix: jagged slicing for ListArray by @agoose77 in #1408
• fix: to_list must follow getitem implementations, even in Records by @jpivarski in #1652
• perf: ak.concatenate should flatten for first axis, single-array by @agoose77 in #1641

Other

• docs: add Saransh-cpp as a contributor for code by @allcontributors in #1653
• ci: run semantic-pr-title in concurrency group by @agoose77 in #1632
• ci: add Linux ROOT build to run ROOT-based tests by @ianna in #1629

Full Changelog: v1.9.0rc12...v1.9.0rc13

Version 1.9.0rc12

What's Changed

• ci: aarch64 should also use cibuildwheel v2.9.0 by @jpivarski in #1627

Full Changelog: 1.9.0rc11...v1.9.0rc12

1.9.0rc11

@ManasviGoyal implemented a new LayoutBuilder in header-only C++ templates: PR #1494. Documented new GrowableBuffer and LayoutBuilder in doxygen: PR #1579.

@ianna prepared ArrayBuilderOptions for the new LayoutBuilder: PR #1560. Fixed tolayout for Cling generators and Numba: PR #1613. Refactored from_rdataframe to use the new LayoutBuilder: PR #1620.

@ioanaif fixed missing axis_wrap_if_negative in Record (v1 and v2): PR #1565. Prevented ufuncs from being used on records unless overridden: PR #1559. Added an int64_t definition on Windows: PR #1572. Fixed "longlong" type bug discovered in Uproot: PR #1567. Fixed lengths of empty regular slices: PR #1568. Fixed maybe_toNumpyArray for RegularArrays: PR #1589.

@agoose77 protected ak.Array from unintended attribute assignment (when user wants to add a record field): PR #1573. Allowed empty RecordArrays in ak.to_layout: PR #1581. Switched from Azure to GitHub Actions (no more Azure!): PR #1550. Fixed from_iter performance: PR #1614.

@jpivarski prevented mixed Awkward-like and NumPy-like slicing: PR #1598. Renamed ak.to_pandas → ak.to_dataframe: PR #1603. Ensured that ak.concatenate preserves regularness of list types: PR #1604 and #1609. Ensured that ak.concatenate of identical Forms preserves the Form (for Dask): PR #1605. Added a public API for recursively_apply and broadcast_and_apply (ak._v2.transform): PR #1610. Finished refactoring all of the from_json* functions into just one: PR #1617 and #1621.

@henryiii enabled testing for Python 3.11: PR #1602.

In this release, we adopted a PR naming convention (Angular's, PR #1615) that should facilitate auto-generation of release notes, so this may be the last hand-written release note, organized by committer. From now on, I believe they'll be organized by type (new features, bug-fixes, CI changes, etc.).

Also, this is the last release with tag and release title formatted as just f"{version}"; starting with the next one, the tag name with be f"v{version}" and the release title will be f"Version {version}".

1.9.0rc9

@ianna migrated ArrayBuilder to the new GrowableBuffer: PR #1542.

@Moelf fixed a misleading error message: PR #1544.

@martindurant updated ak.from_parquet metadata file handling: PR #1476.

1.9.0rc10

@jpivarski removed std::cout from ArrayBuilder and print from several of the tests.

1.9.0rc8

@henryiii cleaned up dependabot: PR #1518.

@ianna set up Awkward's new header-only library: PRs #1523, #1524. Migrated old LayoutBuilder to v2: #1484.

@ManasviGoyal developed a new GrowableBuffer, using panels and in a header-only library: PR #1535.

@ioanaif fixed pre-commit issues: #1533. Initialized values behind mask in to_numpy: PR #1531. Fixed argument lists in deepcopy: PR #1532. Fixed is_none's use of axis: PR #1539.

@aryan26roy fixed the so-called "endianness bug on Mac" (actually the byte-swap was being optimized away): PR #1538.

@jpivarski fixed RecordArray.__repr__: PR #1537.

1.9.0rc7

@agoose77 fixed the MANIFEST file to include the new src/awkward/_v2/cpp-headers directory (which is the main reason for this pre-release): PR #1516.

@jpivarski updated the type-parser for v2: PR #1514.

1.9.0rc6

(Unless otherwise noted, from this release onward, all changes described in release notes are for v2.)

@ianna added a one-column implementation of ak.from_rdataframe: PR #1508.

@jpivarski added ak._v2.types.from_datashape, though the parser will need work to make it accept all v2 type strings: PR #1513.