0.14.0

Release notes

Enhancements

• This version begins the transition to DBN version 2 (DBNv2). In this version, the
  decoders support decoding both versions of DBN and the DBN encoders default to
  keeping version of the input. However, in a future version, decoders will by default
  convert DBNv1 to DBNv2 and support will be dropped for encoding DBNv1.
  • Affects SymbolMappingMsg, InstrumentDefMsg, and Metadata. All other record
    types and market data schemas are unchanged
  • Version 1 structs can be converted to version 2 structs with the From trait
• Added symbol_cstr_len field to Metadata to indicate the length of fixed symbol
  strings
• Added stype_in and stype_out fields to SymbolMappingMsg to provide more context
  with live symbology updates
• Added smart wrapping to dbn CLI help output
• Updated rtype_dispatch family of macros to check record length to handle both
  versions of records. This is temporary during the transition period
• Added VersionUpgradePolicy enum and associated methods to the decoders to
  allow specifying how to handle decoding records from prior DBN versions
• Added Metadata::upgrade() method to update Metadata from a prior DBN version to
  the latest version
• Added -u/--upgrade flags to dbn CLI that when passed upgrades DBN data from
  previous versions. By default data is decoded as-is
• Made AsyncDbnDecoder::decode_record, AsyncDbnDecoder::decode_record_ref,
  dbn::AsyncRecordDecoder::decode, and dbn::AsyncRecordDecoder::decode_ref
  cancellation safe. This makes them safe to use within a
  tokio::select!(https://docs.rs/tokio/latest/tokio/macro.select.html) statement
• Added documention around cancellation safety for async APIs
• Improved error messages for conversion errors
• Added TOB flag to denote top-of-book messages
• Added new publisher values in preparation for IFEU.IMPACT and NDEX.IMPACT datasets
• Added new publisher values for consolidated DBEQ.BASIC and DBEQ.PLUS
• Added MAX_RECORD_LEN constant for the length of the largest record type
• Exposed record flag constants in databento_dbn with F_ prefix
• Added export to Python for RType

Breaking changes

• The old InstrumentDefMsg is now compat::InstrumentDefMsgV1
• compat::InstrumentDefMsgV2 is now an alias for InstrumentDefMsg
• The old SymbolMappingMsg is now compat::SymbolMappingMsgV1
• compat::SymbolMappingMsgV2 is now an alias for SymbolMappingMsg
• Changed SYMBOL_CSTR_LEN constant to 71. Previous value is now in
  compat::SYMBOL_CSTR_V1
• Changed DBN_VERSION constant to 2
• security_update_action was converted to a raw c_char to safely support adding
  variants in the future
• Renamed _dummy in InstrumentDefMsg to _reserved
• Removed _reserved2, _reserved3, and _reserved5 from InstrumentDefMsg
• Removed _dummy from SymbolMappingMsg
• Moved position of strike_price within InstrumentDefMsg but left text serialization
  order unchanged
• Made Error non-exhaustive, meaning it can no longer be exhaustively matched against.
  This allows adding additional error variants in the future without a breaking change
• Added upgrade_policy parameter to RecordDecoder::with_version constructor to
  control whether records of previous versions will be upgraded
• Added upgrade_policy parameter to DynDecoder constructors to control whether
  records of previous versions will be upgraded
• Renamed symbol_map parameter for Python Transcoder to symbol_interval_map to
  better reflect the date intervals it contains

Deprecations

• Deprecated unused write_dbn_file function from Python interface. Please use
  Transcoder instead

Bug fixes

• Fixed typo in Python type definition for InstrumentDefMsg.pretty_high_limit_price
• Fixed type signature for Metadata.stype_in and Metadata.stype_out Python methods
• Fixed incorrect version in pyproject.toml

0.13.0

Release notes

Enhancements

• Added SymbolMappingMsgV2::new method
• Added Record trait for all types beginning with a RecordHeader
  • Added new index_ts and raw_index_ts methods to Record trait, which returns the
    primary timestamp for a record
• Added RecordMut trait for accessing a mutable reference to a RecordHeader
• Implemented PartialOrd for all record types, based on raw_index_ts
• Loosened DbnEncodable from requiring HasRType to only requiring Record. This means
  RecordRefs and concrete records can be encoded with the same methods

Breaking changes

• Split part of HasRType into new Record and RecordMut traits, which are object-
  safe: they can be used in Box<dyn>. RecordRef also implements Record, so it's
  easier to write code that works for both concrete records as well as RecordRef
• Removed RecordRef methods made redundant by it implementing Record
• Removed input_compression parameter from Python Transcoder

Deprecations

• Deprecated SymbolIndex::get_for_rec_ref, which was made redundant by loosening the
  trait bound on SymbolIndex::get_for_rec to accept RecordRefs

Bug fixes

• Fixed TsSymbolMap not always using the correct timestamp for getting the mapped
  symbol

0.12.0

Release notes

Enhancements

• Added map_symbols support to Python Transcoder
• Added new publisher variants in preparation for DBEQ.PLUS dataset
• Added from_dataset_venue function to Publisher to facilitate destructuring
• Implemented Default for most records to make testing easier
• Added from_zstd function to AsyncDbnEncoder to match synchronous encoder
• Added re-exports for enums::flags, enums::rtype, record::BidAskPair,
  record::RecordHeader, and record::WithTsOut to simplify imports
• Added --fragment CLI flag for writing DBN without the metadata header
• Added --input-dbn-version CLI option for specifying the DBN version of a DBN
  fragment
• Added serde::Deserialize implementations for Dataset, Venue, and Publisher
• Added support for Python 3.12 to databento_dbn
• Added RecordDecoder::with_version for future use when dealing with compatibility
  between different DBN versions
• Added new dispatch macros: rtype_ts_out_method_dispatch,
  rtype_ts_out_async_method_dispatch, rtype_method_dispatch, and
  schema_ts_out_method_dispatch
• Added InstrumentDefMsgV2 and SymbolMappingMsgV2 for forward compatibility with a
  version of DBN
• Added TsSymbolMap and PitSymbolMap to aid with both historical and live symbology
  • Added support for inverse symbology, i.e. with stype_in=InstrumentId

Breaking changes

• Changed Metadata::symbol_map to return TsSymbolMap
• Changed Metadata::symbol_map_for_date to return PitSymbolMap
• Changed Default implementation for BidAskPair by setting prices to UNDEF_PRICE
• Added new publisher values in preparation for DBEQ.PLUS
• Added ts_out parameter to encode_header_for_schema in CsvEncoder and
  DynEncoder to allow controlling whether "ts_out" is in the header
• Added from_dataset_venue function to Publisher to facilitate
  destructuring.

0.11.1

Release notes

Enhancements

• Upgraded async-compression to 0.4.3
• Upgraded csv to 1.3
• Upgraded num_enum to 0.7

Bug fixes

• Changed DBN stream detection to ignore the DBN version

0.11.0

Release notes

Enhancements

• Added new EncodeRecordTextExt trait which is implemented for the CSV and JSON
  encoders. It adds two methods for encoding a symbol field along side the rest of the
  record fields, matching the behavior of map_symbols in the historical API
• Added encode_header and encode_header_for_schema methods to CsvEncoder and
  DynEncoder to give more flexibility for encoding CSV headers
• Added from_file and from_zstd_file functions to AsyncDbnDecoder to match
  synchronous decoder
• Implemented Copy for RecordRef to make it behave more like a reference
• Added AsyncDbnEncoder for simpler DBN encoding and to match sync API
• Added RecordEnum and RecordRefEnum to more easily be able to pattern match on
  records of different types
• Added ARCX.PILLAR.ARCX publisher
• Added From DBN records for RecordRef
• Added re-exports to the top level of the crate for all enums and records for simpler
  imports
• Added ClosePrice and NetChange StatTypes used in the OPRA.PILLAR dataset

Breaking changes

• Split encode_record_ref into a safe method with no arguments and an unsafe method
  with a ts_out parameter to reduce unsafe usage when not working with live data
  that may contain ts_out

Bug fixes

• Fixed dbn CLI not writing CSV header when using --fragment and --zstd-fragment
  flags
• Fixed lifetime on return value from RecordRef::get_unchecked
• Fixed missing check for stype_out before building Metadata symbology maps

0.10.2

Release notes

Bug fixes

• Fixed query range checking in Metadata::symbol_map_for_date
• Added debug_assert_eq! check for alignment in RecordRef::new

0.10.1

Release notes

Bug fixes

• Changed Metadata::symbol_map and symbol_map_for_date to return String values
  instead of &str, which made it difficult to use

0.10.0

Release notes

Enhancements

• Added start and end getters to Metadata that return time::OffsetDateTime
• Added symbol_map and symbol_map_for_date methods to Metadata to aid historical
  symbology mapping from the instrument IDs in records
• Added DynReader struct for being agnostic about whether an input stream is
  zstd-compressed
• Improved safety of RecordRef::get by adding length check
• Added Python DBN Transcoder class for converting DBN to JSON and CSV with optional
  zstd compression
• Added optional has_metadata parameter to Python DBNDecoder to allow
  decoding plain records by passing False. By default DBNDecoder expects a complete
  DBN stream, which begins with metadata
• Added get_ref methods to dbn::Decoder and dbn::RecordDecoder which return a
  reference to the inner reader
• Added UNDEF_PRICE, UNDEF_ORDER_SIZE, UNDEF_STAT_QUANTITY, and UNDEF_TIMESTAMP
  constants to databento_dbn Python package to make it easier to filter null values
• Added Metadata::builder() function to create a new builder instance

Breaking changes

• Split out EncodeRecordRef trait from EncodeDbn to have a boxable trait (i.e.
  Box<dyn EncodeRecordRef>) for dynamic encoding
• Split out EncodeRecord trait from EncodeDbn
• Split out DecodeRecordRef trait from DecodeDbn to have a boxable trait (i.e.
  Box<dyn DecodeRecordRef>) for dynamic decoding
• Changed DynWriter from an enum to a struct with only private fields

Bug fixes

• Fixed typo in BATY.PITCH.BATY publisher
• Fixed typo in README.md (credit: @thomas-k-cameron)

0.9.0

Release notes

Enhancements

• Added publisher method to RecordHeader and all record types for converting
  the publisher_id to an enum
• Added getters that return time::OffsetDateTime for the following fields:
  ts_event, ts_recv, ts_ref, activation, expiration, start_ts, end_ts,
  ts_out
• Added getters for ts_in_delta that return time::Duration

0.8.3

Release notes

Bug fixes

• Fixed missing raw_instrument_id field in Python InstrumentDefMsg
• Fixed missing OHLCV_EOD variant in Python Schema type hint