Skip to content

Releases: timgent/data-flare

Changes in package visibility to support data-flare API and UI development

30 Apr 12:21
Compare
Choose a tag to compare

QcTypes made public

06 Mar 20:41
8360ca8
Compare
Choose a tag to compare
v0.1.13

made QcType public (#117)

Min/Max metrics & checks, identical table checks, schema checks

01 Mar 09:26
Compare
Choose a tag to compare
  • MinValueMetric, MaxValueMetric, minValueCheck, maxValueCheck added
  • dfsMatchUnordered and dfsMatchOrdered check helpers added
  • dsSchemasMatch check added

Distinctness metric, compliance empty DS fix, runBlocking method

15 Oct 10:57
Compare
Choose a tag to compare
  • Distinctness metric added (docs still pending)
  • runBlocking method added for running check suite and waiting for the result before returning (for those who don't want to have to deal with a Future)
  • Compliance metric bug fixed where it was giving a value of 0 for empty dataset, rather than 1

Cross-build to Scala 2.12 and improve error handling

14 Jul 21:59
Compare
Choose a tag to compare
  • If arbChecks or metric based checks throw exceptions they are now caught, and details of those returned in the check results for the relevant checks
  • Deequ support removed due to issues it would cause for building Scala 2.12 version of this library
  • Now cross-builds to Scala 2.12

Interface simplification and new features

13 Jul 08:42
Compare
Choose a tag to compare

BREAKING CHANGES

  • Simplification of API - see docs for updates

ADDITIONAL CHANGES

  • Additional constructors for writing filters with less boilerplate
  • Add method to pretty print QC Results
  • Make compilation for checks stricter so types have to align
  • Allow storage of metrics even when no checks for them
  • Comparator helper added for checking if values are within X percent of each other
  • SumValues metric bug fixed where onColumn info was not stored or returned
  • Made check descriptions a sealed trait so they display more information more clearly

Interface tweak

08 Jul 07:40
Compare
Choose a tag to compare
  • Renamed DatasetComparisonCheck to DualDatasetCheck
  • No other functional changes

First release

07 Jul 05:17
Compare
Choose a tag to compare

Testing CI...