Releasing 0.6.0

CHANGELOG.md

@@ -3,17 +3,24 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
-## [UNRELEASED] - YYYY-MM-DD
+## [0.6.0] - 2021-03-05
 
 ### Added
 - Adds support to read string predicates with language tags like `<http://www.w3.org/2000/01/rdf-schema#label@en>` ([issue #63](https://github.com/G-Research/spark-dgraph-connector/issues/63)).
   This works with any source and mode except the node source in wide mode.
   Note that reading into GraphFrames is based on the wide mode, so only the untagged
   language strings can be read there.
-  Filter pushdown is not supported yet for multi-language predicates ([issue #68](https://github.com/G-Research/spark-dgraph-connector/issues/68)).
+  Filter pushdown is not supported for multi-language predicates yet ([issue #68](https://github.com/G-Research/spark-dgraph-connector/issues/68)).
+- Adds readable exception and suggests next steps when GRPC fails with `RESOURCE_EXHAUSTED` code.
+- Missing `maxLeaseId` in cluster state response defaults to `1000L` to avoid an exception.
 
 ### Changed
-- Upgraded all dependencies to latest versions
+- Improves predicate partitioning on projection pushdown as it creates full partitions.
+- Fixes bug that did not push predicate value filter correctly down to Dgraph causing incorrect results ([issue #82](https://github.com/G-Research/spark-dgraph-connector/pull/82))
+- Fixes bug in reading `geo` and `password` data types.
+- Tests against Dgraph 20.03, 20.07 and 20.11.
+- Moved Java Dgraph client to 20.11.0.
+- Upgraded all dependencies to latest versions.
 
 ## [0.5.0] - 2020-10-21
 
@@ -28,7 +35,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ### Changed
 - Refactored connector API, renamed `spark.read.dgraph*` methods to `spark.read.dgraph.*`.
 - Moved `triples`, `edges` and `nodes` sources from package `uk.co.gresearch.spark.dgraph.connector` to `uk.co.gresearch.spark.dgraph`.
-- Moved Dgraph client to 20.03.1 and Dgraph test cluster to 20.07.0.
+- Moved Java Dgraph client to 20.03.1 and Dgraph test cluster to 20.07.0.
 
 ## [0.4.0] - 2020-07-24
 

README.md

@@ -59,7 +59,7 @@ The connector is under continuous development. It has the following known limita
 
 The Spark Dgraph Connector is available for Spark 2.4, Spark 3.0 and Spark 3.1, all with Scala 2.12.
 Use Maven artifact ID `spark-dgraph-connector_2.12`. The Spark version is part of the package version,
-e.g. 0.5.0-2.4, 0.5.0-3.0 and  0.5.0-3.1, respectively.
+e.g. 0.6.0-2.4, 0.6.0-3.0 and  0.6.0-3.1, respectively.
 Minor versions are kept in sync between those two packages,
 such that identical minor versions contain identical feature sets
 (where supported by the respective Spark version).
@@ -69,7 +69,7 @@ such that identical minor versions contain identical feature sets
 Add this line to your `build.sbt` file to use the latest version for Spark 2.4:
 
 ```sbt
-libraryDependencies += "uk.co.gresearch.spark" %% "spark-dgraph-connector" % "0.5.0-2.4"
+libraryDependencies += "uk.co.gresearch.spark" %% "spark-dgraph-connector" % "0.6.0-2.4"
 ```
 
 ### Maven
@@ -80,7 +80,7 @@ Add this dependency to your `pom.xml` file to use the latest version:
 <dependency>
   <groupId>uk.co.gresearch.spark</groupId>
   <artifactId>spark-dgraph-connector_2.12</artifactId>
-  <version>0.5.0-2.4</version>
+  <version>0.6.0-2.4</version>
 </dependency>
 ```
 
@@ -89,21 +89,21 @@ Add this dependency to your `pom.xml` file to use the latest version:
 Launch the Scala Spark REPL (Spark ≥2.4.0) with the Spark Dgraph Connector dependency (version ≥0.5.0) as follows:
 
 ```shell script
-spark-shell --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.5.0-2.4
+spark-shell --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.6.0-2.4
 ```
 
 ### PySpark Shell and Python script
 
 Launch the Python Spark REPL (pyspark 2.4.2 and ≥3.0) with the Spark Dgraph Connector dependency (version ≥0.5.0) as follows:
 
 ```shell script
-pyspark --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.5.0-2.4
+pyspark --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.6.0-2.4
 ```
 
 Run your Python scripts that use PySpark (pyspark 2.4.2 and ≥3.0) and the Spark Dgraph Connector (version ≥0.5.0) via `spark-submit`:
 
 ```shell script
-spark-submit --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.5.0-2.4 [script.py]
+spark-submit --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.6.0-2.4 [script.py]
 ```
 
 ## Examples

examples/scala/pom.xml

@@ -3,7 +3,7 @@
 
   <groupId>uk.co.gresearch.spark</groupId>
   <artifactId>spark-dgraph-connector-examples_2.12</artifactId>
-  <version>0.6.0-2.4-SNAPSHOT</version>
+  <version>0.6.0-2.4</version>
   <name>Spark Dgraph Connector Example</name>
 
   <properties>

pom.xml

@@ -2,7 +2,7 @@
   <modelVersion>4.0.0</modelVersion>
   <groupId>uk.co.gresearch.spark</groupId>
   <artifactId>spark-dgraph-connector_2.12</artifactId>
-  <version>0.6.0-2.4-SNAPSHOT</version>
+  <version>0.6.0-2.4</version>
   <name>Spark Dgraph Connector</name>
   <description>A Spark connector for Dgraph databases</description>
   <inceptionYear>2020</inceptionYear>