Code

3.4+#218

Code

Draft

Jolanrensen wants to merge 40 commits intomainKotlin/kotlin-spark-api:mainfrom 3.4+Kotlin/kotlin-spark-api:3.4+Copy head branch name to clipboard

+5,575-4,780

Commits

Commits on Mar 17, 2024

• updating version range to 3.4.2 and 3.5.1

  Jolanrensen

  committed
  cda9548

  Copy full SHA for cda9548

• (re)moving some stuff from the decoupled :core module

  Jolanrensen

  committed
  42ba339

  Copy full SHA for 42ba339

• adding initial new KotlinTypeInference Encoder generator, this time purely in Kotlin. Not complete yet

  Jolanrensen

  committed
  4896354

  Copy full SHA for 4896354

• updating all references to the old encoder<>() function in favor of the new kotlinEncoderFor<>()

  Jolanrensen

  committed
  e234f40

  Copy full SHA for e234f40

• updating tests tests

  Jolanrensen

  committed
  66a42ac

  Copy full SHA for 66a42ac

• made name hack optional as it is not waterproof

  Jolanrensen

  committed
  3e9261f

  Copy full SHA for 3e9261f

• added tests and fixed name hack

  Jolanrensen

  committed
  0d061b6

  Copy full SHA for 0d061b6

• Map support

  Jolanrensen

  committed
  0c8f4b1

  Copy full SHA for 0c8f4b1

Commits on Mar 18, 2024

• disable name hack by default again, added JCP case for auto-applying the expression encoder without spark-connect

  Jolanrensen

  committed
  9149607

  Copy full SHA for 9149607

• udt fix

  Jolanrensen

  committed
  4364022

  Copy full SHA for 4364022

Commits on Mar 20, 2024

• enabled core as scala-helpers with VarargUnwrapper. Removed name hack in favor of upcoming IR compiler plugin

  Jolanrensen

  committed
  2c875ff

  Copy full SHA for 2c875ff

• enabled core as scala-helpers with VarargUnwrapper. Removed name hack in favor of upcoming IR compiler plugin. Removed spark dependency in scala-helpers

  Jolanrensen

  committed
  d60e4dc

  Copy full SHA for d60e4dc

Commits on Mar 21, 2024

• updating some build parameters

  Jolanrensen

  committed
  4f8d874

  Copy full SHA for 4f8d874

• bumped to kotlin 2.0.0-Beta5, included the compiler plugin with tests as separate module

  Jolanrensen

  committed
  2a878f1

  Copy full SHA for 2a878f1

Commits on Mar 22, 2024

• Added gradle plugin and fixed package naming errors

  Jolanrensen

  committed
  c0a3140

  Copy full SHA for c0a3140

• enabling the compiler plugin on modules, sparkifying data classes

  Jolanrensen

  committed
  1b0b316

  Copy full SHA for 1b0b316

• bumping down to kotlin 1.9.23 to make lambda's serializable again

  Jolanrensen

  committed
  b7c1711

  Copy full SHA for b7c1711

• fixing tests

  Jolanrensen

  committed
  7069a9a

  Copy full SHA for 7069a9a

Commits on Mar 23, 2024

• fixing more tests. Can now remain at Kotlin 2.0 if we set -Xlambdas=class, which can be done with gradle plugin

  Jolanrensen

  committed
  df021c0

  Copy full SHA for df021c0

Commits on Mar 24, 2024

• added conversion for @sparkify'ed classes to scala.Product with tests.

  Jolanrensen

  committed
  4c17859

  Copy full SHA for 4c17859

• started WIP fir plugin, disabled for now

  Jolanrensen

  committed
  2f62d07

  Copy full SHA for 2f62d07

• disabled RddTest "Work with any number" for now: Caused by: java.lang.NoSuchMethodException: org.jetbrains.kotlinx.spark.api.RddTest$1$1$1$2$2$1.$deserializeLambda$(java.lang.invoke.SerializedLambda)

  Jolanrensen

  committed
  770698b

  Copy full SHA for 770698b

Commits on Mar 25, 2024

• Merge branch 'main' into 3.4+

  Jolanrensen

  authored
  5d5fe84

  Copy full SHA for 5d5fe84

• added publishToMavenLocal for the plugins in gh actions build.yml

  Jolanrensen

  committed
  24952b7

  Copy full SHA for 24952b7

• Merge remote-tracking branch 'origin/3.4+' into 3.4+

  Jolanrensen

  committed
  ffedb29

  Copy full SHA for ffedb29

• editing build so that

  Show description for 6b63c96
  Jolanrensen

  committed
  6b63c96

  Copy full SHA for 6b63c96

• fixing compiler-plugin tests

  Jolanrensen

  committed
  4e5438b

  Copy full SHA for 4e5438b

• fixed conversions for scala 2.12

  Jolanrensen

  committed
  723152b

  Copy full SHA for 723152b

• small fixes, disabled jupyter module for now

  Jolanrensen

  committed
  0db290a

  Copy full SHA for 0db290a

• spark 3.4/3.5 compatibility issue with ProductEncoder

  Jolanrensen

  committed
  37eec64

  Copy full SHA for 37eec64

• fixed streaming test scala 2.12

  Jolanrensen

  committed
  0ab212b

  Copy full SHA for 0ab212b

Commits on Mar 27, 2024

• added checkIsSparkified warnings when building an encoder

  Jolanrensen

  committed
  1c34429

  Copy full SHA for 1c34429

• updated compiler plugin to make @sparkify classes Serializable too, plus added test to see if the plugin is enabled in the project

  Jolanrensen

  committed
  68e830a

  Copy full SHA for 68e830a

Commits on Mar 30, 2024

• added java bean class fallback support

  Jolanrensen

  committed
  48db819

  Copy full SHA for 48db819

Commits on Apr 7, 2024

• added encoding for DatePeriod, DateTimePeriod, Instant, LocalDateTime, and LocalDate, Duration not working

  Jolanrensen

  committed
  ab4c455

  Copy full SHA for ab4c455

Commits on Apr 10, 2024

• kotlin 2.0.0-RC1, enabled jupyter module. Added basic Sparkify conversion for jupyter. Disabled html renderes in favor of just outputting them as text. Notebooks can render them however they like. …

  Show description for e05feac
  Jolanrensen

  committed
  e05feac

  Copy full SHA for e05feac

• make render in jupyter use show instead

  Jolanrensen

  committed
  7fd77be

  Copy full SHA for 7fd77be

Commits on May 18, 2024

• disabled jupyter tests relying on DISPLAY(), update jupyter to latest java 8 version

  Jolanrensen

  committed
  92b699d

  Copy full SHA for 92b699d

• disabled unsupported rddtests

  Jolanrensen

  committed
  eae0196

  Copy full SHA for eae0196

Commits on Jun 15, 2024

• added spark connect example which does not yet work with kotlin-spark-api

  Jolanrensen

  committed
  22fa5ae

  Copy full SHA for 22fa5ae