Code

Spark 3.5.4#5

Code

Closed

ejblanco wants to merge 49 commits intospark-3.5Telefonica/spark:spark-3.5from spark-3.5.4Telefonica/spark:spark-3.5.4Copy head branch name to clipboard

+597-241

Commits

Commits on Oct 25, 2024

• [SPARK-50123][TESTS] Move BitmapExpressionUtilsSuite & ExpressionImplUtilsSuite from java to scala test sources folder

  Show description for e3e79e5
  

  yaooqinn

  authored and

  MaxGekk

  committed
  e3e79e5

  Copy full SHA for e3e79e5

Commits on Oct 29, 2024

• [SPARK-50150][BUILD][3.5] Upgrade Jetty to 9.4.56.v20240826

  Show description for 2f5e0fa
  dongjoon-hyun

  committed
  2f5e0fa

  Copy full SHA for 2f5e0fa

• [MINOR][BUILD] Skip deepspeed in requirements on MacOS

  Show description for 175b5e9
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  175b5e9

  Copy full SHA for 175b5e9

Commits on Oct 30, 2024

• [SPARK-50155][3.5] Move scala and java files to their default folders

  Show description for 4205b79
  

  panbingkun

  authored and

  MaxGekk

  committed
  4205b79

  Copy full SHA for 4205b79

Commits on Nov 1, 2024

• [SPARK-50176][CONNECT][3.5] Disallow reattaching after the session is closed

  Show description for 0229c0e
  

  changgyoopark-db

  authored and

  HyukjinKwon

  committed
  0229c0e

  Copy full SHA for 0229c0e

• [SPARK-50195][CORE] Fix StandaloneRestServer to propagate spark.app.name to SparkSubmit properly

  Show description for 6df1966
  dongjoon-hyun

  committed
  6df1966

  Copy full SHA for 6df1966

• [SPARK-50199][PYTHON][TESTS] Use Spark 3.4.4 instead of 3.0.1 in test_install_spark

  Show description for 9d47266
  dongjoon-hyun

  committed
  9d47266

  Copy full SHA for 9d47266

Commits on Nov 4, 2024

• [SPARK-50176][CONNECT][FOLLOWUP][3.5] Fix ReattachableExecuteSuite failure

  Show description for 08023c0
  

  changgyoopark-db

  authored and

  HyukjinKwon

  committed
  08023c0

  Copy full SHA for 08023c0

• [MINOR][DOCS][3.5] Fix specified java versions in install.rst

  Show description for 1ddf4a9
  

  dvorst

  authored and

  dongjoon-hyun

  committed
  1ddf4a9

  Copy full SHA for 1ddf4a9

• [SPARK-50210][CORE] Fix SparkSubmit to show REST API kill response properly

  Show description for 9b172de
  dongjoon-hyun

  committed
  9b172de

  Copy full SHA for 9b172de

Commits on Nov 5, 2024

• [SPARK-50212][BUILD][3.5] Fix the conditional check for executing the build task in build_and_test.yml

  Show description for acccf53
  LuciferYang

  committed
  acccf53

  Copy full SHA for acccf53

Commits on Nov 6, 2024

• [SPARK-50235][SQL] Clean up ColumnVector resource after processing all rows in ColumnarToRowExec

  Show description for d39f5ab
  

  viirya

  authored and

  yaooqinn

  committed
  d39f5ab

  Copy full SHA for d39f5ab

Commits on Nov 13, 2024

• [SPARK-50300][BUILD] Use mirror host instead of archive.apache.org

  Show description for 8da6987
  dongjoon-hyun

  committed
  8da6987

  Copy full SHA for 8da6987

Commits on Nov 14, 2024

• [SPARK-50304][INFRA] Remove (any|empty).proto from RAT exclusion

  Show description for 8502a22
  dongjoon-hyun

  committed
  8502a22

  Copy full SHA for 8502a22

• [SPARK-50316][BUILD][3.5] Upgrade ORC to 1.9.5

  Show description for e0bdfee
  dongjoon-hyun

  committed
  e0bdfee

  Copy full SHA for e0bdfee

Commits on Nov 15, 2024

• [SPARK-50312][SQL] SparkThriftServer createServer parameter passing error when kerberos is true

  Show description for 242d333
  

  cuiyanxiang

  authored and

  yaooqinn

  committed
  242d333

  Copy full SHA for 242d333

Commits on Nov 20, 2024

• [MINOR][DOCS] Fix a HTML/Markdown syntax error in sql-migration-guide.md

  Show description for 08b195c
  yaooqinn

  committed
  08b195c

  Copy full SHA for 08b195c

• [SPARK-50258][SQL] Fix output column order changed issue after AQE optimization

  Show description for df9b9de
  

  wangyum

  authored and

  cloud-fan

  committed
  df9b9de

  Copy full SHA for df9b9de

Commits on Nov 27, 2024

• [SPARK-50430][CORE] Use the standard Properties.clone instead of manual clone

  Show description for 5ff129a
  HyukjinKwon

  committed
  5ff129a

  Copy full SHA for 5ff129a

Commits on Dec 2, 2024

• [SPARK-49294][UI] Add width attribute for shuffle-write-time checkbox

  Show description for 5e51e2c
  

  xunxunmimi5577

  authored and

  panbingkun

  committed
  5e51e2c

  Copy full SHA for 5e51e2c

Commits on Dec 3, 2024

• [SPARK-50433][DOCS][TESTS][3.5] Fix configuring log4j2 guide docs for Spark on YARN and UT

  Show description for 1d6f7ad
  

  pan3793

  authored and

  dongjoon-hyun

  committed
  1d6f7ad

  Copy full SHA for 1d6f7ad

Commits on Dec 4, 2024

• [SPARK-50487][DOCS] Update broken jira link

  Show description for 5dc927b
  

  2 peopleauthored and

  yaooqinn

  committed
  5dc927b

  Copy full SHA for 5dc927b

Commits on Dec 5, 2024

• [SPARK-50498][PYTHON] Avoid unnecessary py4j call in listFunctions

  Show description for acedb15
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  acedb15

  Copy full SHA for acedb15

• [SPARK-50505][DOCS] Fix spark.storage.replication.proactive default value documentation

  Show description for 86e29e9
  dongjoon-hyun

  committed
  86e29e9

  Copy full SHA for 86e29e9

Commits on Dec 6, 2024

• [SPARK-50492][SS] Fix java.util.NoSuchElementException when event time column is dropped after dropDuplicatesWithinWatermark

  Show description for d01f34f
  

  liviazhu-db

  authored and

  HeartSaVioR

  committed
  d01f34f

  Copy full SHA for d01f34f

• [SPARK-50492][SS][FOLLOWUP][3.5] Change def references to lazy val references in DeduplicateWithinWatermark to fix the compilation issue

  Show description for 153cd9e
  LuciferYang

  committed
  153cd9e

  Copy full SHA for 153cd9e

• [SPARK-50421][CORE][3.5] Fix executor related memory config incorrect when multiple resource profiles worked

  Show description for bf29ab9
  

  zjuwangg

  authored and

  dongjoon-hyun

  committed
  bf29ab9

  Copy full SHA for bf29ab9

Commits on Dec 7, 2024

• [SPARK-50514][DOCS] Add IDENTIFIER clause page to menu-sql.yaml

  Show description for d8f3afa
  

  dongjoon-hyun

  authored and

  LuciferYang

  committed
  d8f3afa

  Copy full SHA for d8f3afa

• [SPARK-49695][SQL][3.5] Postgres fix xor push-down

  Show description for 305d2a0
  

  andrej-db

  authored and

  dongjoon-hyun

  committed
  305d2a0

  Copy full SHA for 305d2a0

Commits on Dec 8, 2024

• [SPARK-50483][CORE][SQL][3.5] BlockMissingException should be thrown even if ignoreCorruptFiles is enabled

  Show description for a57f3c2
  

  wangyum

  authored and

  dongjoon-hyun

  committed
  a57f3c2

  Copy full SHA for a57f3c2

Commits on Dec 9, 2024

• Preparing Spark release v3.5.4-rc1

  LuciferYang

  committed
  929a19f

  Copy full SHA for 929a19f

• Preparing development version 3.5.5-SNAPSHOT

  LuciferYang

  committed
  8e6507a

  Copy full SHA for 8e6507a

• [SPARK-49134][INFRA][3.5] Support retry for deploying artifacts to Nexus staging repository

  Show description for deabe49
  

  yaooqinn

  authored and

  LuciferYang

  committed
  deabe49

  Copy full SHA for deabe49

Commits on Dec 10, 2024

• [SPARK-50463][SQL][3.5] Fix ConstantColumnVector with Columnar to Row conversion

  Show description for a3cf28e
  

  richardc-db

  authored and

  LuciferYang

  committed
  a3cf28e

  Copy full SHA for a3cf28e

Commits on Dec 11, 2024

• [SPARK-50510][CONNECT][3.5] Fix sporadic ReattachableExecuteSuite failure

  Show description for bb953f9
  

  changgyoopark-db

  authored and

  LuciferYang

  committed
  bb953f9

  Copy full SHA for bb953f9

Commits on Dec 12, 2024

• [SPARK-50087][SQL][3.5] Robust handling of boolean expressions in CASE WHEN for MsSqlServer and future connectors

  Show description for e97580a
  

  andrej-db

  and

  cloud-fan

  committed
  e97580a

  Copy full SHA for e97580a

• [SPARK-50545][CORE][SQL][3.5] AccessControlException should be thrown even if ignoreCorruptFiles is enabled

  Show description for 92e650c
  

  pan3793

  authored and

  LuciferYang

  committed
  92e650c

  Copy full SHA for 92e650c

Commits on Dec 16, 2024

• Preparing Spark release v3.5.4-rc2

  LuciferYang

  committed
  91af6f9

  Copy full SHA for 91af6f9

• Preparing development version 3.5.5-SNAPSHOT

  LuciferYang

  committed
  a764524

  Copy full SHA for a764524

• [SPARK-50430][CORE][FOLLOW-UP] Keep the logic of manual putting key and values in Properties

  Show description for 8168ea8
  HyukjinKwon

  committed
  8168ea8

  Copy full SHA for 8168ea8

• Revert "[SPARK-50430][CORE][FOLLOW-UP] Keep the logic of manual putting key and values in Properties"

  Show description for 0fbe292
  HyukjinKwon

  committed
  0fbe292

  Copy full SHA for 0fbe292

• Revert "[SPARK-50430][CORE] Use the standard Properties.clone instead of manual clone"

  Show description for f7c48fe
  HyukjinKwon

  committed
  f7c48fe

  Copy full SHA for f7c48fe

• [SPARK-50587][INFRA][3.5] Remove unsupported curl option --retry-all-errors from release-build.sh

  Show description for b0a7d4d
  LuciferYang

  committed
  b0a7d4d

  Copy full SHA for b0a7d4d

Commits on Dec 17, 2024

• Preparing Spark release v3.5.4-rc3

  LuciferYang

  committed
  a6f220d

  Copy full SHA for a6f220d

• Preparing development version 3.5.5-SNAPSHOT

  LuciferYang

  committed
  bcaa5a9

  Copy full SHA for bcaa5a9

• [SPARK-50510][CONNECT][TEST][3.5] Fix flaky ReattachableExecuteSuite

  Show description for 45349b6
  

  changgyoopark-db

  authored and

  HyukjinKwon

  committed
  45349b6

  Copy full SHA for 45349b6

Commits on Dec 18, 2024

• [MINOR][SS] Minor update to watermark propagation comments

  Show description for a3d23fd
  

  neilramaswamy

  authored and

  HeartSaVioR

  committed
  a3d23fd

  Copy full SHA for a3d23fd

Commits on Dec 23, 2024

• [SPARK-50483][SPARK-50545][DOC][FOLLOWUP][3.5] Mention behavior changes in migration guide

  Show description for 5a91172
  

  pan3793

  authored and

  HyukjinKwon

  committed
  5a91172

  Copy full SHA for 5a91172

Commits on Jan 7, 2025

• Merge remote-tracking branch 'spark/branch-3.5' into spark-3.5

  ejblanco

  committed
  fc601e2

  Copy full SHA for fc601e2