Skip to content

Commit

Permalink
[SPARK-47305][SQL][TESTS][FOLLOWUP][3.4] Fix the compilation error re…
Browse files Browse the repository at this point in the history
…lated to `PropagateEmptyRelationSuite`

### What changes were proposed in this pull request?
apache#45406 has been backported to branch-3.4, where the newly added test case in `PropagateEmptyRelationSuite` uses `DataTypeUtils`, but `DataTypeUtils` is a utility class added in Apache Spark 3.5(SPARK-44475), so this triggered a compilation failure in branch-3.4:

- https://github.com/apache/spark/actions/runs/8183755511/job/22377119069

```
[error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:229:27: not found: value DataTypeUtils
[error]     val schemaForStream = DataTypeUtils.fromAttributes(outputForStream)
[error]                           ^
[error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:233:26: not found: value DataTypeUtils
[error]     val schemaForBatch = DataTypeUtils.fromAttributes(outputForBatch)
[error]                          ^
[info] done compiling
[info] compiling 1 Scala source to /home/runner/work/spark/spark/connector/connect/common/target/scala-2.12/test-classes ...
[info] compiling 25 Scala sources and 1 Java source to /home/runner/work/spark/spark/connector/connect/client/jvm/target/scala-2.12/classes ...
[info] done compiling
[error] two errors found
```

Therefore, this PR changes to use the `StructType.fromAttributes` function to fix the compilation failure."

### Why are the changes needed?
Fix the compilation failure in branch-3.4

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass Github Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#45428 from LuciferYang/SPARK-47305-FOLLOWUP-34.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
  • Loading branch information
LuciferYang committed Mar 8, 2024
1 parent ddb112d commit 8b43164
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -226,11 +226,11 @@ class PropagateEmptyRelationSuite extends PlanTest {
val data = Seq(Row(1))

val outputForStream = Seq($"a".int)
val schemaForStream = DataTypeUtils.fromAttributes(outputForStream)
val schemaForStream = StructType.fromAttributes(outputForStream)
val converterForStream = CatalystTypeConverters.createToCatalystConverter(schemaForStream)

val outputForBatch = Seq($"b".int)
val schemaForBatch = DataTypeUtils.fromAttributes(outputForBatch)
val schemaForBatch = StructType.fromAttributes(outputForBatch)
val converterForBatch = CatalystTypeConverters.createToCatalystConverter(schemaForBatch)

val streamingRelation = LocalRelation(
Expand Down

0 comments on commit 8b43164

Please sign in to comment.