spark-sql-streaming-ForeachWriter.adoc

ForeachWriter

ForeachWriter is the contract for a foreach writer that is a streaming format that controls streaming writes.

Note	ForeachWriter is set using foreach operator.

val foreachWriter = new ForeachWriter[String] { ... }
streamingQuery.
  writeStream.
  foreach(foreachWriter).
  start

ForeachWriter Contract

package org.apache.spark.sql

abstract class ForeachWriter[T] {
  def open(partitionId: Long, version: Long): Boolean
  def process(value: T): Unit
  def close(errorOrNull: Throwable): Unit
}

Table 1. ForeachWriter Contract

Method	Description
open	Used when…​
process	Used when…​
close	Used when…​