spark-sql-streaming-StreamWriteSupport.adoc

StreamWriteSupport Contract — Writable Streaming Data Sources

StreamWriteSupport is the abstraction of DataSourceV2 sinks that create StreamWriters for streaming write (when used in streaming queries in MicroBatchExecution and ContinuousExecution).

StreamWriter createStreamWriter(
  String queryId,
  StructType schema,
  OutputMode mode,
  DataSourceOptions options)

createStreamWriter creates a StreamWriter for streaming write and is used when the stream execution thread for a streaming query is started and requests the stream execution engines to start, i.e.

• ContinuousExecution is requested to runContinuous

• MicroBatchExecution is requested to run a single streaming batch

Table 1. StreamWriteSupports

StreamWriteSupport	Description
ConsoleSinkProvider	Streaming sink for console data source format
ForeachWriterProvider
KafkaSourceProvider
MemorySinkV2