spark-webui.adoc

WebUI — Spark Application’s web UI

Web UI (aka webUI or Spark UI after SparkUI) is the web interface of a Spark application to inspect job executions in the SparkContext using a browser.

Figure 1. Welcome page - Jobs page

Every SparkContext launches its own instance of Web UI which is available at http://[master]:4040 by default (the port can be changed using spark.ui.port setting).

web UI comes with the following tabs:

• Jobs

• Stages

• Storage with RDD size and memory use

• Environment

• Executors

• SQL

This information is available only until the application is running by default.

Tip	You can use the web UI after the application is finished by persisting events using EventLoggingListener.

Note	All the information that are displayed in web UI is available thanks to JobProgressListener. One could say that web UI is a web layer to JobProgressListener.

Environment Tab

Figure 2. Environment tab in Web UI

SparkUI

SparkUI is…​FIXME

createLiveUI

Caution

FIXME

appUIAddress

Caution

FIXME

Settings

spark.ui.enabled

spark.ui.enabled (default: true) setting controls whether the web UI is started at all.

spark.ui.port

spark.ui.port (default: 4040) controls the port Web UI binds to.

If multiple SparkContexts attempt to run on the same host (it is not possible to have two or more Spark contexts on a single JVM, though), they will bind to successive ports beginning with spark.ui.port.

spark.ui.killEnabled

spark.ui.killEnabled (default: true) - whether or not you can kill stages in web UI.