Skip to content

Commit 51d0550

Browse files
committed
chore: cleanup
1 parent 2f520e9 commit 51d0550

File tree

4 files changed

+8
-16
lines changed

4 files changed

+8
-16
lines changed

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ golint:
3434
.PHONY: gobuild
3535
gobuild:
3636
mkdir -p .bin/
37-
CGO_ENABLED=0 go build -a -o .bin/okdp-server main.go
37+
CGO_ENABLED=0 go build -a -o .bin/spark-web-proxy main.go
3838

3939
.PHONY: gocompile
4040
gocompile:

Readme.md

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,17 +22,11 @@ The proxy is non-intrusive and independent of any specific version of Spark Hist
2222
2323
## Installation
2424

25-
The web proxy can be deployed either as a sidecar container alongside your Spark History Server or as an independent helm chart.
25+
To deploy the spark web proxy, refer to helm chart [README](helm/spark-web-proxy/README.md) for customization options and installation guidelines.
2626

27-
1. As a sidecar container:
27+
The web proxy can also be deployed as a sidecar container alongside your existing Spark History Server. Ensure to set the property `configuration.spark.service` to `localhost`.
2828

29-
Refer to [helm/spark-history-server](https://github.com/OKDP/spark-history-server/tree/main/helm/spark-history-server) repository for guidlines and examples.
30-
31-
2. As an independent chart:
32-
33-
Refer to [README](helm/spark-web-proxy/README.md) for customization options and installation guidelines.
34-
35-
In both cases, you need to use the web proxy ingress instead of your spark history ingress.
29+
In both cases, you need to use the spark web proxy ingress instead of your spark history ingress.
3630

3731
## Spark History and spark jobs Configuration
3832

@@ -86,7 +80,7 @@ spec:
8680
8781
In a client mode, the web proxy relies on [/api/v1/applications/[app-id]/environment](https://spark.apache.org/docs/latest/monitoring.html) Spark History Rest API to get the Spark driver IP and UI port and [/api/v1/applications/[app-id]](https://spark.apache.org/docs/latest/monitoring.html) to get the application status.
8882
89-
By default, Spark does not not render the property `spark.ui.port` in the environment properties. So, you should set the property during the job submission or using a listener.
83+
By default, Spark does not render the property `spark.ui.port` in the environment properties. So, you should set the property during the job submission or using a listener.
9084

9185
Here is an example of how to set the `spark.ui.port` on a jupyter notebook:
9286

cmd/root.go

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@ func init() {
4242
viper.SetDefault("spark.history.service", "localhost")
4343
viper.SetDefault("spark.history.port", 18080)
4444

45-
viper.SetDefault("spark.ui.port", 18080)
4645
viper.SetDefault("spark.ui.proxyBase", "/sparkui")
4746
viper.SetDefault("spark.jobNamespaces", "default")
4847

helm/spark-web-proxy/Chart.yaml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,8 @@ maintainers:
1515
name: idirze
1616
url: https://github.com/mlahouar
1717
sources:
18-
- https://github.com/apache/spark
18+
- https://github.com/OKDP/spark-web-proxy
19+
- https://github.com/OKDP/spark-history-server/tree/main/helm/spark-history-server
1920
- https://github.com/OKDP/okdp-spark-auth-filter
2021
- https://github.com/OKDP/spark-images
21-
- https://github.com/OKDP/spark-history-server/tree/main/helm/spark-history-server
22-
- https://github.com/OKDP/spark-web-proxy
23-
22+
- https://github.com/apache/spark

0 commit comments

Comments
 (0)