chore: update readme

Author: idirze

Readme.md

@@ -6,7 +6,7 @@
 
 spark-web-proxy acts as a reverse proxy for [Spark History Server](https://spark.apache.org/docs/latest/monitoring.html) and [Spark UI](https://spark.apache.org/docs/latest/web-ui.html). It completes [Spark History Server](https://spark.apache.org/docs/latest/monitoring.html) by seamlessly integrating live (running) Spark applications UIs. The web proxy enables real-time dynamic discovery and monitoring of running spark applications (without delay) alongside completed applications, all within your existing Spark History Server Web UI.
 
-The proxy is non-intrusive and independent of any specific version of Spark History Server or Spark. It supports all Spark application deployment modes, including Kubernetes Jobs, Spark Operator, Jupyter Spark notebooks, etc.
+The proxy is non-intrusive and independent of any specific version of Spark History Server or Spark. It supports all Spark application deployment modes, including Kubernetes Jobs, Spark Operator, notebooks (Jupyter, etc), etc.
 
 ![Spark History](docs/images/spark-history.png)
 
@@ -59,17 +59,50 @@ For more configuration properties, refer to [Spark Monitoring](https://spark.apa
 
 ## Spark jobs deployment
 
-In a cluster mode, spark by default adds the label `spark-role: driver` in the spark driver pods.
+### Cluster mode
 
-In a client mode, add the following label into your driver pods:
+In a cluster mode, `no additional configuration` is needed as spark by default adds the label `spark-role: driver` and the `spark-ui` port in the spark driver pods as shown in the following:
 
 ```yaml
-kind: ...
+apiVersion: v1
+kind: Pod
 metadata:
   labels:
     ...
     spark-role: driver
+spec:
+  containers:
+  - args:
+    - driver
+    name: spark-kubernetes-driver
+    ports:
     ...
+    - containerPort: 4040
+      name: spark-ui
+      protocol: TCP
+```
+
+### Notebooks and Client mode
+
+In a client mode, the web proxy relies on [/api/v1/applications/[app-id]/environment](https://spark.apache.org/docs/latest/monitoring.html) Spark History Rest API to get the Spark driver IP and UI port and [/api/v1/applications/[app-id]](https://spark.apache.org/docs/latest/monitoring.html) to get the application status.
+
+By default, Spark does not not render the property `spark.ui.port` in the environment properties. So, you should set the property during the job submission or using a listener.
+
+Here is an example of how to set the `spark.ui.port` on a jupyter notebook:
+
+```python
+import socket
+def find_available_port(start_port=4041, max_port=4100):
+    """Find the next available port starting from start_port."""
+    for port in range(start_port, max_port):
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+            if s.connect_ex(("localhost", port)) != 0:
+                return port
+    raise Exception(f"No available ports found in range {start_port}-{max_port}")
+```
+
+```python
+conf.set("spark.ui.port", find_available_port())
 ```
 
 ## Authentication