Skip to content

Commit 78397b0

Browse files
kushalShukla-webkushalShukla-web
committed
written an instruct to create bucket-config.yml file on Readme.
Signed-off-by: Kushal Shukla <kushalshukla110@gmail.com>
1 parent 6d462bc commit 78397b0

File tree

4 files changed

+196
-16
lines changed

4 files changed

+196
-16
lines changed

‎prombench/docs/eks.md

Lines changed: 66 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,8 +107,72 @@ Run Prombench tests in [Elastic Kubernetes Service (EKS)](https://aws.amazon.com
107107
```bash
108108
make node_create
109109
```
110-
111-
3. **Deploy the Kubernetes Objects**:
110+
3. **Setting Up Benchmarking Data**
111+
112+
When setting up a benchmarking environment, it’s often useful to have pre-generated data available. This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.
113+
114+
In this setup, you have two choices:
115+
116+
Here’s how each option works:
117+
- **Option 1: Download data from object storage**
118+
119+
To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
120+
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`
121+
122+
- **Option 2: Skip downloading data**
123+
124+
If you don’t Want to Download data create an empty secret like this -
125+
126+
```yaml
127+
# Empty Secret to Skip Downloading Data
128+
apiVersion: v1
129+
kind: Secret
130+
metadata:
131+
name: bucket-secret
132+
namespace: prombench-{{ .PR_NUMBER }}
133+
type: Opaque
134+
stringData:
135+
object-config.yml:
136+
```
137+
138+
Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).
139+
140+
> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.
141+
142+
4. **Downloading Directory configuration**
143+
144+
PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
145+
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .
146+
147+
- **Option 1: To Download Data from a Specific Directory**
148+
149+
Create a ConfigMap with the following structure:
150+
```yaml
151+
apiVersion: v1
152+
kind: ConfigMap
153+
metadata:
154+
name: blocksync-config
155+
namespace: prombench-{{ .PR_NUMBER }}
156+
data:
157+
bucket-config.yml: |
158+
path: your-directory-name
159+
minTime: block-starting-time
160+
maxTime: block-ending-time
161+
```
162+
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
163+
- **Option 2: To Skip Data Download**
164+
165+
If you do not want to download data, create an empty ConfigMap with the same name:
166+
```yaml
167+
apiVersion: v1
168+
kind: ConfigMap
169+
metadata:
170+
name: blocksync-config
171+
namespace: prombench-{{ .PR_NUMBER }}
172+
data:
173+
```
174+
175+
5. **Deploy the Kubernetes Objects**:
112176

113177
```bash
114178
make resource_apply

‎prombench/docs/gke.md

Lines changed: 65 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,71 @@ Run Prombench tests in [Google Kubernetes Engine (GKE)](https://cloud.google.com
9696
```bash
9797
make node_create
9898
```
99-
100-
3. **Deploy the Kubernetes Objects**:
99+
3. **Setting Up Benchmarking Data**
100+
When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.
101+
102+
In this setup, you have two choices:
103+
104+
Here’s how each option works:
105+
- **Option 1: Download data from object storage**
106+
107+
To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
108+
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`
109+
110+
- **Option 2: Skip downloading data**
111+
112+
If you don’t Want to Download data create an empty secret like this -
113+
114+
```yaml
115+
# Empty Secret to Skip Downloading Data
116+
apiVersion: v1
117+
kind: Secret
118+
metadata:
119+
name: bucket-secret
120+
namespace: prombench-{{ .PR_NUMBER }}
121+
type: Opaque
122+
stringData:
123+
object-config.yml:
124+
```
125+
126+
Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).
127+
128+
> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.
129+
130+
4. **Downloading Directory configuration**
131+
132+
PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
133+
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .
134+
135+
- **Option 1: To Download Data from a Specific Directory**
136+
137+
Create a ConfigMap with the following structure:
138+
```yaml
139+
apiVersion: v1
140+
kind: ConfigMap
141+
metadata:
142+
name: blocksync-config
143+
namespace: prombench-{{ .PR_NUMBER }}
144+
data:
145+
bucket-config.yml: |
146+
path: your-directory-name
147+
minTime: block-starting-time
148+
maxTime: block-ending-time
149+
```
150+
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
151+
- **Option 2: To Skip Data Download**
152+
153+
If you do not want to download data, create an empty ConfigMap with the same name:
154+
```yaml
155+
apiVersion: v1
156+
kind: ConfigMap
157+
metadata:
158+
name: blocksync-config
159+
namespace: prombench-{{ .PR_NUMBER }}
160+
data:
161+
```
162+
163+
5. **Deploy the Kubernetes Objects**:
101164

102165
```bash
103166
make resource_apply

‎prombench/docs/kind.md

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,71 @@ If used with the GitHub integration:
114114
export PR_NUMBER=<PR to benchmark against the selected $RELEASE>
115115
```
116116

117-
2. Deploy the Kubernetes objects:
117+
2. Setting Up Benchmarking Data
118+
When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.
119+
120+
In this setup, you have two choices:
121+
122+
Here’s how each option works:
123+
- **Option 1: Download data from object storage**
124+
125+
To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
126+
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`
127+
128+
- **Option 2: Skip downloading data**
129+
130+
If you don’t Want to Download data create an empty secret like this -
131+
132+
```yaml
133+
# Empty Secret to Skip Downloading Data
134+
apiVersion: v1
135+
kind: Secret
136+
metadata:
137+
name: bucket-secret
138+
namespace: prombench-{{ .PR_NUMBER }}
139+
type: Opaque
140+
stringData:
141+
object-config.yml:
142+
```
143+
144+
Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).
145+
146+
> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.
147+
148+
3. Downloading Directory configuration
149+
150+
PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
151+
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .
152+
153+
- **Option 1: To Download Data from a Specific Directory**
154+
155+
Create a ConfigMap with the following structure:
156+
```yaml
157+
apiVersion: v1
158+
kind: ConfigMap
159+
metadata:
160+
name: blocksync-config
161+
namespace: prombench-{{ .PR_NUMBER }}
162+
data:
163+
bucket-config.yml: |
164+
path: your-directory-name
165+
minTime: block-starting-time
166+
maxTime: block-ending-time
167+
```
168+
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
169+
- **Option 2: To Skip Data Download**
170+
171+
If you do not want to download data, create an empty ConfigMap with the same name:
172+
```yaml
173+
apiVersion: v1
174+
kind: ConfigMap
175+
metadata:
176+
name: blocksync-config
177+
namespace: prombench-{{ .PR_NUMBER }}
178+
data:
179+
```
180+
181+
4. Deploy the Kubernetes objects:
118182
> **_Note:_** If you encounter a `too many files open` error caused by promtail, increase the default value of `/proc/sys/fs/inotify/max_user_instances` from 128 to 512:
119183
> ```bash
120184
> sudo sysctl fs.inotify.max_user_instances=512

‎prombench/manifests/prombench/benchmark/3b_prometheus-test-pr_deployment.yaml

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,3 @@
1-
apiVersion: v1
2-
kind: ConfigMap
3-
metadata:
4-
name: blocksync-config
5-
namespace: prombench-{{ .PR_NUMBER }}
6-
data:
7-
bucket-config.yml: |
8-
path: gendata
9-
minTime: 1729694072541
10-
maxTime: 1729706400000
11-
---
121
apiVersion: apps/v1
132
kind: Deployment
143
metadata:

0 commit comments

Comments
 (0)