Scheduling queries of a collector

[majordhote]

Hello everybody,

Is there a way to schedule some collectors with a specific time execution instead of an interval ?

For example, for some technical reasons, I sometimes needs to execute a collector every 24 hours, but at 8 am ever day.

I have tried with min_interval: 1d, it works if I start the service (or docker in my case) at 8 am, but if I need to restart it, the execution is delayed in time.

Regards,

Nicolas

[burningalchemist]

Hi @majordhote,

sql_exporter doesn't scrape targets on its own, the scrapes are controlled and fully performed by a Prometheus instance. In other words, if Prometheus doesn't hit the /metrics endpoint of sql_exporter, nothing is scraped.

What can be done here (an idea):

• There is an option in sql_exporter to organise jobs (can be multi-target if needed) and assign collectors to them. This way if you hit /metrics?jobs[]=myjob1, only collectors against the specified targets will be scraped. If you just hit /metrics, all jobs are scraped. Please check the readme for the Multiple database connections section. You can use the same target across multiple jobs. Here is the reference on job filters: feat: scrape selected jobs via query params #433

The problem still remains as Prometheus doesn't support any scheduled scrapes as well, only intervals (every N seconds/minutes/hours). So what can be done next is (which sounds a bit like a workaround but might worth a try):

1. Given that with the jobs query parameter we can hit particular collector/target combination, we can take this endpoint [1];
2. Configure file-based scrape discovery mechanism (https://prometheus.io/docs/guides/file-sd/) to discover endpoints from the file to scrape;
3. Set up a cron schedule in the operating system that populates the file with the endpoint [1] at 8am and removes it a few minutes later;
4. Prometheus will track the file content, and once it's updated, it should perform the scrape, then stop it as the file updated again.

Have a look and give it a try. If you need some assistance or advice, please let me know. 👍

[majordhote]

Hi,

Thanks for your quick answer. We need to provide some changes on our current configuration to try with the idea you have submitted.

We will keep you updated.

Regards,