Restriction of autocompaction to a time window doesn't work

[cluxter]

Hi everyone

I am using the official Docker image of CouchDB, v3.1.0.

I created a couchdb.ini file in which I put my custom CouchDB parameters. I know this file is well taken into account by CouchDB because I checked in Fauxton/Configuration and my custom settings are shown.

The configuration is default except that I added this:

[couchdb]
single_node = true

Now, I want to limit autocompaction to a time window from 04:00 am to 5:30 am, and I want this to be as strict as possible. So I added these parameters in the same file:

[smoosh.overnight_channel]
from = 04:00
to = 05:30
strict_window = true

Then I started to insert 3 millions of documents one by one in one database using ApacheBench like this:

ab -r -n 3000000 -c 1000 -p data.json -T "application/json" -A admin:password "http://localhost:5984/benchmark"

and data.json contains:

{
    "text": "Some text"
}

It works very well, I get about 10,000 inserts/s.
But very quickly an autocompaction task is run and slows everything down to about 2,000 inserts/s. I checked that in Fauxton/Active tasks/All tasks, there are clearly a couple of compaction jobs running that I never asked for. The thing is: I'm not doing all this between 4am and 5:30am and the time of my computer is right.

So it looks like the restriction to the time window doesn't work.

Am I doing something wrong or missing something?

Thanks.

Kind regards,

cluxter

[wohali]

Two things:

1. You have to set those from and to values on every channel in smoosh. You've not applied those strict times to the default 4 channels of ratio_dbs, ratio_views, slack_dbs and slack_views.
2. CouchDB determines time via the UTC (GMT) timezone. Are you sure you're not running this test when it is 04:00-05:30 UTC?

[cluxter]

Thank you Joan! It works great now! :-)

1. I didn't get that from the documentation. I thought [smoosh.overnight_channel] was kind of a special channel, but it makes sense now.

For other readers, here is what I put in my couchdb.ini file instead of the [smoosh.overnight_channel]:

[smoosh.ratio_dbs]
from = 02:00
to = 03:30
strict_window = true

[smoosh.ratio_views]
from = 02:00
to = 03:30
strict_window = true

[smoosh.slack_dbs]
from = 02:00
to = 03:30
strict_window = true

[smoosh.slack_views]
from = 02:00
to = 03:30
strict_window = true

2. I may have read somewhere one day that CouchDB was using UTC, but I definitely forgot about that. I'm in a UTC+2 timezone so even if my tests were not conducted in a period of time which was problematic, it would have bitten me one day! Thanks for this useful precision. It might be worth improving the documentation about this in 5.1.3.3 Scheduling Windows to clarify it. I will take a look at how to do this so others don't get caught :-)

[cluxter]

I just improved the documentation and created a PR here: apache/couchdb-documentation#569.

[ajitpsingh]

I want to control when the db , view and indexing job to run at UTC 2 am to 3 am so as per my current understanding this should be the setting for it. Please confirm if this is not correct. Is there any other setting that I would also have to do?

[smoosh.ratio_dbs]
from = 02:00
to = 03:30
strict_window = true

[smoosh.ratio_views]
from = 02:00
to = 03:30
strict_window = true

[smoosh.slack_dbs]
from = 02:00
to = 03:30
strict_window = true

[smoosh.slack_views]
from = 02:00
to = 03:30
strict_window = true

[smoosh.index_cleanup]
from = 02:00
to = 03:30
strict_window = true

[smoosh.upgrade_dbs]
from = 02:00
to = 03:30
strict_window = true

[smoosh.upgrade_views]
from = 02:00
to = 03:30
strict_window = true

[rnewson]

smoosh only controls automation of compaction jobs, a separate application called ken handles background indexing. Querying a stale view or search index will cause it be to updated to the current database state before the query is processed.

In future please open new tickets for new problems and not tack a message to the end of an old ticket.