[Bug]: rpc couch_db:get_db_info/1 {badmatch,{error,enoent}} on database after compaction

[Gidsik]

Version

3.5.1

Describe the problem you're encountering

I tried to run one small single node of couchdb for personal use in docker desktop for obsidian selfhosted live sync.
During Initialization of obsidian vault with 2k .md files (~2.2mb size of obisidian vault) and uploading it to database (~6.6mb database size in fauxton ui) I encountered an http 500 error that prevents me from getting synced.
Error get fixed with reboot of couchdb container.
Noticed that GET /db queries fails with error 500, but GET\PUT /db/_local/<somestr> queries finishes with 200 and 201 and database size still grows, so uploading files is ok, but getting it back doesnt.
Noticed that this error begin to pop after couchdb decides to run auto-compaction (because of db grows??).

Tried to manually run compaction on new database to confirm and yes - after compaction database became unavailable.

Expected Behaviour

database works stable as intended without errors.

Steps to Reproduce

1. run couchdb:latest docker container with pretty much default configuration
2. create new database (leave it empty or add some data - doesn't matter)
3. run curl -H "Content-Type: application/json" -X POST http://<name>:<pass>@localhost:5984/<db-name>/_compact
4. run curl http://<name>:<pass>@localhost:5984/<db-name>

it returns {"error":"error","reason":"{badmatch,{error,enoent}}"}

if compaction is still running it will return normal json as intended with "compact_running":true value)

Your Environment

I'm running couchdb as just single node in docker.desktop under win11 (with wsl)
image: couchdb:latest
version 3.5.1

Additional Context

here is debug logs from reproduction with empty test database

curl http://gidsik:<pass>@localhost:5984/test

[notice] 2026-03-03T00:50:17.729237Z nonode@nohost <0.643.0> 0884c5319c localhost:5984 172.18.0.1 gidsik GET /test 200 ok 156

curl -H "Content-Type: application/json" -X POST http://gidsik:<pass>@localhost:5984/test/_compact

[debug] 2026-03-03T00:53:19.917826Z nonode@nohost <0.925.0> -------- Starting compaction for db "shards/80000000-ffffffff/test.1772498476" at 0
[debug] 2026-03-03T00:53:19.917847Z nonode@nohost <0.919.0> -------- Starting compaction for db "shards/00000000-7fffffff/test.1772498476" at 0
[notice] 2026-03-03T00:53:19.917873Z nonode@nohost <0.1212.0> fccfbe44c6 localhost:5984 172.18.0.1 gidsik POST /test/_compact 202 ok 1
[debug] 2026-03-03T00:53:19.920434Z nonode@nohost <0.1252.0> -------- Compaction process spawned for db "shards/00000000-7fffffff/test.1772498476"
[debug] 2026-03-03T00:53:19.920432Z nonode@nohost <0.1251.0> -------- Compaction process spawned for db "shards/80000000-ffffffff/test.1772498476"
[debug] 2026-03-03T00:53:20.185521Z nonode@nohost <0.242.0> -------- New task status for <0.1252.0>: [{changes_done,0},{database,<<"shards/00000000-7fffffff/test.1772498476">>},{phase,document_copy},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]
[debug] 2026-03-03T00:53:20.205917Z nonode@nohost <0.242.0> -------- New task status for <0.1251.0>: [{changes_done,0},{database,<<"shards/80000000-ffffffff/test.1772498476">>},{phase,document_copy},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]
[debug] 2026-03-03T00:53:20.361294Z nonode@nohost <0.242.0> -------- New task status for <0.1252.0>: [{changes_done,0},{database,<<"shards/00000000-7fffffff/test.1772498476">>},{phase,docid_sort},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]
[debug] 2026-03-03T00:53:20.383774Z nonode@nohost <0.242.0> -------- New task status for <0.1251.0>: [{changes_done,0},{database,<<"shards/80000000-ffffffff/test.1772498476">>},{phase,docid_sort},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]
[debug] 2026-03-03T00:53:20.461284Z nonode@nohost <0.242.0> -------- New task status for <0.1252.0>: [{changes_done,0},{database,<<"shards/00000000-7fffffff/test.1772498476">>},{phase,docid_copy},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]
[debug] 2026-03-03T00:53:20.490932Z nonode@nohost <0.242.0> -------- New task status for <0.1251.0>: [{changes_done,0},{database,<<"shards/80000000-ffffffff/test.1772498476">>},{phase,docid_copy},{progress,0},{started_on,1772499200},{total_changes,0},{type,database_compaction},{updated_on,1772499200}]

curl http://gidsik:<pass>@localhost:5984/test

[error] 2026-03-03T00:53:22.528529Z nonode@nohost <0.1317.0> aaa3bb5814 rpc couch_db:get_db_info/1 {badmatch,{error,enoent}} [{couch_bt_engine,get_size_info,1,[{file,"src/couch_bt_engine.erl"},{line,264}]},{couch_db,get_db_info,1,[{file,"src/couch_db.erl"},{line,625}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,376}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,141}]}]
[error] 2026-03-03T00:53:22.528553Z nonode@nohost <0.1316.0> aaa3bb5814 rpc couch_db:get_db_info/1 {badmatch,{error,enoent}} [{couch_bt_engine,get_size_info,1,[{file,"src/couch_bt_engine.erl"},{line,264}]},{couch_db,get_db_info,1,[{file,"src/couch_db.erl"},{line,625}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,376}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,141}]}]
[notice] 2026-03-03T00:53:22.528732Z nonode@nohost <0.1278.0> aaa3bb5814 localhost:5984 172.18.0.1 gidsik GET /test 500 ok 2

[rnewson]

Can't reproduce this (tested with image 8f29a3517318).

Perhaps there is something specific to your environment? It seems maybe files (or volumes) are disappearing outside of couchdb's control.

[Gidsik]

Yes it seems like volumes part is crucial...
But I am sure that both files and volumes stays intact on host part, I run setup with different services inside containers with their data mounted to host filesystem and there was no issue.
I was able to reproduce bug running all steps on other physical disk with running Process Monitor in background to check if other processes interfere with couchdb files.
There was windows defender, but i disabled it and result was the same.
Only com.docker.backend.exe was accessing ./data folder and only DllHost.exe (so couchdb container) was accessing subfolders and files inside ./data folder.

More info about my setup:

Windows 11 Pro 25H2, OS build 26200.7462
WSL version: 2.6.3.0
Kernel version: 6.6.87.2-1
Docker version 29.1.3, build f52814d

tried also with image: couchdb:3.5.0

I'll be able to try to reproduce this on different physical machine with similar setup within a week

With binded host:container volumes:

all actions from temp folder on different physical disk for testing purpose

F:/.temp/compose.yaml

name: test-couchdb

services:
  couchdb:
    image: couchdb:latest
    container_name: couchdb    
    environment:
      - COUCHDB_USER=admin
      - COUCHDB_PASSWORD=admin
    restart: always
    volumes:
      - ${PWD}/data:/opt/couchdb/data
      - ${PWD}/etc:/opt/couchdb/etc/local.d
    ports:
      - 5984:5984

Clean setup powershell history:

PS F:\.temp> tree /F .\
Folder PATH listing
Volume serial number is <redacted>
F:\.TEMP
    docker-compose.yml

PS F:\.temp> docker-compose up -d
[+] Running 2/2
 ✔ Network test-couchdb_default  Created                                                                                                                                                                              0.3s
 ✔ Container couchdb             Started    

PS F:\.temp> curl -X PUT http://admin:admin@localhost:5984/test-db
{"ok":true}

PS F:\.temp> tree /F .\
Folder PATH listing
Volume serial number is <redacted>
F:\.TEMP
│   docker-compose.yml
│
├───data
│   │   _dbs.couch
│   │   _nodes.couch
│   │
│   ├───.delete
│   └───shards
│       ├───00000000-7fffffff
│       │       test-db.1772570414.couch
│       │
│       └───80000000-ffffffff
│               test-db.1772570414.couch
│
└───etc
        docker.ini

PS F:\.temp> curl -H "Content-Type: application/json" -X POST http://admin:admin@localhost:15984/test-db/_compact
{"ok":true}

PS F:\.temp> curl http://admin:admin@localhost:5984/test-db
{"error":"error","reason":"{badmatch,{error,enoent}}"}

Without volumes, completely inside container

Works fine indeed, but in this case there is no way to preserve data reliably except setting automated backups from within container to host over some file transfer protocol...

F:/.temp/compose.yaml

name: test-couchdb

services:
  couchdb:
    image: couchdb:latest
    container_name: couchdb    
    environment:
      - COUCHDB_USER=admin
      - COUCHDB_PASSWORD=admin
    restart: always
    ports:
      - 5984:5984

Clean setup powershell history:

PS F:\.temp> docker-compose up -d
[+] Running 2/2
 ✔ Network test-couchdb_default  Created                                                                                                                                                                              0.3s
 ✔ Container couchdb             Started                                                                                                                                                                              0.7s

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"error":"not_found","reason":"Database does not exist."}

PS F:\.temp> curl -X PUT http://admin:admin@localhost:15984/test-db
{"ok":true}

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"instance_start_time":"1772570563","db_name":"test-db","purge_seq":"0-g1AAAABPeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyEhlwqEtkSKqHKMgCAIT2GV4","update_seq":"0-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyMpgTGXKBAuzmxoZJSamp6HpwmJLIkFSPot0oyTTRzDIJXXEWABp_Kt4","sizes":{"file":16692,"external":0,"active":0},"props":{},"doc_del_count":0,"doc_count":0,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1}}

PS F:\.temp> curl -H "Content-Type: application/json" -X POST http://admin:admin@localhost:15984/test-db/_compact
{"ok":true}

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"instance_start_time":"1772570563","db_name":"test-db","purge_seq":"0-g1AAAABPeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyEhlwqEtkSKqHKMgCAIT2GV4","update_seq":"0-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyMpgTGXKBAuzmxoZJSamp6HpwmJLIkFSPot0oyTTRzDIJXXEWABp_Kt4","sizes":{"file":33082,"external":0,"active":0},"props":{},"doc_del_count":0,"doc_count":0,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1}}

edited compose.yaml to bring back volumes within same setup

PS F:\.temp> docker-compose up -d
[+] Running 1/1
 ✔ Container couchdb  Started                                                                                                                                                                                         2.3s

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"error":"not_found","reason":"Database does not exist."}

PS F:\.temp> curl -X PUT http://admin:admin@localhost:15984/test-db
{"ok":true}

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"instance_start_time":"1772570647","db_name":"test-db","purge_seq":"0-g1AAAABPeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyEhlwqEtkSKqHKMgCAIT2GV4","update_seq":"0-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCeexAEmGBiD1HwiyMpgTGXKBAuyGScaWFqam6HpwmJLIkFSPot3S0NAsMdUcXXEWAOs0Kik","sizes":{"file":16692,"external":0,"active":0},"props":{},"doc_del_count":0,"doc_count":0,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1}}

PS F:\.temp> curl -H "Content-Type: application/json" -X POST http://admin:admin@localhost:15984/test-db/_compact
{"ok":true}

PS F:\.temp> curl http://admin:admin@localhost:15984/test-db
{"error":"error","reason":"{badmatch,{error,enoent}}"}

[rnewson]

This seems a docker-specific issue and perhaps specific to your configuration or environment, and not a couchdb issue as such? The error you shared does not occur generally and we've much experience with couchdb in a lot of different environments.

[big-r81]

Another option is to use our provided Windows binaries and test this outside of Docker.

[rnewson]

we're now pondering if this is (indirectly) related to #4459

[Gidsik]

@rnewson Thank you for looking into this.
I was able to reproduce the problem on another device, with the same Windows 11 + Docker Desktop on wsl2 setup.
Also I asked a friend of mine to install wsl + docker desktop on his win10 machine and run commands to reproduce issue and it is indeed reproducible.

I respect your experience and understand that you, as developers, have worked with various configurations and environments!
But seems like this is real and reproducible issue with couchdb setup in Docker Desktop specifically (only wsl2 backend or hyper-v backend too???). I think regular unix docker works fine otherwise you would have known and fix this a long time ago.

I understand that docker desktop is not a priority as it is not a common configuration and is not intended for production or large customers and enterprises, as most of them are using either unix systems and containers/clusters with k8s etc., or windows binaries with windows server.

But I hope that couchdb team will consider to add some kind of fallback over this specific error so database wont became inaccessible or at least add some note about this issue in documentation and provide advice\configuration to avoid such error for users who wants to setup couchdb in docker desktop.

I managed to mitigate the issue in my setup by setting smoosh.db_channels = ratio_dbs and smoosh.ratio_dbs.min_priority = 5.0 with very thin smoosh.overnight_channel basically preventing compaction from triggering.

[jonasplaum]

It's most likely not a good idea to keep the files on windows while running CouchDB (or any DB) in WSL. If possible, you should keep the shard-files inside WSL to make sure that file operations work as expected in POSIX / Linux.

Before compaction:

root@cd476862bf19:/# lsof -p 7 | grep shards
beam.smp   7 couchdb   27u      REG               0,63     12480 1125899907347446 /opt/couchdb/data/shards/00000000-7fffffff/_users.1773001834.couch
beam.smp   7 couchdb   29u      REG               0,63      8346 1125899907347447 /opt/couchdb/data/shards/80000000-ffffffff/_users.1773001834.couch
beam.smp   7 couchdb   36u      REG               0,63 206168265 2533274790900723 /opt/couchdb/data/shards/00000000-7fffffff/test-db.1773001837.couch
beam.smp   7 couchdb   37u      REG               0,63     16541 7881299348271625 /opt/couchdb/data/shards/80000000-ffffffff/test-db.1773001837.couch

While compaction is running:

root@cd476862bf19:/# lsof -p 7 | grep shards
beam.smp   7 couchdb   27u      REG               0,63     12480  1125899907347446 /opt/couchdb/data/shards/00000000-7fffffff/_users.1773001834.couch
beam.smp   7 couchdb   29u      REG               0,63      8346  1125899907347447 /opt/couchdb/data/shards/80000000-ffffffff/_users.1773001834.couch
beam.smp   7 couchdb   36u      REG               0,63 206168265  2533274790900723 /opt/couchdb/data/shards/00000000-7fffffff/test-db.1773001837.couch
beam.smp   7 couchdb   38u      REG               0,63  52323902  3377699721032703 /opt/couchdb/data/shards/00000000-7fffffff/test-db.1773001837.couch.compact.data
beam.smp   7 couchdb   39u      REG               0,63       165  2533274790900741 /opt/couchdb/data/shards/00000000-7fffffff/test-db.1773001837.couch.compact.meta
beam.smp   7 couchdb   40u      REG               0,63     16541 16325548649388895 /opt/couchdb/data/shards/80000000-ffffffff/test-db.1773001837.couch

After the compaction:

root@cd476862bf19:/# lsof -p 7 | grep shards
beam.smp   7 couchdb   27u      REG               0,63    12480  1125899907347446 /opt/couchdb/data/shards/00000000-7fffffff/_users.1773001834.couch
beam.smp   7 couchdb   29u      REG               0,63     8346  1125899907347447 /opt/couchdb/data/shards/80000000-ffffffff/_users.1773001834.couch
beam.smp   7 couchdb   38u  unknown                                               /opt/couchdb/data/shards/00000000-7fffffff/test-db.1773001837.couch (stat: No such file or directory)
beam.smp   7 couchdb   40u      REG               0,63    16541 16325548649388895 /opt/couchdb/data/shards/80000000-ffffffff/test-db.1773001837.couch

The file handle for the compacted shard is broken. After restarting CouchDB (and after reopening the db), the database is available again and compacted successfully. This shows that the swap does not work because of non-atomic filesystem operations under Windows / NTFS.

[rnewson]

Erlang compensates for Windows' non-atomic swap, but perhaps what's happening here is that Erlang is not doing that, as the container is running Linux, but the filesystem has Windows semantics. That's not a situation I think couchdb (or erlang) should have to handle.

I also note that compaction is not optional, so I agree with @jonasplaum that the fix is to move the shard files inside. The erlang vm needs to know which OS the filesystem it is reading/writing is on in order to do the right thing.