Hard-Deleted table not reclaiming disk space in InfluxDB 3 Core

[sevibals]

Our goal is to permanently remove table data in order to reclaim disk space on resource-constrained edge devices. If the described behavior is expected, we are looking for the recommended way to achieve this.
Retention policies are not suitable for our schema and use case.

Steps to reproduce:
List the minimal actions needed to reproduce the behaviour.

1. Start InfluxDB 3 Core (Serverless) version influxdb-3.8-core using the Docker configuration provided below.
2. Create a database and a table.
3. Insert data into the table until noticeable disk usage is observed.
4. Delete the table via the CLI using the command:

influxdb3 delete table my_table \
  --database my_database \
  --hard-delete now \
  --host http://localhost:8181 \
  --token my_token

5. Query the database data and metadata, and observe disk usage on the host.

Expected behaviour:
The specified table and its underlying data should be immediately and permanently removed. Disk usage on the host should decrease accordingly, as the data is no longer stored.

Actual behaviour:
After executing the delete command:

• The original table (my_table) is no longer queryable.
• Disk usage on the host remains unchanged.
• A new table named my_table_ appears.
• Querying this new table returns all of the previously “deleted” data.

This behavior was observed consistently when using either filesystem-based storage or S3 object storage (MinIO).

Config:
Docker Compose Configuration used:

  influxdb-test:
    image: influxdb:3.8-core
    container_name: influxdb-test
    ports:
      - "8181:8181"
    volumes:
      - ./influxdb-data:/var/lib/influxdb3
    command:
      - influxdb3
      - serve
      - --without-auth
      - --node-id=testnode
      - --object-store=file
      - --data-dir=/var/lib/influxdb3

[hiltontj]

@sevibals thank you for opening a detailed issue. The deletion of files on disk may be delayed, but should happen eventually - are you not seeing the disk space reclaimed at all, even after some time has elapsed?

• A new table named my_table_ appears.
• Querying this new table returns all of the previously “deleted” data.

That may be a bug - on deletion, tables are renamed to free up the namespace in the catalog because the deletion job itself is asynchronous, but a table my_table should be renamed to my_table-<deletion_time>, not my_table_.

Can you provide some query results to system tables following deletion:

select * from system.tables

[sevibals]

@hiltontj Thank you for the quick response.
You are right, the new appearing table is called my_table-<deletion_time>; Markdown hid that in my original text.

I have waited at least one hour after deletion and the disk space is not reclaimed. All data of the deleted table can still be queried using my_table-<deletion_time>. Is it expected that hard deletion via the --hard-delete now flag is delayed by more than an hour? Are additional steps required to reclaim the disk space?

The system.tables table does not appear to be available in our setup, so we queried information_schema.tables instead:

influxdb3 query --database my_database "Select * from information_schema.tables" --host "http://localhost:8181" --token my_token 
                      
+---------------+--------------------+-------------------------------------------+------------+
| table_catalog | table_schema       | table_name                                | table_type |
+---------------+--------------------+-------------------------------------------+------------+
| public        | iox                | my_table-20260210T105806                  | BASE TABLE |
| public        | iox                | my_table_2                                | BASE TABLE |
| public        | system             | distinct_caches                           | BASE TABLE |
| public        | system             | influxdb_schema                           | BASE TABLE |
| public        | system             | last_caches                               | BASE TABLE |
| public        | system             | parquet_files                             | BASE TABLE |
| public        | system             | processing_engine_logs                    | BASE TABLE |
| public        | system             | processing_engine_trigger_arguments       | BASE TABLE |
| public        | system             | processing_engine_triggers                | BASE TABLE |
| public        | system             | queries                                   | BASE TABLE |
| public        | information_schema | tables                                    | VIEW       |
| public        | information_schema | views                                     | VIEW       |
| public        | information_schema | columns                                   | VIEW       |
| public        | information_schema | df_settings                               | VIEW       |
| public        | information_schema | schemata                                  | VIEW       |
| public        | information_schema | routines                                  | VIEW       |
| public        | information_schema | parameters                                | VIEW       |
+---------------+--------------------+-------------------------------------------+------------+

[hiltontj]

Is it expected that hard deletion via the --hard-delete now flag is delayed by more than an hour?

No, it shouldn't be lingering that long. Are there any ERROR or WARN entries in your logs, either near server startup or around the time that you performed the deletion?

Furthermore, you may try starting the server with DEBUG logs enabled for the deleter with the following env var:

LOG_FILTER=info,influxdb3_write::table_index_cache=debug

or via the CLI:

influxdb3 serve [...] --log-filter info,influxdb3_write::table_index_cache=debug

---

The system.tables table does not appear to be available in our setup, so we queried information_schema.tables instead

Ah, my apologies, I forgot that the system.tables table is only currently available to be queried on the _internal database, e.g.,

influxdb3 query --database _internal "select * from system.tables where database_name = '<db_name>'"

I opened #27205 to allow queries to system.tables when scoped to a particular database (as you tried, and which should be possible).