Vaccum/ Merge small file command

[zhousun]

What feature are you requesting?

Currently, each write statement will create a parquet file, so if inserts are not batched, columnstore tables can have many small files (which will cause performance and S3 cost problems)

Current workaround is to run a no-op update query: UPDATE T set a=a, this will set your table to optimal state.

However, a built-in vacuum or compaction command that merges small files into larger ones would be ideal.

Why are you requesting this feature?

Optimize a table.

What is your proposed implementation for this feature?

Hook the vaccum command. Find small parquet files and then use the update/delete code path to rewrite them.

[nbiscaro]

I can grab this!

Noting it may also make sense to implement our deletion vector file compaction here when they eventually move out of heap tables.

[dpxcc]

Discussed on Slack. @nbiscaro will be working on #89 instead