--float-precision Not Being Considered in qsv sqlp

[brian-mendicino]

Describe the bug
The --float-precision option in the qsv sqlp command does not appear to be applied when converting CSV files to Parquet format. Despite specifying a precision value (e.g., 16), the resulting Parquet file always uses the default precision of 6, leading to truncated decimal values.

To Reproduce

1. Run the following command:
   qsv sqlp /tmp/table.csv 'select * from table' --float-precision 16 --infer-len 0 --format parquet --output table.parquet
2. Inspect the resulting table.parquet file.
3. Observe that decimal values are truncated to 6 decimal places, regardless of the --float-precision value.

Expected behavior
The --float-precision option should ensure that floating-point numbers in the Parquet file retain the specified precision (e.g., 16 decimal places).

Screenshots/Backtrace/Sample Data
If applicable, add screenshots/backtraces/sample data to help explain your problem.

Desktop (please complete the following information):

• OS: Debian GNU/Linux 12
• qsv Version 3.1.1-mimalloc-polars-0.46.0

Additional Notes
Ideally there would be an option to retain precision, skipping truncation...

---

[jqnatividad]

Currently, the sqlp --float-precision option only applies to the CSV output format.

Looking at the Polars parquet writer - https://docs.pola.rs/api/rust/dev/polars/prelude/struct.ParquetWriter.html, it's not currently available.

Will explore setting the precision on the Dataframe and see if its preserved downstream when saving to parquet.

[brian-mendicino]

Thanks for the clarification.

In the meantime, could you confirm if there are any workarounds or alternative approaches to explicitly control the precision for Parquet output? Like, would defining a schema with specific precision for floating-point columns help?

[jqnatividad]

Your idea to define a schema beforehand may very well work @brian-mendicino

Give it a try! Just be sure you're doing it on the latest release - v3.3.0.

[rickhg12hs]

ROUND might be useful.

$ cat table.csv 
constant,value
e,2.71828182845904523536
pi,3.14159265358979323844
phi,1.61803398874989484820

$ qsv table table.csv 
constant  value
e         2.71828182845904523536
pi        3.14159265358979323844
phi       1.61803398874989484820

$ qsv sqlp table.csv 'select constant, ROUND(value, 10) from table' --format parquet --output table.parquet --quiet && qsv sqlp SKIP_INPUT "select * from read_parquet('table.parquet')" --quiet | qsv table
constant  value
e         2.7182818285
pi        3.1415926536
phi       1.6180339887

$ qsv --version
qsv 3.3.0-mimalloc-apply;fetch;foreach;geocode;Luau 0.663;prompt;python-3.12.9 (main, Feb  4 2025, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)];to;polars-0.46.0:py-1.26.0;self_update-4-4;6.12 GiB-17.02 GiB-3.13 GiB-7.65 GiB (x86_64-unknown-linux-gnu compiled with Rust 1.85) compiled