How to handle duplicate column names?

[evbo]

So another disgusting csv format someone handed me:

animal,max_height,average,max_weight,average
zebra,62,53,460,350,

The thinking was, why distinguish average when we can assume it is the average of the column that comes before it?

But this leads to errors such as:

duplicate field average

Is the cleanest way to handle this so some string replacement on the raw headers and then performing set_headers with the augmented versions?

[BurntSushi]

I think you should just be able to do set_headers without any sort of string replacement.

[evbo]

the problem is during deserialization though, because average cannot map to multiple struct fields, can it?

But if I uniquify the column names it works, I just thought to ask in case there was a built-in way to handle this.

[BurntSushi]

Did you try what I suggested? If not, why not? And if you did, what happened? It helps to share code. It tells me exactly what you're trying. Because it may be the case that something is going wrong that is incidental to the actual question you're asking.

Here's an example:

#[allow(dead_code)]
#[derive(Debug, serde::Deserialize)]
struct Row {
    animal: String,
    max_height: i64,
    average_height: i64,
    max_weight: i64,
    average_weight: i64,
}

fn main() -> anyhow::Result<()> {
    let data = "\
animal,max_height,average,max_weight,average
zebra,62,53,460,350
";

    let mut rdr = csv::Reader::from_reader(data.as_bytes());
    rdr.set_headers(csv::StringRecord::from(vec![
        "animal",
        "max_height",
        "average_height",
        "max_weight",
        "average_weight",
    ]));
    for result in rdr.deserialize().skip(1) {
        let row: Row = result?;
        println!("{row:?}");
    }
    Ok(())
}

with this Cargo.toml:

[package]
publish = false
name = "csv-duplicate-headers"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = "1.0.86"
csv = "1.3.0"
serde = { version = "1.0.203", features = ["derive"] }

[[bin]]
name = "csv-duplicate-headers"
path = "main.rs"

[profile.release]
debug = true

And its output is:

$ cargo r -q
Row { animal: "zebra", max_height: 62, average_height: 53, max_weight: 460, average_weight: 350 }

[evbo]

Ok, so it is necessary to rename them as I was suggestion, e.g. average becomes average_height or average_weight

The confusing part was I am reading these headers from the file dynamically, they aren't fixed like you have shown.

That is why I asked, because it's a fair amount of work necessary to replace the header names dynamically and you could imagine a feature where duplicate column names are automatically indexed: average_1, average_2 by the Reader to avoid collisions.

[BurntSushi]

The confusing part was I am reading these headers from the file dynamically

Yeah you didn't include this detail in your question. But the strategy is the same regardless of how you do it: you'll need to set the headers as shown. Whether it's by hard-coding the field names or generating them from some existing headers.

I don't see resolving this being the job of this library unfortunately. It's much easier to solve it on a case-by-case basis. Solving it in the general case would be a pretty annoying API to expose IMO.

[evbo]

agreed and thanks for confirming - just wanted to make sure I wasn't reinventing wheels