Merging Bulk Data Files to Create Full Dataset

[inelson2]

I am working on merging the CourtListener bulk data files (opinions, opinion clusters, dockets) to create a complete full dataset, but I am encountering challenges. Specifically, I am finding that the ids and docket numbers do not uniquely identify observations, which Is hindering the merging process. Could anyone please provide guidance on how to uniquely identify and merge cases and opinions?

Thank you very much!

[mlissner]

Can you say more about what you mean by "observations" and things that are not unique? Maybe an example?

[inelson2]

I attempted to merge opinion-clusters-2022-08-02.csv to dockets-2022-08-02.csv using
merge 1:1 date_created date_modified id using "${data}/orig/dockets.dta", force I get the following output:

    -----------------------------------------
    Not matched                     1,701,180
        from master                   701,181  (_merge==1)
        from using                    999,999  (_merge==2)

    Matched                                 0  (_merge==3)

When tagging duplicates with respect to id I get the following breakdown of duplicates:

--------------+--------------------------------------------
Valid   0     |    1697866      99.81      99.81      99.81
        1     |       3314       0.19       0.19     100.00
        Total |    1701180     100.00     100.00           

Here are the first several duplicates I find:

[mlissner]

Sorry, I'm not even sure what language or tool this is, so I'm not able to understand these results, but I'll say this: This is relational data, so you can't expect it all to flatten down to 1:1 rows. For example, multiple opinions can be associated with a single cluster. Multiple clusters can be associated with a single docket, etc.

[inelson2]

Ok thank you. I've also attempted m:1 and 1:m (cannot do m:m on STATA), but neither of those work. I just want the datasets to merge, if possible, so are there a combination of variables that would allow that? The data is only appending currently.

[mlissner]

I don't know what you mean that the data is only appending. In general, you should be able to join on id fields. I'm sorry I'm not better equiped to help with these issues.