Can't pull up cases in the API using recap-query

[walinchus]

I'm looking to see how many cases are filed in each bankruptcy court per day. I messed around with the API for a bit and read the docs. And it seemed to be working for me to get opinions. But of course few cases ever go to opinion. I just want to see the metadata for each case filing, not any of the documents from it.

So here's an example of trying to pull up all the cases filed on January 5, 2023 in Connecticut bankruptcy court:

Steps to reproduce in R:

# Construct the API request URL and parameters
base_url <- "https://www.courtlistener.com/api/rest/v3/recap-query/"
query_params <- list(
  court = "ctb",                # Connecticut Bankruptcy Court
  date_filed = "2023-01-05",    # January 5, 2023
  format = "json"               # Response format: JSON
)

# Set your desired headers
headers <- c(
  "Authorization" = paste("Token",Sys.getenv("api_key")),  # Replace with your actual token
  "User-Agent" = "Lucia Walinchus"                    # Replace with your app's user agent
)

# Make the API request with headers
response <- GET(url = base_url, query = query_params, add_headers(headers))


api_data <- content(response, "text", encoding = "UTF-8")
parsed_data <- fromJSON(api_data)

data <- as.data.frame(parsed_data$results)`

This yields the html for this error: "Hmmm, something didn't work, sorry :/
A funny image of our mascot, Flip, with lightning bolts coming out of his head and smashing a computer instead of a dollar sign. On the computer is a fractal image of him smashing a computer, which contains him, and so forth.
This usually happens when we are doing maintenance or our server is overloaded.
Usually things are fixed within a few minutes."

Am I missing something here? Does anyone have code to pull just one day's worth of case names or case numbers or something? Thanks so much for your help.

[johnhawkinson]

Ultimately, I suspect the RECAP database in Courtlistener is not going to be sufficiently comprehensive for this purpose.
But there is reason to think I am wrong (see below).

Generally speaking

Primarily, cases are contributed when users upload them using the RECAP extension, and that is also supplemented by the courts' RSS feeds.

Many if not most cases are not uploaded by users, because there are a lot of cases that are not terribly interesting to people who use the RECAP extension.

Many courts do have RSS feeds and some of them (such as ecf.ctd) do indeed cover all filing events, but the result is that the entries for those cases are created in the RECAP corpus, but that does not include metadata like the date filed.
Err, I guess there has been some onetime scraping of the iquery data that PACER does not bill for? I forget exactly.

But looking at https://www.courtlistener.com/?q=&type=r&order_by=score%20desc&docket_number=23-30004%20&court=ctd%20ctb , the search alleges that 23-30004 was Date Filed Jan. 4, 2023. But the docket page, https://www.courtlistener.com/docket/66701592/alexis-hernandez/ says Date Filed Jan 5., 2023. CMECF says Jan. 5.

If I wanted this information, I would probably try to scrape it from CMECF directly, although there are a lot of Gotchas. For instance, in Connecticut, according to the District Court, there are three office codes https://ecf.ctd.uscourts.gov/cgi-bin/CourtInfo.pl :

Court Offices
Code	Name
2	Hartford
3	New Haven
5	Bridgeport

And Conn. Bk. court uses the first digit of the case number to indicate the office code, in addition to the office code itself.
So, e.g., 23-bk-20001 is the first case in Hartford, 23-bk-30001 is the first case in New Haven, etc. So finding all the cases from Jan. 5 (there are not many!) by searching the sequence space requires searching three seperate sequence spaces.

I don't do enough BK to know if all BK courts use this convention (but based on the fact that only a handful of the district courts do something similar, I would tend to doubt it), but the point is that it is tricky and every district has its own way of doing things.

API

I am the worst possible person to address questions of the API, but you are using the recap-query endpoint, and that endpoint, per the API docs, that endpoint is only "[t]o check if a document is in RECAP," i.e. by PACER document id number. That's not what you want to use.

I'd recommend starting with the HTML interface first. If you'd tried https://www.courtlistener.com/api/rest/v3/recap-query/? you'd have gotten back:

HTTP 400 Bad Request
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    "pacer_doc_id is a required filter."
]

which definitely hints at issues.

But my recommendations here are not worth the paper they are printed on.

[mlissner]

These are all great points, thanks John. Distilling them a bit you wind up with:

1. We won't have all the new cases each day, because our sources are not comprehensive.
2. The recap-query endpoint isn't a great one to use.

I think I'd want to know a bit about what you're hoping to accomplish. If it's really about being comprehensive, you might do better by using the PACER Case Locator (though it is not real time), or crawling PACER directly (though John correctly points out that that can be difficult).

Finally, I'd also add that if you really wanted to do this, another solution could be for us to actually make sure that our sources are comprehensive. We don't do that currently because we just haven't had the time to implement it, but it's a good thing to get done if we had somebody that needed it and was willing to help make it happen via a contribution.

If that's the case, probably the next best step is for us to chat so I can understand your needs and we can hash things out.

[johnhawkinson]

But on the other hand, Mike, for the district in question (D. Conn), our sources are comprehensive. So it would be nice to show Lucia the API query that returns all PACER cases in ctb filed on a given day.

[mlissner]

Right! For that, I'd use the docket endpoint and do something like:

https://www.courtlistener.com/api/rest/v3/dockets/?court=ctb&date_filed=2023-08-25

That should do it, I think.

[walinchus]

Thank you so much for this detailed and prompt reply. Is there a way to tell which courts are comprehensive and which are not?

[johnhawkinson]

Is there a way to tell which courts are comprehensive and which are not?

The CourtInfo page for each court, referenced above, indicates the alleged comprehensiveness of the RSS feed, e.g. https://ecf.ctb.uscourts.gov/cgi-bin/CourtInfo.pl:

RSS Feed:
Docket entries of type: all

Or if you look at the top of the RSS feed itself, https://ecf.ctb.uscourts.gov/cgi-bin/rss_outside.pl:

District of Connecticut - Recent Entries
Last updated: Friday, August 25, 2023 10:20:47 PM
Docket entries of type: all

Aside from EDNY's district court, they might even be accurate!

[mlissner]

That works well. We also have this information gathered on CourtListener (it's updated constantly): https://www.courtlistener.com/help/alerts/#coverage-gaps

[walinchus]

Thank you so much this is extremely helpful. And thank you both so much for doing this in general as this is an extremely important public service!