Skip to content

Commit

Permalink
Schema validate correctly in fimfic stats
Browse files Browse the repository at this point in the history
Validate the data with the schema for the API and stats JSON at the right times.
Also remove rate limit check, FIMFiction does not have rate limiting.
  • Loading branch information
SilkRose committed Mar 9, 2024
1 parent a64014c commit b6b0bc5
Showing 1 changed file with 4 additions and 8 deletions.
12 changes: 4 additions & 8 deletions fimficstats/fimfic-stats.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,6 @@ async function mane() {
return response.json();
});

// Check for rate limiting.
if (api_status === 429) {
await sleep(start_time, Date.now(), 5000);
id = id - 1;
continue;
}

// Get html of the stats page.
const stats_html = await fetch(`${stats_domain}/${id}`).then((response) => {
html_status = response.status;
Expand All @@ -84,6 +77,8 @@ async function mane() {
continue;
}

const api = api_schema.parse(api_json);

// Load the HTML with Cheerio.
const document = cheerio.load(stats_html);

Expand All @@ -101,7 +96,8 @@ async function mane() {
});

// Format the historical data into JSON.
const data = document(".layout-two-columns[data-data]").attr("data-data");
const data = document(".layout-two-columns[data-data]").attr("data-data")!;
const stats = stats_schema.parse(JSON.parse(data));

// Get the ranking and word count rankings from the HTML.
const rankings = document('h1:contains("Rankings")').next("ul").find("li");
Expand Down

0 comments on commit b6b0bc5

Please sign in to comment.