Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Example nextjs site that imports repositories from GitHub to Splitgraph, then exports them to Seafowl, then renders a chart of stargazers #21

Open
wants to merge 36 commits into
base: generated-import-plugins
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0f64a81
Add stub `examples/nextjs-import-airbyte-github-export-seafowl/`
milesrichardson May 23, 2023
776e11f
Stub out layout and sidebar of GitHub analytics example
milesrichardson May 23, 2023
1f3b6d1
Add backend config and API routes for starting, awaiting import task
milesrichardson May 24, 2023
82db03b
Add Stepper component with ImportPanel and ExportPanel
milesrichardson May 24, 2023
0975075
Implement backend API routes `start-export-to-seafowl` and `await-exp…
milesrichardson May 25, 2023
5cd10be
Implement the components for the "Export Panel"
milesrichardson May 25, 2023
8ba4151
Move lib-backend to lib/backend
milesrichardson May 26, 2023
2dcec66
Hardcode list of "relevant" table names for ingestion from GitHub
milesrichardson May 26, 2023
65ca23e
Serialize the stepper state into the URL, so that awaiting can be res…
milesrichardson May 26, 2023
9e47b5a
Move `lib/config.ts` -> `lib/config/github-tables.ts`
milesrichardson May 26, 2023
d60cd4d
Export analytics queries to Seafowl in addition to tables
milesrichardson May 26, 2023
17d8bc9
Track completed import/exports in a meta repository on Splitgraph
milesrichardson May 29, 2023
3ba9e97
Support ?debug=1 parameter in URL of stepper to render DebugPane
milesrichardson May 29, 2023
e6c592e
Implement sidebar and stub out page for imported repository to show t…
milesrichardson May 29, 2023
613a774
Install `@observablehq/plot` in `example-nextjs-import-airbyte-github…
milesrichardson May 29, 2023
309203d
Render stargazers line chart with Observable Plot querying Seafowl
milesrichardson May 30, 2023
9b58189
Refactor Observable plot: Add `useSqlPlot` hook and make file per plot
milesrichardson Jun 1, 2023
5f4d6b5
Refactor: use better name than `acc` in reduce function
milesrichardson Jun 8, 2023
b362bc2
Add styling, bells and whistles to stepper (text, buttons, loading ba…
milesrichardson Jun 17, 2023
4454199
Refactor
milesrichardson Jun 19, 2023
cb71604
Display preview table for each loading/completed: either embed Splitg…
milesrichardson Jun 19, 2023
cc8cde1
Allow toggling between Splitgraph/Seafowl embeds (once export has com…
milesrichardson Jun 19, 2023
2b51141
Use `<a>` tag for "Import Your Repository" link, to completely reset …
milesrichardson Jun 20, 2023
f43d0e8
Add `formatTimeElapsed` function prop to `LoadingBar` for custom form…
milesrichardson Jun 20, 2023
cf97067
Productionize the export step: add lots of nice styling, bells and wh…
milesrichardson Jun 20, 2023
ecd1e43
Refactor
milesrichardson Jun 20, 2023
418ecfd
Only poll unique export taskIds, and do the polling from one hook ins…
milesrichardson Jun 21, 2023
9835599
Add workaround for failed exports of queries: create a fallback table
milesrichardson Jun 23, 2023
c260e10
When embedding Seafowl query of exported queries, select from the des…
milesrichardson Jun 23, 2023
4cc3e41
Move embedded preview components to be shared with export panel and r…
milesrichardson Jun 28, 2023
ab57659
Add `reduceRows` method to `useSqlPlot` for case where mapping isn't …
milesrichardson Jun 28, 2023
15d14e2
Add stacked bar chart of issue reactions by month with bars broken do…
milesrichardson Jun 28, 2023
06c57b9
Make repo page have three tabs: tables, queries and charts
milesrichardson Jun 28, 2023
93022b1
Rename chart to IssueReactsByMonth
milesrichardson Jun 29, 2023
afa4c07
Bump GitHub import page size to 100, start date to 2023-01-01
milesrichardson Jun 29, 2023
06d94f7
Add scatter plot of user comment length vs. lines of code
milesrichardson Jun 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move lib/config.ts -> lib/config/github-tables.ts
milesrichardson committed Jun 29, 2023
commit 9e47b5ad014e62e7346e0493d166454ba150d6c9
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ import { useStepper } from "./StepperContext";
import styles from "./ExportPanel.module.css";
import { ExportLoadingBars } from "./ExportLoadingBars";

import { relevantGitHubTableNames } from "../../lib/config";
import { relevantGitHubTableNames } from "../../lib/config/github-tables";

export const ExportPanel = () => {
const [
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@ import {
makeAuthenticatedSplitgraphDb,
claimsFromJWT,
} from "../../lib/backend/splitgraph-db";
import { relevantGitHubTableNames } from "../../lib/config";
import { relevantGitHubTableNames } from "../../lib/config/github-tables";

const GITHUB_PAT_SECRET = process.env.GITHUB_PAT_SECRET;