-
Notifications
You must be signed in to change notification settings - Fork 106
Add tenders finder to TinyFish-cookbook #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
KrishnaAgarwal7531
wants to merge
7
commits into
tinyfish-io:main
Choose a base branch
from
KrishnaAgarwal7531:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
b3c21c2
Create summer school finding tool
KrishnaAgarwal7531 1b42b39
Add tenders finder
KrishnaAgarwal7531 5512355
Add sumer school finder
KrishnaAgarwal7531 de4d936
Merge branch 'tinyfish-io:main' into main
KrishnaAgarwal7531 00b5777
Merge branch 'tinyfish-io:main' into main
KrishnaAgarwal7531 75ab10c
Add tutors finder
KrishnaAgarwal7531 abf7c85
Add tutor finder
KrishnaAgarwal7531 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| VITE_SUPABASE_PROJECT_ID="dksfgbuuciwhicmpdkys" | ||
| VITE_SUPABASE_PUBLISHABLE_KEY="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6ImRrc2ZnYnV1Y2l3aGljbXBka3lzIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NjkyNTE5NDIsImV4cCI6MjA4NDgyNzk0Mn0.vgKY9Gc5qtZOP5ZZOzGvKihqLgWXrwMSbTYYpSCFpEg" | ||
| VITE_SUPABASE_URL="https://dksfgbuuciwhicmpdkys.supabase.co" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Logs | ||
| logs | ||
| *.log | ||
| npm-debug.log* | ||
| yarn-debug.log* | ||
| yarn-error.log* | ||
| pnpm-debug.log* | ||
| lerna-debug.log* | ||
|
|
||
| node_modules | ||
| dist | ||
| dist-ssr | ||
| *.local | ||
|
|
||
| # Editor directories and files | ||
| .vscode/* | ||
| !.vscode/extensions.json | ||
| .idea | ||
| .DS_Store | ||
| *.suo | ||
| *.ntvs* | ||
| *.njsproj | ||
| *.sln | ||
| *.sw? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| # Project Title - Government Tender Finder for Singapore | ||
|
|
||
| **Live Link**: https://tender-scout-singapore.lovable.app | ||
|
|
||
|
|
||
| ## What This Project Is - | ||
| This project is an AI-powered summer school discovery and comparison tool that automatically finds, scans, and extracts information from official summer school websites worldwide. | ||
| Instead of relying on outdated lists or manual searches, the app pulls live data directly from source websites and returns it in a clean, structured format. | ||
|
|
||
| ## What This Project Is - | ||
|
|
||
| **AI-based Link Discovery** | ||
|
|
||
| The system first uses an AI layer to identify and curate relevant summer school websites based on the user’s query (region, subject, age group, duration, etc.). | ||
|
|
||
| **Automated Web Browsing with TinyFish** | ||
| The curated links are passed to the TinyFish Web Agent API, which launches multiple browser agents in parallel to: | ||
|
|
||
| 1) Navigate dynamic pages | ||
|
|
||
| 2) Handle real websites (no static scraping) | ||
|
|
||
| 3) Extract structured details such as program name, location, dates, eligibility, fees, and deadlines | ||
|
|
||
| **Real-Time Streaming & Aggregation** | ||
| Progress updates are streamed via SSE, and extracted results are normalised into a unified JSON format for easy comparison. | ||
|
|
||
| ## What to Expect | ||
|
|
||
| 1) Live, up-to-date data pulled directly from official websites | ||
|
|
||
| 2) Parallel web scanning for fast results | ||
|
|
||
| 3) Real-time status updates during execution | ||
|
|
||
| 4) Structured, comparable output (JSON) | ||
|
|
||
|
|
||
| **Demo Video** - https://drive.google.com/file/d/1GXZhJOjiVUP5XcGvTAvRGcYhTWoKXlsE/view?usp=sharing | ||
|
|
||
|
|
||
| ## Code snippet - | ||
| ```bash | ||
| const response = await fetch("https://mino.ai/v1/automation/run-sse", { | ||
| method: "POST", | ||
| headers: { | ||
| "Content-Type": "application/json", | ||
| "X-API-Key": "sk-mino-YOUR_API_KEY", | ||
| }, | ||
| body: JSON.stringify({ | ||
| url: "https://www.gebiz.gov.sg", | ||
| goal: "Extract the latest open government tenders. Return JSON with tenderTitle, agency, tenderID, procurementCategory, submissionDeadline, eligibilityCriteria, estimatedValue, tenderStatus, and tenderLink.", | ||
| browser_profile: "lite", | ||
| }), | ||
| }); | ||
|
|
||
| const reader = response.body!.getReader(); | ||
| const decoder = new TextDecoder(); | ||
|
|
||
| while (true) { | ||
| const { done, value } = await reader.read(); | ||
| if (done) break; | ||
|
|
||
| const chunk = decoder.decode(value); | ||
| for (const line of chunk.split("\n")) { | ||
| if (line.startsWith("data: ")) { | ||
| const data = JSON.parse(line.slice(6)); | ||
|
|
||
| // Live browser view | ||
| if (data.streamingUrl) { | ||
| console.log("Live view:", data.streamingUrl); | ||
| } | ||
|
|
||
| // Final structured output | ||
| if (data.type === "COMPLETE" && data.resultJson) { | ||
| console.log("Extracted tenders:", data.resultJson); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| ## Tech Stack | ||
| **Next.js (TypeScript)** | ||
|
|
||
| **Mino API** | ||
|
|
||
| **AI** | ||
|
|
||
| ## Architecture Diagram | ||
| ```mermaid | ||
| flowchart TB | ||
|
|
||
| %% ======================= | ||
| %% UI LAYER | ||
| %% ======================= | ||
| UI["USER INTERFACE<br/>(React + Tailwind + Lovable)"] | ||
|
|
||
| %% ======================= | ||
| %% ORCHESTRATION | ||
| %% ======================= | ||
| ORCH["Tender Search Orchestration Layer<br/>(Next.js API / Server Actions)"] | ||
|
|
||
| %% ======================= | ||
| %% SERVICES | ||
| %% ======================= | ||
| DB["SUPABASE<br/>(Cached Tenders & Metadata)"] | ||
| MINO["MINO API<br/>(Browser Automation)"] | ||
|
|
||
| %% ======================= | ||
| %% DETAILS | ||
| %% ======================= | ||
| DBD["• Cached tender listings<br/>• Deduplicated tenders<br/>• Historical records"] | ||
| MINOD["• Parallel web agents<br/>• Browse govt tender portals<br/>• Open tender pages<br/>• Extract structured fields<br/>• SSE streaming updates"] | ||
|
|
||
| %% ======================= | ||
| %% CONNECTIONS | ||
| %% ======================= | ||
| UI --> ORCH | ||
|
|
||
| ORCH --> DB | ||
| ORCH --> MINO | ||
|
|
||
| DB --> DBD | ||
| MINO --> MINOD | ||
| ``` |
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| { | ||
| "$schema": "https://ui.shadcn.com/schema.json", | ||
| "style": "default", | ||
| "rsc": false, | ||
| "tsx": true, | ||
| "tailwind": { | ||
| "config": "tailwind.config.ts", | ||
| "css": "src/index.css", | ||
| "baseColor": "slate", | ||
| "cssVariables": true, | ||
| "prefix": "" | ||
| }, | ||
| "aliases": { | ||
| "components": "@/components", | ||
| "utils": "@/lib/utils", | ||
| "ui": "@/components/ui", | ||
| "lib": "@/lib", | ||
| "hooks": "@/hooks" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| import js from "@eslint/js"; | ||
| import globals from "globals"; | ||
| import reactHooks from "eslint-plugin-react-hooks"; | ||
| import reactRefresh from "eslint-plugin-react-refresh"; | ||
| import tseslint from "typescript-eslint"; | ||
|
|
||
| export default tseslint.config( | ||
| { ignores: ["dist"] }, | ||
| { | ||
| extends: [js.configs.recommended, ...tseslint.configs.recommended], | ||
| files: ["**/*.{ts,tsx}"], | ||
| languageOptions: { | ||
| ecmaVersion: 2020, | ||
| globals: globals.browser, | ||
| }, | ||
| plugins: { | ||
| "react-hooks": reactHooks, | ||
| "react-refresh": reactRefresh, | ||
| }, | ||
| rules: { | ||
| ...reactHooks.configs.recommended.rules, | ||
| "react-refresh/only-export-components": ["warn", { allowConstantExport: true }], | ||
| "@typescript-eslint/no-unused-vars": "off", | ||
| }, | ||
| }, | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| <!doctype html> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="UTF-8" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> | ||
| <!-- TODO: Set the document title to the name of your application --> | ||
| <title>Lovable App</title> | ||
| <meta name="description" content="Lovable Generated Project" /> | ||
| <meta name="author" content="Lovable" /> | ||
|
|
||
| <!-- TODO: Update og:title to match your application name --> | ||
| <meta property="og:title" content="Lovable App" /> | ||
| <meta property="og:description" content="Lovable Generated Project" /> | ||
| <meta property="og:type" content="website" /> | ||
| <meta property="og:image" content="https://lovable.dev/opengraph-image-p98pqg.png" /> | ||
|
|
||
| <meta name="twitter:card" content="summary_large_image" /> | ||
| <meta name="twitter:site" content="@Lovable" /> | ||
| <meta name="twitter:image" content="https://lovable.dev/opengraph-image-p98pqg.png" /> | ||
| </head> | ||
|
|
||
| <body> | ||
| <div id="root"></div> | ||
| <script type="module" src="/src/main.tsx"></script> | ||
| </body> | ||
| </html> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove committed secrets from
.envand rotate the exposed key.The publishable key (JWT) is in version control, which can enable unauthorized access. Replace with placeholders, add
.envto.gitignore, and rotate/revoke the exposed key in Supabase.🔒 Proposed fix (keep only placeholders in repo)
🧰 Tools
🪛 dotenv-linter (4.0.0)
[warning] 1-1: [QuoteCharacter] The value has quote characters (', ")
(QuoteCharacter)
[warning] 2-2: [QuoteCharacter] The value has quote characters (', ")
(QuoteCharacter)
[warning] 3-3: [QuoteCharacter] The value has quote characters (', ")
(QuoteCharacter)
🪛 Gitleaks (8.30.0)
[high] 2-2: Uncovered a JSON Web Token, which may lead to unauthorized access to web applications and sensitive user data.
(jwt)
🤖 Prompt for AI Agents