Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
504 changes: 504 additions & 0 deletions COMPONENT_STRUCTURE.md

Large diffs are not rendered by default.

407 changes: 407 additions & 0 deletions DEBUGGING_GUIDE.md

Large diffs are not rendered by default.

465 changes: 465 additions & 0 deletions ERROR_ANALYSIS.md

Large diffs are not rendered by default.

470 changes: 470 additions & 0 deletions LEARNING_GUIDE.md

Large diffs are not rendered by default.

535 changes: 535 additions & 0 deletions NEXTJS_LEARNING_GUIDE.md

Large diffs are not rendered by default.

167 changes: 167 additions & 0 deletions TEAM_REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# OpenML Search Implementation - Team Report

**Date:** November 16, 2025
**Status:** ✅ Completed

---

## Summary

Successfully migrated OpenML search functionality from old routes to new SEO-friendly URLs with direct Elasticsearch integration, bypassing the problematic API proxy layer.

---

## What Changed

### 1. New SEO-Friendly Routes

All search pages now use clean, SEO-optimized URLs:

| Old URL | New URL | Status |
| ----------- | ----------- | ------- |
| `/d/search` | `/datasets` | ✅ Live |
| `/t/search` | `/tasks` | ✅ Live |
| `/f/search` | `/flows` | ✅ Live |
| `/r/search` | `/runs` | ✅ Live |

**Benefits:**

- Better Google indexing
- Cleaner URLs for sharing
- Improved user experience
- All old URLs automatically redirect (no broken links)

---

### 2. Custom Elasticsearch Connector

**Problem:** The `@elastic/search-ui-elasticsearch-connector` library was returning HTML error pages instead of JSON responses.

**Solution:** Created `OpenMLSearchConnector.js` that directly queries Elasticsearch 6.8.23 at `https://www.openml.org/es/`

**Key Features:**

- Direct fetch() API calls to Elasticsearch
- Handles search terms, filters, pagination, sorting, and facets
- Wraps results in Search UI's expected `{ raw: value }` format
- Strips invalid `name` fields from range aggregations
- Compatible with ES 6.8.23 query format

**Files Updated:**

- `app/src/services/OpenMLSearchConnector.js` (new)
- `app/src/search_configs/dataConfig.js`
- `app/src/search_configs/taskConfig.js`
- `app/src/search_configs/flowConfig.js`
- `app/src/search_configs/runConfig.js`

---

### 3. Bug Fixes

| Issue | Fix | File |
| --------------------------------------- | ------------------------------------------------------------------ | ---------------------------------- |
| Elasticsearch 400 error on range facets | Removed `name` field from range aggregations | `OpenMLSearchConnector.js` |
| Sort not working | Added `sortList` array handling instead of individual fields | `OpenMLSearchConnector.js` |
| Missing values sort error | Fixed typo: `"NumberOfMissing values"` → `"NumberOfMissingValues"` | `datasets.js` |
| Description font size not applying | Changed `fontSize: "12px"` to `fontSize: "inherit"` | `Teaser.js` |
| Grid responsive layout | Set xs=12, sm=6, md=4 for proper column display | `ResultGridCard.js` |
| Container width constraints | Set width: 100% with 24px horizontal margins | `Wrapper.js`, `SearchContainer.js` |

---

### 4. Architecture Changes

**Before:**

```
Browser → Next.js → SearchAPIConnector → API Proxy → Elasticsearch
```

**After:**

```
Browser → Next.js → OpenMLSearchConnector → Elasticsearch (direct)
```

**Benefits:**

- One less failure point
- Faster queries (no proxy overhead)
- Easier debugging with direct ES error messages
- No dependency on Flask backend for read operations

---

## Testing Results

✅ **Datasets:** 24,498 results, search/filter/sort working
✅ **Tasks:** Search and filters functional
✅ **Flows:** Search and filters functional
✅ **Runs:** Search and filters functional

**Elasticsearch Status:**

- Cluster: `openmlelasticsearch`
- Version: 6.8.23
- Endpoint: `https://www.openml.org/es/`

---

## Files Created/Modified

### New Files

- `app/src/pages/datasets.js`
- `app/src/pages/tasks.js`
- `app/src/pages/flows.js`
- `app/src/pages/runs.js`
- `app/src/services/OpenMLSearchConnector.js`
- `app/src/components/search/DatasetSearchResults.jsx`

### Modified Files

- `app/src/pages/d/search.js` (now redirects)
- `app/src/pages/t/search.js` (now redirects)
- `app/src/pages/f/search.js` (now redirects)
- `app/src/pages/r/search.js` (now redirects)
- All config files in `app/src/search_configs/`
- `app/src/components/search/SearchContainer.js`
- `app/src/components/search/Teaser.js`
- `app/src/components/Wrapper.js`
- `app/src/components/search/ResultGridCard.js`

---

## Known Issues

None at this time. All major functionality working as expected.

---

## Next Steps (Optional)

1. **Performance:** Consider adding request caching
2. **Analytics:** Add tracking for search queries
3. **UI:** Remove debug console.log statements (currently helpful for monitoring)
4. **Tests:** Add unit tests for OpenMLSearchConnector
5. **Documentation:** Update API documentation to reflect new routes

---

## Technical Notes

- **No Docker Required:** Frontend connects directly to production Elasticsearch
- **No Flask Backend Needed:** For read-only search operations
- **Backwards Compatible:** All old URLs redirect automatically
- **SEO Optimized:** Each page has proper meta tags, Open Graph tags, and canonical URLs

---

## Contact

For questions or issues, check:

- Browser console logs (prefixed with `[OpenMLSearchConnector]`)
- Network tab for Elasticsearch requests to `https://www.openml.org/es/`
- Server terminal for Next.js errors
41 changes: 41 additions & 0 deletions app-next/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# dependencies
/node_modules
/.pnp
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/versions

# testing
/coverage

# next.js
/.next/
/out/

# production
/build

# misc
.DS_Store
*.pem

# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*

# env files (can opt-in for committing if needed)
.env*

# vercel
.vercel

# typescript
*.tsbuildinfo
next-env.d.ts
3 changes: 3 additions & 0 deletions app-next/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"plugins": ["prettier-plugin-tailwindcss"]
}
36 changes: 36 additions & 0 deletions app-next/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.

This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.
22 changes: 22 additions & 0 deletions app-next/components.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "new-york",
"rsc": true,
"tsx": true,
"tailwind": {
"config": "",
"css": "src/app/globals.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"iconLibrary": "lucide",
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"registries": {}
}
18 changes: 18 additions & 0 deletions app-next/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { defineConfig, globalIgnores } from "eslint/config";
import nextVitals from "eslint-config-next/core-web-vitals";
import nextTs from "eslint-config-next/typescript";

const eslintConfig = defineConfig([
...nextVitals,
...nextTs,
// Override default ignores of eslint-config-next.
globalIgnores([
// Default ignores of eslint-config-next:
".next/**",
"out/**",
"build/**",
"next-env.d.ts",
]),
]);

export default eslintConfig;
7 changes: 7 additions & 0 deletions app-next/next.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
/* config options here */
};

export default nextConfig;
Loading