Skip to content

v.0.1.0
Compare
Choose a tag to compare
@asim-shrestha asim-shrestha released this 08 Nov 19:00
· 850 commits to main since this release
v.0.1.0
136aa06
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Super excited for the first version of Banana-lyzer, an open source AI Agent evaluation framework and dataset for web tasks with Playwright (And has a banana theme because why not) 🍌

We aim to solve the following issues with testing web agents:

  • Websites change overtime, are affected by latency, and may have anti bot protections.
    We need a system that can reliably save and deploy historic/static snapshots of websites.
  • Standard web practices are loose and there is an abundance of different underlying ways to represent a single individual website. For an agent to best generalize, we require building a diverse dataset of websites across industries and use-cases.
  • We have specific evaluation criteria and agent use cases focusing on structured and direct information retrieval across websites.
  • There exists valuable web task datasets and evaluations that we'd like to unify in a single repo (Mind2Web, WebArena, etc).
Assets 2
Loading