Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native mobile app testing #227

Open
slavingia opened this issue Dec 31, 2024 · 12 comments · May be fixed by #260
Open

Native mobile app testing #227

slavingia opened this issue Dec 31, 2024 · 12 comments · May be fixed by #260

Comments

@slavingia
Copy link
Contributor

What

  • Native mobile app testing support in Shortest for iOS and Android.
  • Core workflows to install, launch, and interact with real/emulated apps using Shortest’s plain-English test syntax.
  • Additional setup docs or scripts to simplify environment configuration for mobile testing.

Why

  • Users need the same straightforward, AI-assisted test coverage for native mobile apps that Shortest offers for web.
  • Reduces QA overhead by allowing devs to reuse the existing Shortest approach (plain English tests) across all platforms.
  • Unlocks the $2,000 bounty for the first contributor to implement iOS + Android support, payable in cash, Anti-Work (Gumroad, Inc.) equity, or a combo.
@gladyshcodes
Copy link
Contributor

gladyshcodes commented Jan 1, 2025

Maybe worth creating new shortest-mobile repo for it?

@slavingia
Copy link
Contributor Author

Why? I think one repo is much less cognitive load and seems reasonable to have it be able to test all 3 platforms using computer use.

@gladyshcodes
Copy link
Contributor

gladyshcodes commented Jan 2, 2025

Why? I think one repo is much less cognitive load and seems reasonable to have it be able to test all 3 platforms using computer use

Ok

One more question: do you think that is a good idea to spin up expo react native? we need env to run and test mobile Shortest tests

@slavingia
Copy link
Contributor Author

Yeah not sure the approach but something like that for sure will be useful.

@crmchattie
Copy link

Is someone/are people actively working on this? I'm interested in helping out. If not, I will start.

@rmarescu
Copy link
Member

@gladyshcodes has a draft PR #260. Probalby best to collaborate on that one, to reduce duplicate effort.

@rmarescu rmarescu moved this to In Progress in Shortest Jan 19, 2025
@rmarescu rmarescu linked a pull request Jan 19, 2025 that will close this issue
@gladyshcodes
Copy link
Contributor

gladyshcodes commented Jan 28, 2025

Following up on our conversation with @rmarescu, we’ve decided to split #260 into several atomic PRs to reduce cognitive load, lower the risk of bugs, and ensure clearer, more focused code reviews. Thoughts:

Ensuring SRP:

  • Refactor repository into a modular monolith architecture
  • Create WebDriver and WebBrowser, deprecate BrowserManager and BrowserTool

Scaling for mobile:

  • Mobile repo setup
  • Add support for Android; AndroidDriver and AndroidBrowser
    • Appium environment setup
    • Image resizing caps for Anthropic models compatibility
  • Add support for IOS; IOSDriver and IOSBrowser

Testing:

  • In addition to exploratory testing of the above prs, write Vitest tests for core functionalities across all three platforms supported (after integrating Vitest into the codebase). Our goal is to achieve strong test coverage over time.

Final:

  • Finalize implementation by addressing technical debt, refining modularity, and polishing for production readiness

Miscellaneous:

  • Add JS shim
  • Make __shortest__ global variable with proper typing
  • Convert anthropic tools to dynamic functions

cc @slavingia

@rmarescu
Copy link
Member

I think we can discuss technical details next week, and scope for best approach. For now, we should pause development on this until we have a better foundation in place.

@gladyshcodes
Copy link
Contributor

@rmarescu, I remember you mentioned that Browser isn't the best name. I'd love to hear your thoughts on these alternatives:

AndroidInterface
AndroidController
AndroidUIDriver (may be confusing as we already have Driver suffix)
AndroidSession (Manager)
AndroidNavigator
AndroidContext
AndroidAgent
AndroidPage (just as playwright)

@crabest
Copy link
Contributor

crabest commented Jan 28, 2025

instead of another repo, wouldn't be better to have a seperated folder for mobile with its core functions just my draft https://github.com/anti-work/shortest/pull/230/files ?

Image

@gladyshcodes
Copy link
Contributor

gladyshcodes commented Jan 28, 2025

I think we can discuss technical details next week, and scope for best approach. For now, we should pause development on this until we have a better foundation in place.

Yep! Just mapped out my thoughts on how pr should be split

@gladyshcodes
Copy link
Contributor

instead of another repo, wouldn't be better to have a seperated folder for mobile with its core functions just my draft https://github.com/anti-work/shortest/pull/230/files ?

We need our own playground to test things. Regarding folder structure, we will most probably have a separate driver package (see)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

5 participants