Skip to content

the coordination is wrong at times #329

@MrShang

Description

@MrShang

System Info / 系統信息

When I input a description and request a click on a specific icon, the x-coordinate is correctly based on a coordinate system with (0,0) at the top-left of the page and (999,999) at the bottom-right of the page. However, the y-coordinate seems to use the physical screen dimensions instead of the same page-relative coordinate system.

I tried to update the prompt, it seems not work,

Who can help? / 谁可以帮助到您?

No response

Information / 问题信息

  • The official example scripts / 官方的示例脚本
  • My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

  1. input your task: 打开[途虎养车],直接点击首页上方搜索框最右侧的方形扫一扫图标(特征为一个正方形,中间一条横线),进入相机扫描页面任务停止。
  2. try to debug and check the coordinate of the icon returned by the model

NOTE: sometimes it's correct, and can be wrong when you test lots of times and make sure to test in different phones

Expected behavior / 期待表现

To return the correct coordinate based on the coordinate system defined in the prompt;
OR just to return the real coordinate based on the physical size of the phone.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions