Spaces:
Running
Running
remove
Browse files
README.md
CHANGED
@@ -236,8 +236,6 @@ We launch a Chromium browser and navigate to the `homepage` provided in the `Run
|
|
236 |
|
237 |
Actions in an environment are defined through available tool calls, which in the browser case are set as default in the `BrowserTool` class. This allows the model to click, type, etc. at relevant `mark_id` elements on the page. These elements are extracted using JavaScript injected into the page in order to make interaction easier for the models.
|
238 |
|
239 |
-
If you want to not use this set-of-marks approach, you can set the `no_pois_in_image` flag to `True`, and the `include_poi_text` flag to `False` in the `EnvironmentConfig`. This way the model will only see the original image, and not the annotated image with these points-of-interest (POIs). In this case, you would want to update the `BrowserTool` to interact with pixel coordinates instead of the `mark_id`s.
|
240 |
-
|
241 |
**Note:** We use `playwright_stealth` to lower the chance of detection by anti-bot services, but this isn't foolproof and Proxy Lite may still get blocked with captchas or other anti-bot measures, especially when using the `headless` flag. We recommend using network proxies to avoid this issue.
|
242 |
|
243 |
|
@@ -247,12 +245,6 @@ This model has not currently been designed to act as a full assistant that can i
|
|
247 |
As such, it will struggle with tasks that require credentials or user interaction such as actually purchasing items if you don't give all the required details in the prompt.
|
248 |
|
249 |
|
250 |
-
## Future Work
|
251 |
-
|
252 |
-
- [ ] Pixel level control over the mouse movements.
|
253 |
-
- [ ] Full computer sandbox.
|
254 |
-
- [ ] Multi agent support.
|
255 |
-
|
256 |
## Citation
|
257 |
|
258 |
|
|
|
236 |
|
237 |
Actions in an environment are defined through available tool calls, which in the browser case are set as default in the `BrowserTool` class. This allows the model to click, type, etc. at relevant `mark_id` elements on the page. These elements are extracted using JavaScript injected into the page in order to make interaction easier for the models.
|
238 |
|
|
|
|
|
239 |
**Note:** We use `playwright_stealth` to lower the chance of detection by anti-bot services, but this isn't foolproof and Proxy Lite may still get blocked with captchas or other anti-bot measures, especially when using the `headless` flag. We recommend using network proxies to avoid this issue.
|
240 |
|
241 |
|
|
|
245 |
As such, it will struggle with tasks that require credentials or user interaction such as actually purchasing items if you don't give all the required details in the prompt.
|
246 |
|
247 |
|
|
|
|
|
|
|
|
|
|
|
|
|
248 |
## Citation
|
249 |
|
250 |
|