Title: Trouble Remotely Connecting Flask App to Selenium Grid using Docker
Tags: python;selenium;selenium-webdriver;docker-compose;selenium-grid
Question: I am new to both Docker and Selenium grid and am having issues getting my web app to connect to the selenium hub.
docker-compose.yml
version: '3.8'
services:
image: postgres
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
ports:
- &quot;${POSTGRES_PORT}:5432&quot;
volumes:
- pgdata:/var/lib/postgresql/data
web:
build:
context: ..
dockerfile: docker/Dockerfile
environment:
FLASK_ENV: ${FLASK_ENV}
FLASK_CONFIG: ${FLASK_CONFIG}
APPLICATION_DB: ${APPLICATION_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_HOSTNAME: &quot;db&quot;
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_PORT: ${POSTGRES_PORT}
command: flask run --host 151.154.11.224
volumes:
- ..:/opt/code
ports:
- &quot;5000:5000&quot;
chrome:
image: selenium/node-chrome:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- &quot;6900:5900&quot;
edge:
image: selenium/node-edge:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- &quot;6901:5900&quot;
firefox:
image: selenium/node-firefox:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- &quot;6902:5900&quot;
selenium-hub:
image: selenium/hub:4.0.0-20211013
container_name: selenium-hub
ports:
- &quot;4442:4442&quot;
- &quot;4443:4443&quot;
- &quot;4444:4444&quot;
volumes:
pgdata:
When running the stack of containers and checking netstat -a, I can see my desktop listening to port 4444 and when I kill the containers its not.
I can also verify that the hub is running and all of my nodes are connecting fine by visiting https//:localhost/4444, however when I run driver = webdriver.Remote(command_executor=&quot;http://localhost:4444&quot;) from my python flask app (which is running in the container specified as web above) I get the error:
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries
exceeded with url: /session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
object at 0x7fd4855b7730&gt;: Failed to establish a new connection: [Errno 111] Connection refused'))
I have tried specifying the desired capabilities to match the driver for specific nodes, however I receive this same error regardless.
I am using Selenium's latest build &quot;4.0.0&quot; and as you can see 4.0.0 images for the parts of the grid so I don't think its a compatibility issue.
docker ps
Name                       Command              State                            Ports
------------------------------------------------------------------------------------------------------------------------
development_chrome_1    /opt/bin/entry_point.sh         Up      151.154.11.224:6900-&gt;5900/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:6900-&gt;5900/tcp
development_db_1        docker-entrypoint.sh postgres   Up      151.154.11.224:5432-&gt;5432/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:5432-&gt;5432/tcp
development_edge_1      /opt/bin/entry_point.sh         Up      151.154.11.224:6901-&gt;5900/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:6901-&gt;5900/tcp
development_firefox_1   /opt/bin/entry_point.sh         Up      151.154.11.224:6902-&gt;5900/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:6902-&gt;5900/tcp
development_web_1       flask run --host 151.154.11.224        Up      151.154.11.224:5000-&gt;5000/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:5000-&gt;5000/tcp
selenium-hub            /opt/bin/entry_point.sh         Up      151.154.11.224:4442-&gt;4442/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:4442-&gt;4442/tcp,
151.154.11.224:4443-&gt;4443/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:4443-&gt;4443/tcp,
151.154.11.224:4444-&gt;4444/tcp,75b5:f2c1:f5d7:b954:8dc3:cee9:baf:a8c0:4444-&gt;4444/tcp
I feel like I'm fundamentally missing something here, Any thoughts?

Here is the accepted answer: I see the mistake now. I was mistakenly attempting to connect to http://localhost:4444 with my client, when I was needing to specify the network name deployed by selenium grid.
Change this line in your flask_app.py
driver = webdriver.Remote(command_executor=&quot;http://localhost:4444&quot;)
driver = webdriver.Remote(command_executor=&quot;http://container-name:4444&quot;)
Where container-name is the selenium hub name set in docker-compose.yml
selenium-hub:
image: selenium/hub:4.0.0-20211013
container_name: selenium-hub
ports:
- &quot;4442:4442&quot;
- &quot;4443:4443&quot;
- &quot;4444:4444&quot;
in my case: &quot;selenium-hub&quot;
Resource on docker networking used: https://docs.docker.com/compose/networking/
Final Thoughts
I guess I got tripped up by the fact that I am still using http://localhost:port to connect to both the grid hub and web container. I guess the difference is where the client request comes from? From outside the docker stack vs within? Anyway, hope this helps someone.


{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 1, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 4.0', 'CreationDate': '2021:00:10/15/21', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<python><selenium><selenium-webdriver><docker-compose><selenium-grid>', 'Title': 'Trouble Remotely Connecting Flask App to Selenium Grid using Docker', 'ViewCount': 419, 'id': 69580039, 'user_AboutMe': None, 'user_AccountId': 23037159.0, 'user_DisplayName': 'Asmith9555', 'user_Reputation': 65.0, 'user_Views': 3.0, 'user_id': 17155760.0}}