Spaces:
Sleeping
Sleeping
rmm
commited on
Commit
·
656770c
1
Parent(s):
ca9f317
docs: added some notes on running the visual tests
Browse files- selenium requires running the app, and then running a second browser
or tab that tests the app.
- docs/dev_notes.md +52 -0
docs/dev_notes.md
CHANGED
@@ -44,6 +44,25 @@ mkdocs build -c
|
|
44 |
|
45 |
# Testing
|
46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
## local testing
|
48 |
To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
|
49 |
|
@@ -74,6 +93,39 @@ pytest --junit-xml=test-results.xml
|
|
74 |
pytest --cov-report=lcov --cov=src
|
75 |
```
|
76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
77 |
|
78 |
## CI testing
|
79 |
|
|
|
44 |
|
45 |
# Testing
|
46 |
|
47 |
+
## use of markers
|
48 |
+
|
49 |
+
The CI runs with `--strict-markers` so any new marker must be registered in
|
50 |
+
`pytest.ini`.
|
51 |
+
|
52 |
+
- the basic CI action runs the fast tests only, skipping all tests marked
|
53 |
+
`visual` and `slow`
|
54 |
+
- the CI action on PR runs the `slow` tests, but stil excluding `visual`.
|
55 |
+
- TODO: a new action for the visual tests is to be developed.
|
56 |
+
|
57 |
+
Check all tests are marked ok, and that they are filtered correctly by the
|
58 |
+
groupings used in CI:
|
59 |
+
```bash
|
60 |
+
pytest --collect-only -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
|
61 |
+
pytest --collect-only -m "not visual" --strict-markers --ignore=tests/visual_selenium
|
62 |
+
```
|
63 |
+
|
64 |
+
|
65 |
+
|
66 |
## local testing
|
67 |
To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
|
68 |
|
|
|
93 |
pytest --cov-report=lcov --cov=src
|
94 |
```
|
95 |
|
96 |
+
## local testing for visual tests
|
97 |
+
|
98 |
+
We use seleniumbase to test the visual appearance of the app, including the
|
99 |
+
presence of elements that appear through the workflow. This testing takes quite
|
100 |
+
a long time to execute and is not yet configured with CI.
|
101 |
+
|
102 |
+
```bash
|
103 |
+
# install packages for app and for visual testing
|
104 |
+
pip install ./requirements.txt
|
105 |
+
pip install -r tests/visual_selenium/requirements_visual.txt
|
106 |
+
```
|
107 |
+
|
108 |
+
**Running tests**
|
109 |
+
The execution of these tests requires that the site/app is running already.
|
110 |
+
|
111 |
+
In one tab:
|
112 |
+
```bash
|
113 |
+
streamlit run src/main.py
|
114 |
+
```
|
115 |
+
|
116 |
+
In another tab:
|
117 |
+
```bash
|
118 |
+
# run just the visual tests
|
119 |
+
pytest -m "visual" --strict-markers
|
120 |
+
# run in demo mode, using firefox (default is chrome)
|
121 |
+
pytest -m "visual" --strict-markers -s browser=firefox --demo
|
122 |
+
|
123 |
+
# the inverse set:
|
124 |
+
pytest -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
|
125 |
+
|
126 |
+
```
|
127 |
+
|
128 |
+
|
129 |
|
130 |
## CI testing
|
131 |
|