rmm commited on
Commit
656770c
·
1 Parent(s): ca9f317

docs: added some notes on running the visual tests

Browse files

- selenium requires running the app, and then running a second browser
or tab that tests the app.

Files changed (1) hide show
  1. docs/dev_notes.md +52 -0
docs/dev_notes.md CHANGED
@@ -44,6 +44,25 @@ mkdocs build -c
44
 
45
  # Testing
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## local testing
48
  To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
49
 
@@ -74,6 +93,39 @@ pytest --junit-xml=test-results.xml
74
  pytest --cov-report=lcov --cov=src
75
  ```
76
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
 
78
  ## CI testing
79
 
 
44
 
45
  # Testing
46
 
47
+ ## use of markers
48
+
49
+ The CI runs with `--strict-markers` so any new marker must be registered in
50
+ `pytest.ini`.
51
+
52
+ - the basic CI action runs the fast tests only, skipping all tests marked
53
+ `visual` and `slow`
54
+ - the CI action on PR runs the `slow` tests, but stil excluding `visual`.
55
+ - TODO: a new action for the visual tests is to be developed.
56
+
57
+ Check all tests are marked ok, and that they are filtered correctly by the
58
+ groupings used in CI:
59
+ ```bash
60
+ pytest --collect-only -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
61
+ pytest --collect-only -m "not visual" --strict-markers --ignore=tests/visual_selenium
62
+ ```
63
+
64
+
65
+
66
  ## local testing
67
  To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
68
 
 
93
  pytest --cov-report=lcov --cov=src
94
  ```
95
 
96
+ ## local testing for visual tests
97
+
98
+ We use seleniumbase to test the visual appearance of the app, including the
99
+ presence of elements that appear through the workflow. This testing takes quite
100
+ a long time to execute and is not yet configured with CI.
101
+
102
+ ```bash
103
+ # install packages for app and for visual testing
104
+ pip install ./requirements.txt
105
+ pip install -r tests/visual_selenium/requirements_visual.txt
106
+ ```
107
+
108
+ **Running tests**
109
+ The execution of these tests requires that the site/app is running already.
110
+
111
+ In one tab:
112
+ ```bash
113
+ streamlit run src/main.py
114
+ ```
115
+
116
+ In another tab:
117
+ ```bash
118
+ # run just the visual tests
119
+ pytest -m "visual" --strict-markers
120
+ # run in demo mode, using firefox (default is chrome)
121
+ pytest -m "visual" --strict-markers -s browser=firefox --demo
122
+
123
+ # the inverse set:
124
+ pytest -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
125
+
126
+ ```
127
+
128
+
129
 
130
  ## CI testing
131