davidslater: What about adding a test that ensures that the "docker_image" field in each of the configs in `scenario_configs` matches the current version of armory? 
seanpmorgan: > What about adding a test that ensures that the "docker_image" field in each of the configs in `scenario_configs` matches the current version of armory?

Good idea. Added
diff --git a/armory/__init__.py b/armory/__init__.py
index d690774f..cca49c04 100644
--- a/armory/__init__.py
+++ b/armory/__init__.py
@@ -27,7 +27,7 @@
 
 
 # Semantic Version
-__version__ = "0.6.0"
+__version__ = "0.7.0-dev"
 DEV = "-dev"
 
 
diff --git a/scenario_configs/cifar10_baseline.json b/scenario_configs/cifar10_baseline.json
index 2189af9b..aaee4cc6 100644
--- a/scenario_configs/cifar10_baseline.json
+++ b/scenario_configs/cifar10_baseline.json
@@ -40,7 +40,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/cifar10_defended_example.json b/scenario_configs/cifar10_defended_example.json
index f7e73b23..ea0ad0b0 100644
--- a/scenario_configs/cifar10_defended_example.json
+++ b/scenario_configs/cifar10_defended_example.json
@@ -49,7 +49,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/gtsrb_scenario_baseline.json b/scenario_configs/gtsrb_scenario_baseline.json
index 5fe37a7b..8859d848 100644
--- a/scenario_configs/gtsrb_scenario_baseline.json
+++ b/scenario_configs/gtsrb_scenario_baseline.json
@@ -45,7 +45,7 @@
         "name": "GTSRB"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/librispeech_baseline_sincnet.json b/scenario_configs/librispeech_baseline_sincnet.json
index 17bd007f..4406e1b7 100644
--- a/scenario_configs/librispeech_baseline_sincnet.json
+++ b/scenario_configs/librispeech_baseline_sincnet.json
@@ -45,7 +45,7 @@
         "name": "AudioClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": "hkakitani/SincNet",
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/mnist_baseline.json b/scenario_configs/mnist_baseline.json
index 64d6b2f8..33128148 100644
--- a/scenario_configs/mnist_baseline.json
+++ b/scenario_configs/mnist_baseline.json
@@ -40,7 +40,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/resisc45_baseline_densenet121.json b/scenario_configs/resisc45_baseline_densenet121.json
index edc00a25..ff1c6d3d 100644
--- a/scenario_configs/resisc45_baseline_densenet121.json
+++ b/scenario_configs/resisc45_baseline_densenet121.json
@@ -38,7 +38,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/resisc45_baseline_densenet121_finetune.json b/scenario_configs/resisc45_baseline_densenet121_finetune.json
index 393c32b7..da2e7c5a 100644
--- a/scenario_configs/resisc45_baseline_densenet121_finetune.json
+++ b/scenario_configs/resisc45_baseline_densenet121_finetune.json
@@ -38,7 +38,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0-dev",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/ucf101_baseline_finetune.json b/scenario_configs/ucf101_baseline_finetune.json
index ad8ff7f1..5f5569fc 100644
--- a/scenario_configs/ucf101_baseline_finetune.json
+++ b/scenario_configs/ucf101_baseline_finetune.json
@@ -44,7 +44,7 @@
         "name": "Ucf101"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": "yusong-tan/MARS",
         "gpus": "all",
         "use_gpu": false
diff --git a/scenario_configs/ucf101_baseline_pretrained.json b/scenario_configs/ucf101_baseline_pretrained.json
index 9bb748f5..1b6f4512 100644
--- a/scenario_configs/ucf101_baseline_pretrained.json
+++ b/scenario_configs/ucf101_baseline_pretrained.json
@@ -44,7 +44,7 @@
         "name": "Ucf101"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": "yusong-tan/MARS",
         "gpus": "all",
         "use_gpu": false

Move master to 0.7.0-dev
diff --git a/tests/scenarios/broken/invalid_module.json b/tests/scenarios/broken/invalid_module.json
index 75734fe1..99ee5b06 100644
--- a/tests/scenarios/broken/invalid_module.json
+++ b/tests/scenarios/broken/invalid_module.json
@@ -35,7 +35,7 @@
         "name": "fgm_attack"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "",
         "use_gpu": false
diff --git a/tests/scenarios/broken/missing_scenario.json b/tests/scenarios/broken/missing_scenario.json
index e13e12de..e4b0bee0 100644
--- a/tests/scenarios/broken/missing_scenario.json
+++ b/tests/scenarios/broken/missing_scenario.json
@@ -35,7 +35,7 @@
         "wrapper_kwargs": {}
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/tests/scenarios/pytorch/image_classification.json b/tests/scenarios/pytorch/image_classification.json
index b04f9426..48cd0b3f 100644
--- a/tests/scenarios/pytorch/image_classification.json
+++ b/tests/scenarios/pytorch/image_classification.json
@@ -40,7 +40,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/tests/scenarios/pytorch/image_classification_pretrained.json b/tests/scenarios/pytorch/image_classification_pretrained.json
index 8d508ccc..ee0e94a5 100644
--- a/tests/scenarios/pytorch/image_classification_pretrained.json
+++ b/tests/scenarios/pytorch/image_classification_pretrained.json
@@ -38,7 +38,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/pytorch:0.6.0",
+        "docker_image": "twosixarmory/pytorch:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/tests/scenarios/tf1/image_classification.json b/tests/scenarios/tf1/image_classification.json
index c06139e5..2c5ebd6c 100644
--- a/tests/scenarios/tf1/image_classification.json
+++ b/tests/scenarios/tf1/image_classification.json
@@ -40,7 +40,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/tests/scenarios/tf1/image_classification_pretrained.json b/tests/scenarios/tf1/image_classification_pretrained.json
index 9673be27..a42d1f1b 100644
--- a/tests/scenarios/tf1/image_classification_pretrained.json
+++ b/tests/scenarios/tf1/image_classification_pretrained.json
@@ -38,7 +38,7 @@
         "name": "ImageClassificationTask"
     },
     "sysconfig": {
-        "docker_image": "twosixarmory/tf1:0.6.0",
+        "docker_image": "twosixarmory/tf1:0.7.0-dev",
         "external_github_repo": null,
         "gpus": "all",
         "use_gpu": false
diff --git a/tests/test_host/test_config_verification.py b/tests/test_host/test_config_verification.py
index 8be3eb43..bb61f561 100644
--- a/tests/test_host/test_config_verification.py
+++ b/tests/test_host/test_config_verification.py
@@ -5,6 +5,7 @@
 import pytest
 
 from armory.utils.configuration import load_config
+from armory import __version__
 
 
 def test_no_config():
@@ -31,7 +32,10 @@ def test_scenario_configs():
     scenario_jsons = glob("scenario_configs/*.json")
 
     for json_path in scenario_jsons:
-        load_config(str(json_path))
+        config = load_config(str(json_path))
+        assert (
+            __version__ in config["sysconfig"]["docker_image"]
+        ), "Docker image does not match version in repository"
 
 
 def test_all_examples():
@@ -41,4 +45,7 @@ def test_all_examples():
         + glob("tests/scenarios/pytorch/*.json")
     )
     for json_path in test_jsons:
-        load_config(str(json_path))
+        config = load_config(str(json_path))
+        assert (
+            __version__ in config["sysconfig"]["docker_image"]
+        ), "Docker image does not match version in repository"

pip install -e .[dev]
parse_log_pytest
requirements.txt
black flake8 pytest yamllint
3.8
requirements.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
-e git+https://github.com/twosixlabs/armory.git@d6128f89e274f2e42b8a3a16379de984dfa9260e#egg=armory_testbed
attrs==25.3.0
black==24.8.0
certifi==2025.1.31
charset-normalizer==3.4.1
click==8.1.8
coloredlogs==15.0.1
docker==7.1.0
exceptiongroup==1.2.2
flake8==7.1.2
humanfriendly==10.0
idna==3.10
importlib_resources==6.4.5
iniconfig==2.1.0
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
mccabe==0.7.0
mypy-extensions==1.0.0
packaging==24.2
pathspec==0.12.1
pkgutil_resolve_name==1.3.10
platformdirs==4.3.6
pluggy==1.5.0
pycodestyle==2.12.1
pyflakes==3.2.0
pytest==8.3.5
PyYAML==6.0.2
referencing==0.35.1
requests==2.32.3
rpds-py==0.20.1
tomli==2.2.1
typing_extensions==4.13.0
urllib3==2.2.3
yamllint==1.35.1
zipp==3.20.2

name: armory
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - pip=24.2=py38h06a4308_0
  - python=3.8.20=he870216_0
  - readline=8.2=h5eee18b_0
  - setuptools=75.1.0=py38h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - wheel=0.44.0=py38h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - attrs==25.3.0
      - black==24.8.0
      - certifi==2025.1.31
      - charset-normalizer==3.4.1
      - click==8.1.8
      - coloredlogs==15.0.1
      - docker==7.1.0
      - exceptiongroup==1.2.2
      - flake8==7.1.2
      - humanfriendly==10.0
      - idna==3.10
      - importlib-resources==6.4.5
      - iniconfig==2.1.0
      - jsonschema==4.23.0
      - jsonschema-specifications==2023.12.1
      - mccabe==0.7.0
      - mypy-extensions==1.0.0
      - packaging==24.2
      - pathspec==0.12.1
      - pkgutil-resolve-name==1.3.10
      - platformdirs==4.3.6
      - pluggy==1.5.0
      - pycodestyle==2.12.1
      - pyflakes==3.2.0
      - pytest==8.3.5
      - pyyaml==6.0.2
      - referencing==0.35.1
      - requests==2.32.3
      - rpds-py==0.20.1
      - tomli==2.2.1
      - typing-extensions==4.13.0
      - urllib3==2.2.3
      - yamllint==1.35.1
      - zipp==3.20.2
prefix: /opt/conda/envs/armory

tests/test_host/test_config_verification.py::test_all_examples
tests/test_host/test_config_verification.py::test_no_config tests/test_host/test_config_verification.py::test_no_scenario tests/test_host/test_config_verification.py::test_invalid_module tests/test_host/test_config_verification.py::test_scenario_configs
scenario_configs/cifar10_baseline.json armory/__init__.py scenario_configs/cifar10_defended_example.json scenario_configs/resisc45_baseline_densenet121_finetune.json scenario_configs/resisc45_baseline_densenet121.json scenario_configs/gtsrb_scenario_baseline.json scenario_configs/ucf101_baseline_finetune.json scenario_configs/mnist_baseline.json scenario_configs/ucf101_baseline_pretrained.json scenario_configs/librispeech_baseline_sincnet.json
scenario_configs/cifar10_baseline.json armory/__init__.py scenario_configs/cifar10_defended_example.json scenario_configs/resisc45_baseline_densenet121_finetune.json scenario_configs/resisc45_baseline_densenet121.json scenario_configs/gtsrb_scenario_baseline.json scenario_configs/ucf101_baseline_finetune.json scenario_configs/mnist_baseline.json scenario_configs/ucf101_baseline_pretrained.json scenario_configs/librispeech_baseline_sincnet.json