codecov[bot]: # [Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=h1) Report
> Merging [#424](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=desc) into [develop](https://codecov.io/gh/andersbogsnes/ml_tooling/commit/acfa44ef4fd8a0867b4bd03444bc12d9371d8840?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424/graphs/tree.svg?width=650&token=ucKtL3BlxO&height=150&src=pr)](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=tree)

```diff
@@          Coverage Diff           @@
##           develop   #424   +/-   ##
======================================
  Coverage      100%   100%           
======================================
  Files           54     54           
  Lines         1386   1389    +3     
  Branches       157    159    +2     
======================================
+ Hits          1386   1389    +3
```


| [Impacted Files](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/ml\_tooling/data/base\_data.py](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424/diff?src=pr&el=tree#diff-c3JjL21sX3Rvb2xpbmcvZGF0YS9iYXNlX2RhdGEucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [src/ml\_tooling/data/sql.py](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424/diff?src=pr&el=tree#diff-c3JjL21sX3Rvb2xpbmcvZGF0YS9zcWwucHk=) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=footer). Last update [acfa44e...affe790](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/424?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

diff --git a/CHANGELOG.md b/CHANGELOG.md
index b40cc55..27fc088 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,7 @@
 - Fixed typehints in Dataset
 - Dataset.create_train_test now takes a boolean `stratify` parameter.
 - Added default local filestorage when using `save_estimator`
+- Dataset now verifies that `load_training_data` and `load_prediction_data` do not return empty
 
 # v0.10.2
 - Fixed type inferences from data to sql in _load_data
diff --git a/src/ml_tooling/data/base_data.py b/src/ml_tooling/data/base_data.py
index 550823c..88ab4d2 100644
--- a/src/ml_tooling/data/base_data.py
+++ b/src/ml_tooling/data/base_data.py
@@ -104,10 +104,16 @@ class Dataset(metaclass=abc.ABCMeta):
         return self.__class__.__name__
 
     def _load_training_data(self, *args, **kwargs) -> Tuple[pd.DataFrame, DataType]:
-        return self.load_training_data(*args, **kwargs)
+        x, y = self.load_training_data(*args, **kwargs)
+        if x.empty:
+            raise DatasetError("An empty dataset was returned by load_training_data")
+
+        return x, y
 
     def _load_prediction_data(self, *args, **kwargs) -> pd.DataFrame:
         pred_data = self.load_prediction_data(*args, **kwargs)
+        if pred_data.empty:
+            raise DatasetError("An empty dataset was returned by load_prediction_data")
         self.cached_data = pred_data
         return pred_data
 
diff --git a/src/ml_tooling/data/sql.py b/src/ml_tooling/data/sql.py
index 97e12e0..d17aed8 100644
--- a/src/ml_tooling/data/sql.py
+++ b/src/ml_tooling/data/sql.py
@@ -10,7 +10,6 @@ from contextlib import contextmanager
 from ml_tooling.data.base_data import Dataset
 from ml_tooling.utils import DataType, DatasetError
 
-
 logger = logging.getLogger("ml_tooling")
 
 
@@ -102,13 +101,11 @@ class SQLDataset(Dataset, metaclass=abc.ABCMeta):
 
     def _load_training_data(self, *args, **kwargs) -> Tuple[pd.DataFrame, DataType]:
         with self.create_connection() as conn:
-            return self.load_training_data(*args, conn=conn, **kwargs)
+            return super()._load_training_data(*args, conn=conn, **kwargs)
 
     def _load_prediction_data(self, *args, **kwargs) -> pd.DataFrame:
         with self.create_connection() as conn:
-            pred_data = self.load_prediction_data(*args, conn=conn, **kwargs)
-            self.cached_data = pred_data
-            return pred_data
+            return super()._load_prediction_data(*args, conn=conn, **kwargs)
 
     def _dump_data(self) -> pd.DataFrame:
         """

API: What to return when no matching data is found when predicting
We should make it clear what return value is expected when .make_prediction fails due to invalid data - either because the key doesn't exist or key is malformed.

Should we raise an MLToolingError? A subclassed InvalidInput or something similar?
We could also return 0 or None, but that feels like failing silently...
diff --git a/tests/test_baseclass.py b/tests/test_baseclass.py
index eef66b3..3f4879b 100644
--- a/tests/test_baseclass.py
+++ b/tests/test_baseclass.py
@@ -481,7 +481,7 @@ class TestTrainEstimator:
                 self.average = None
 
             def fit(self, x, y=None):
-                self.average = np.mean(x, axis=1)
+                self.average = np.mean(x, axis=0)
                 return self
 
             def predict(self, x):
@@ -489,10 +489,10 @@ class TestTrainEstimator:
 
         class DummyData(Dataset):
             def load_training_data(self):
-                return np.array([[1, 2, 3, 4], [4, 5, 6, 7]]), None
+                return pd.DataFrame({"col1": [1, 2, 3, 4], "col2": [4, 5, 6, 7]}), None
 
             def load_prediction_data(self, *args, **kwargs):
-                return np.array([[1, 2, 3, 4], [4, 5, 6, 7]])
+                return pd.DataFrame({"col1": [1, 2, 3, 4], "col2": [4, 5, 6, 7]})
 
         model = Model(DummyEstimator())
         data = DummyData()
diff --git a/tests/test_data.py b/tests/test_data.py
index 98c8866..62dbe0a 100644
--- a/tests/test_data.py
+++ b/tests/test_data.py
@@ -1,10 +1,12 @@
-import pandas as pd
-import pytest
+import pathlib
 from unittest.mock import MagicMock, patch
-from sqlalchemy.exc import DBAPIError
 
+import pandas as pd
+import pytest
+from ml_tooling import Model
 from ml_tooling.data import Dataset
 from ml_tooling.utils import DatasetError
+from sqlalchemy.exc import DBAPIError
 
 
 class TestDataset:
@@ -39,6 +41,33 @@ class TestDataset:
         dataset.create_train_test(stratify=True)
         assert dataset.has_validation_set is True
 
+    def test_dataset_that_returns_empty_training_data_errors_correctly(self):
+        class FailingDataset(Dataset):
+            def load_training_data(self, *args, **kwargs):
+                return pd.DataFrame(), None
+
+            def load_prediction_data(self, *args, **kwargs):
+                pass
+
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_training_data"
+        ):
+            FailingDataset()._load_training_data()
+
+    def test_dataset_raises_when_load_prediction_data_returns_empty(self, classifier):
+        class FailingDataset(Dataset):
+            def load_training_data(self, *args, **kwargs):
+                pass
+
+            def load_prediction_data(self, *args, **kwargs):
+                return pd.DataFrame()
+
+        data = FailingDataset()
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_prediction_data"
+        ):
+            classifier.make_prediction(data, 0)
+
     def test_cannot_instantiate_an_abstract_baseclass(self):
         with pytest.raises(TypeError):
             Dataset()
@@ -129,6 +158,31 @@ class TestSqlDataset:
             boston_sqldataset(test_engine, "")._dump_data()
         read_sql.assert_called_once()
 
+    def test_sql_dataset_raises_exception_when_load_training_data_returns_empty(
+        self, boston_sqldataset
+    ):
+        class FailingDataset(boston_sqldataset):
+            def load_training_data(self, *args, **kwargs):
+                return pd.DataFrame(), pd.Series()
+
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_training_data"
+        ):
+            FailingDataset("sqlite:///", schema=None).create_train_test()
+
+    def test_sql_dataset_raises_exception_when_load_prediction_data_returns_empty(
+        self, boston_sqldataset, regression
+    ):
+        class FailingDataset(boston_sqldataset):
+            def load_prediction_data(self, *args, **kwargs):
+                return pd.DataFrame()
+
+        data = FailingDataset("sqlite:///", schema=None)
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_prediction_data"
+        ):
+            regression.make_prediction(data, 0)
+
     def test_load_data_throws_error_on_exec_failure(
         self, boston_sqldataset, test_engine
     ):
@@ -204,3 +258,29 @@ class TestFileDataset:
         assert tmp_path.suffix == ""
         with pytest.raises(DatasetError, match="must point to a file"):
             boston_filedataset(tmp_path)
+
+    def test_filedataset_that_returns_empty_training_data_raises_exception(
+        self, boston_csv, boston_filedataset
+    ):
+        class FailingDataset(boston_filedataset):
+            def load_training_data(self, *args, **kwargs):
+                return pd.DataFrame(), pd.Series()
+
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_training_data"
+        ):
+            FailingDataset(boston_csv).create_train_test()
+
+    def test_filedataset_raises_exception_when_load_prediction_data_is_empty(
+        self, regression: Model, boston_filedataset, boston_csv: pathlib.Path
+    ):
+        class FailingDataset(boston_filedataset):
+            def load_prediction_data(self, *args, **kwargs):
+                return pd.DataFrame()
+
+        data = FailingDataset(boston_csv).create_train_test()
+
+        with pytest.raises(
+            DatasetError, match="An empty dataset was returned by load_prediction_data"
+        ):
+            regression.make_prediction(data, 0)

pip install -e .[dev]
parse_log_pytest
requirements.txt
pytest
3.7
requirements.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
alabaster==0.7.13
attrs==19.3.0
Babel==2.14.0
cachetools==5.5.2
certifi @ file:///croot/certifi_1671487769961/work/certifi
cfgv==3.3.1
chardet==5.2.0
charset-normalizer==3.4.1
colorama==0.4.6
coverage==7.2.7
cycler==0.10.0
distlib==0.3.9
docutils==0.19
exceptiongroup==1.2.2
filelock==3.12.2
identify==2.5.24
idna==3.10
imagesize==1.4.1
importlib-metadata==6.7.0
iniconfig==2.0.0
Jinja2==3.1.6
joblib==0.14.1
kiwisolver==1.1.0
MarkupSafe==2.1.5
matplotlib==3.1.3
-e git+https://github.com/andersbogsnes/ml_tooling.git@dc0726dd61a4d45a7125a0aec07e3418fc15fcb8#egg=ml_tooling
nodeenv==1.9.1
numpy==1.18.1
packaging==24.0
pandas==1.0.1
platformdirs==4.0.0
pluggy==1.2.0
pre-commit==2.21.0
Pygments==2.17.2
pyparsing==2.4.6
pyproject-api==1.5.3
pytest==7.4.4
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3
requests==2.31.0
scikit-learn==0.22.1
scipy==1.4.1
six==1.14.0
snowballstemmer==2.2.0
Sphinx==5.3.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
SQLAlchemy==1.3.13
tomli==2.0.1
tox==4.8.0
typing_extensions==4.7.1
urllib3==2.0.7
virtualenv==20.26.6
zipp==3.15.0

name: ml_tooling
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - certifi=2022.12.7=py37h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=1.1.1w=h7f8727e_0
  - pip=22.3.1=py37h06a4308_0
  - python=3.7.16=h7a1cb2a_0
  - readline=8.2=h5eee18b_0
  - setuptools=65.6.3=py37h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - wheel=0.38.4=py37h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - alabaster==0.7.13
      - attrs==19.3.0
      - babel==2.14.0
      - cachetools==5.5.2
      - cfgv==3.3.1
      - chardet==5.2.0
      - charset-normalizer==3.4.1
      - colorama==0.4.6
      - coverage==7.2.7
      - cycler==0.10.0
      - distlib==0.3.9
      - docutils==0.19
      - exceptiongroup==1.2.2
      - filelock==3.12.2
      - identify==2.5.24
      - idna==3.10
      - imagesize==1.4.1
      - importlib-metadata==6.7.0
      - iniconfig==2.0.0
      - jinja2==3.1.6
      - joblib==0.14.1
      - kiwisolver==1.1.0
      - markupsafe==2.1.5
      - matplotlib==3.1.3
      - nodeenv==1.9.1
      - numpy==1.18.1
      - packaging==24.0
      - pandas==1.0.1
      - platformdirs==4.0.0
      - pluggy==1.2.0
      - pre-commit==2.21.0
      - pygments==2.17.2
      - pyparsing==2.4.6
      - pyproject-api==1.5.3
      - pytest==7.4.4
      - python-dateutil==2.8.1
      - pytz==2019.3
      - pyyaml==5.3
      - requests==2.31.0
      - scikit-learn==0.22.1
      - scipy==1.4.1
      - six==1.14.0
      - snowballstemmer==2.2.0
      - sphinx==5.3.0
      - sphinxcontrib-applehelp==1.0.2
      - sphinxcontrib-devhelp==1.0.2
      - sphinxcontrib-htmlhelp==2.0.0
      - sphinxcontrib-jsmath==1.0.1
      - sphinxcontrib-qthelp==1.0.3
      - sphinxcontrib-serializinghtml==1.1.5
      - sqlalchemy==1.3.13
      - tomli==2.0.1
      - tox==4.8.0
      - typing-extensions==4.7.1
      - urllib3==2.0.7
      - virtualenv==20.26.6
      - zipp==3.15.0
prefix: /opt/conda/envs/ml_tooling

tests/test_data.py::TestDataset::test_dataset_that_returns_empty_training_data_errors_correctly tests/test_data.py::TestDataset::test_dataset_raises_when_load_prediction_data_returns_empty tests/test_data.py::TestSqlDataset::test_sql_dataset_raises_exception_when_load_training_data_returns_empty tests/test_data.py::TestSqlDataset::test_sql_dataset_raises_exception_when_load_prediction_data_returns_empty tests/test_data.py::TestFileDataset::test_filedataset_that_returns_empty_training_data_raises_exception tests/test_data.py::TestFileDataset::test_filedataset_raises_exception_when_load_prediction_data_is_empty
tests/test_baseclass.py::TestBaseClass::test_is_properties_works tests/test_baseclass.py::TestBaseClass::test_instantiate_model_with_non_estimator_pipeline_fails tests/test_baseclass.py::TestBaseClass::test_instantiate_model_with_other_object_fails tests/test_baseclass.py::TestBaseClass::test_make_prediction_errors_when_model_is_not_fitted tests/test_baseclass.py::TestBaseClass::test_make_prediction_with_regression_sqldataset_works_as_expected tests/test_baseclass.py::TestBaseClass::test_make_prediction_with_classification_sqldataset_works_as_expected tests/test_baseclass.py::TestBaseClass::test_make_prediction_errors_if_asked_for_proba_without_predict_proba_method tests/test_baseclass.py::TestBaseClass::test_make_prediction_returns_prediction_if_proba_is_false[False-0] tests/test_baseclass.py::TestBaseClass::test_make_prediction_returns_prediction_if_proba_is_false[True-5] tests/test_baseclass.py::TestBaseClass::test_make_prediction_returns_proba_if_proba_is_true[False-0] tests/test_baseclass.py::TestBaseClass::test_make_prediction_returns_proba_if_proba_is_true[True-5] tests/test_baseclass.py::TestBaseClass::test_default_metric_getter_works_as_expected_classifier tests/test_baseclass.py::TestBaseClass::test_default_metric_getter_works_as_expected_regressor tests/test_baseclass.py::TestBaseClass::test_default_metric_works_as_expected_without_pipeline tests/test_baseclass.py::TestBaseClass::test_default_metric_works_as_expected_with_pipeline tests/test_baseclass.py::TestBaseClass::test_regression_model_can_be_saved tests/test_baseclass.py::TestBaseClass::test_regression_model_filename_is_generated_correctly tests/test_baseclass.py::TestBaseClass::test_save_model_saves_pipeline_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_saves_logging_dir_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_with_prod_flag_saves_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_uses_default_storage_if_no_storage_is_passed tests/test_baseclass.py::TestBaseClass::test_can_load_production_estimator tests/test_baseclass.py::TestBaseClass::test_log_context_manager_works_as_expected tests/test_baseclass.py::TestBaseClass::test_log_context_manager_logs_when_scoring_model tests/test_baseclass.py::TestBaseClass::test_test_models_logs_when_given_dir tests/test_baseclass.py::TestBaseClass::test_dump_serializes_correctly_without_pipeline tests/test_baseclass.py::TestBaseClass::test_dump_serializes_correctly_with_pipeline tests/test_baseclass.py::TestBaseClass::test_to_dict_serializes_correctly_with_feature_union tests/test_baseclass.py::TestBaseClass::test_from_yaml_serializes_correctly_with_feature_union tests/test_baseclass.py::TestBaseClass::test_can_load_serialized_model_from_pipeline tests/test_baseclass.py::TestBaseClass::test_can_load_serialized_model_from_estimator tests/test_baseclass.py::TestTrainEstimator::test_train_model_sets_result_to_none tests/test_baseclass.py::TestTrainEstimator::test_train_model_followed_by_score_model_returns_correctly tests/test_baseclass.py::TestTrainEstimator::test_train_model_errors_correctly_when_not_scored tests/test_baseclass.py::TestTrainEstimator::test_can_score_estimator_with_no_y_value tests/test_baseclass.py::TestScoreEstimator::test_score_estimator_fails_if_no_train_test_data_available tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_specified_metric tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_default_metric tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_multiple_metrics tests/test_baseclass.py::TestModelSelection::test_model_selection_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_works_with_default_metric tests/test_baseclass.py::TestModelSelection::test_model_selection_works_with_multiple_metrics tests/test_baseclass.py::TestModelSelection::test_model_selection_with_nonstandard_metric_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_with_pipeline_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_refits_final_model tests/test_baseclass.py::TestGridSearch::test_gridsearch_model_returns_as_expected tests/test_baseclass.py::TestGridSearch::test_gridsearch_model_does_not_fail_when_run_twice tests/test_baseclass.py::TestGridSearch::test_prepare_gridsearch_estimators_has_different_parameters tests/test_baseclass.py::TestGridSearch::test_prepare_gridsearch_estimators_in_pipeline_has_different_parameters tests/test_baseclass.py::TestGridSearch::test_gridsearch_uses_default_metric tests/test_baseclass.py::TestGridSearch::test_gridsearch_can_take_multiple_metrics tests/test_baseclass.py::TestGridSearch::test_gridsearch_can_log_with_context_manager tests/test_data.py::TestDataset::test_repr_is_correct tests/test_data.py::TestDataset::test_dataset_x_attribute_access_works_correctly tests/test_data.py::TestDataset::test_dataset_y_attribute_access_works_correctly tests/test_data.py::TestDataset::test_cant_modify_x_and_y tests/test_data.py::TestDataset::test_dataset_has_validation_set_errors_correctly tests/test_data.py::TestDataset::test_cannot_instantiate_an_abstract_baseclass tests/test_data.py::TestDataset::test_can_copy_one_dataset_into_another tests/test_data.py::TestSqlDataset::test_sqldataset_repr_prints_correctly tests/test_data.py::TestSqlDataset::test_sqldataset_can_be_instantiated_with_engine_string tests/test_data.py::TestSqlDataset::test_sqldataset_errors_when_schema_is_defined_on_instantiation tests/test_data.py::TestSqlDataset::test_sqldataset_can_load_training_data tests/test_data.py::TestSqlDataset::test_sqldataset_can_load_prediction_data tests/test_data.py::TestSqlDataset::test_sqldataset_can_copy_to_another_sqldataset tests/test_data.py::TestSqlDataset::test_dump_data_throws_error_on_exec_failure tests/test_data.py::TestSqlDataset::test_load_data_throws_error_on_exec_failure tests/test_data.py::TestSqlDataset::test_setup_table_creates_schema_if_has_schema_returns_false tests/test_data.py::TestFileDataset::test_filedataset_repr_prints_correctly tests/test_data.py::TestFileDataset::test_load_data_works_as_expected tests/test_data.py::TestFileDataset::test_dump_data_works_as_expected tests/test_data.py::TestFileDataset::test_can_instantiate_filedataset tests/test_data.py::TestFileDataset::test_filedataset_extension_is_correct[test.csv-csv] tests/test_data.py::TestFileDataset::test_filedataset_extension_is_correct[test.parquet-parquet] tests/test_data.py::TestFileDataset::test_filedataset_errors_if_given_folder
src/ml_tooling/data/base_data.py src/ml_tooling/data/sql.py CHANGELOG.md
src/ml_tooling/data/base_data.py src/ml_tooling/data/sql.py CHANGELOG.md