codecov[bot]: # [Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=h1) Report
> Merging [#454](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=desc) into [develop](https://codecov.io/gh/andersbogsnes/ml_tooling/commit/9d950c9b31856c0771094f9ee46e6d34566c6e10?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454/graphs/tree.svg?width=650&token=ucKtL3BlxO&height=150&src=pr)](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=tree)

```diff
@@          Coverage Diff           @@
##           develop   #454   +/-   ##
======================================
  Coverage      100%   100%           
======================================
  Files           55     55           
  Lines         1412   1412           
  Branches       164    164           
======================================
  Hits          1412   1412
```


| [Impacted Files](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/ml\_tooling/data/sql.py](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454/diff?src=pr&el=tree#diff-c3JjL21sX3Rvb2xpbmcvZGF0YS9zcWwucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [src/ml\_tooling/baseclass.py](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454/diff?src=pr&el=tree#diff-c3JjL21sX3Rvb2xpbmcvYmFzZWNsYXNzLnB5) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=footer). Last update [9d950c9...9baf40e](https://codecov.io/gh/andersbogsnes/ml_tooling/pull/454?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5bacac4..04dce69 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,7 @@ human friendly manner
 - Dataset now verifies that `load_training_data` and `load_prediction_data` do not return empty
 - Added a missing data visualization to `Dataset.plot`
 - FillNA now accepts a `is_nan`flag which adds a flag indicating that a value was missing
+- `Model.make_prediction` now accepts a `use_cache`flag to score everything in cached `.x`
 
 # v0.10.2
 - Fixed type inferences from data to sql in _load_data
diff --git a/src/ml_tooling/baseclass.py b/src/ml_tooling/baseclass.py
index 239a12e..e62a5ff 100644
--- a/src/ml_tooling/baseclass.py
+++ b/src/ml_tooling/baseclass.py
@@ -227,6 +227,7 @@ class Model:
         *args,
         proba: bool = False,
         use_index: bool = False,
+        use_cache: bool = False,
         **kwargs,
     ) -> pd.DataFrame:
         """
@@ -246,6 +247,10 @@ class Model:
         use_index: bool
             Whether the index from the prediction data should be used for the result.
 
+        use_cache: bool
+            Whether or not to use the cached data in dataset to make predictions.
+            Useful for seeing probability distributions of the model
+
         Returns
         -------
         pd.DataFrame
@@ -255,7 +260,9 @@ class Model:
             raise MLToolingError(
                 f"{self.estimator_name} does not have a `predict_proba` method"
             )
-        x = data._load_prediction_data(*args, **kwargs)
+
+        x = data.x if use_cache else data._load_prediction_data(*args, **kwargs)
+
         try:
             if proba:
                 data = self.estimator.predict_proba(x)
diff --git a/src/ml_tooling/data/sql.py b/src/ml_tooling/data/sql.py
index d17aed8..03ef3d1 100644
--- a/src/ml_tooling/data/sql.py
+++ b/src/ml_tooling/data/sql.py
@@ -107,7 +107,7 @@ class SQLDataset(Dataset, metaclass=abc.ABCMeta):
         with self.create_connection() as conn:
             return super()._load_prediction_data(*args, conn=conn, **kwargs)
 
-    def _dump_data(self) -> pd.DataFrame:
+    def _dump_data(self, use_cache=False) -> pd.DataFrame:
         """
         Reads the underlying SQL table and returns a DataFrame
 
@@ -116,6 +116,7 @@ class SQLDataset(Dataset, metaclass=abc.ABCMeta):
         pd.DataFrame
 
         """
+
         logger.info(f"Dumping data from {self.table}")
         logger.debug(f"Dumping data from {self.engine}/{self.table.name}")
         stmt = sa.select([self.table])

ENH: Datasets should check cached `.x` and `.y` attributes when dumping data
When dumping data from one Dataset to another, the dataset always fetches a new copy of the data and dumps it. This can potentially take time and is not necessary if the data is already cached

**Describe the solution you'd like**
All data related functionality, such as `make_prediction` or  `dump_data` should check for `x` and `y` and copy those instead of getting a fresh dataset. Should be able to disable this behaviour with a `use_cache` flag
diff --git a/tests/test_baseclass.py b/tests/test_baseclass.py
index 0e82975..77b3022 100644
--- a/tests/test_baseclass.py
+++ b/tests/test_baseclass.py
@@ -53,85 +53,6 @@ class TestBaseClass:
         ):
             Model({})
 
-    def test_make_prediction_errors_when_model_is_not_fitted(self, train_iris_dataset):
-        with pytest.raises(MLToolingError, match="You haven't fitted the estimator"):
-            model = Model(LinearRegression())
-            model.make_prediction(train_iris_dataset, 5)
-
-    def test_make_prediction_with_regression_sqldataset_works_as_expected(
-        self, boston_sqldataset, loaded_boston_db
-    ):
-        dataset = boston_sqldataset(loaded_boston_db, schema=None)
-        dataset.create_train_test(stratify=False)
-        model = Model(LinearRegression())
-        model.train_estimator(dataset)
-
-        result = model.make_prediction(dataset, 0)
-
-        assert result.shape == (1, 1)
-        assert result.columns.tolist() == ["Prediction"]
-
-    def test_make_prediction_with_classification_sqldataset_works_as_expected(
-        self, iris_sqldataset, loaded_iris_db
-    ):
-        dataset = iris_sqldataset(loaded_iris_db, schema=None)
-        dataset.create_train_test()
-        model = Model(LogisticRegression(solver="lbfgs"))
-        model.train_estimator(dataset)
-
-        result = model.make_prediction(dataset, 0, proba=True)
-
-        assert result.shape == (1, 2)
-        assert result.columns.tolist() == ["Probability Class 0", "Probability Class 1"]
-
-    def test_make_prediction_errors_if_asked_for_proba_without_predict_proba_method(
-        self, train_iris_dataset
-    ):
-        with pytest.raises(
-            MLToolingError, match="LinearRegression does not have a `predict_proba`"
-        ):
-            model = Model(LinearRegression())
-            model.train_estimator(train_iris_dataset)
-            model.make_prediction(train_iris_dataset, 5, proba=True)
-
-    @pytest.mark.parametrize("use_index, expected_index", [(False, 0), (True, 5)])
-    def test_make_prediction_returns_prediction_if_proba_is_false(
-        self,
-        classifier: Model,
-        use_index: bool,
-        expected_index: int,
-        train_iris_dataset,
-    ):
-        results = classifier.make_prediction(
-            train_iris_dataset, 5, proba=False, use_index=use_index
-        )
-        assert isinstance(results, pd.DataFrame)
-        assert 2 == results.ndim
-        assert np.all((results == 1) | (results == 0))
-        assert np.all(np.sum(results, axis=1) == 0)
-        assert results.index == pd.RangeIndex(
-            start=expected_index, stop=expected_index + 1, step=1
-        )
-
-    @pytest.mark.parametrize("use_index, expected_index", [(False, 0), (True, 5)])
-    def test_make_prediction_returns_proba_if_proba_is_true(
-        self,
-        classifier: Model,
-        use_index: bool,
-        expected_index: int,
-        train_iris_dataset,
-    ):
-        results = classifier.make_prediction(
-            train_iris_dataset, 5, proba=True, use_index=use_index
-        )
-        assert isinstance(results, pd.DataFrame)
-        assert 2 == results.ndim
-        assert np.all((results <= 1) & (results >= 0))
-        assert np.all(np.sum(results, axis=1) == 1)
-        assert results.index == pd.RangeIndex(
-            start=expected_index, stop=expected_index + 1, step=1
-        )
-
     def test_default_metric_getter_works_as_expected_classifier(self):
         rf = Model(RandomForestClassifier(n_estimators=10))
         assert rf.config.CLASSIFIER_METRIC == "accuracy"
@@ -723,3 +644,92 @@ class TestGridSearch:
             _, _ = classifier.gridsearch(
                 train_iris_dataset, param_grid={"clf__penalty": ["l1", "l2"]}
             )
+
+
+class TestMakePrediction:
+    def test_make_prediction_errors_when_model_is_not_fitted(self, train_iris_dataset):
+        model = Model(LinearRegression())
+        with pytest.raises(MLToolingError, match="You haven't fitted the estimator"):
+            model.make_prediction(train_iris_dataset, 5)
+
+    def test_make_prediction_with_regression_sqldataset_works_as_expected(
+        self, boston_sqldataset, loaded_boston_db
+    ):
+        dataset = boston_sqldataset(loaded_boston_db, schema=None)
+        dataset.create_train_test(stratify=False)
+        model = Model(LinearRegression())
+        model.train_estimator(dataset)
+
+        result = model.make_prediction(dataset, 0)
+
+        assert result.shape == (1, 1)
+        assert result.columns.tolist() == ["Prediction"]
+
+    def test_make_prediction_with_classification_sqldataset_works_as_expected(
+        self, iris_sqldataset, loaded_iris_db
+    ):
+        dataset = iris_sqldataset(loaded_iris_db, schema=None)
+        dataset.create_train_test()
+        model = Model(LogisticRegression(solver="lbfgs"))
+        model.train_estimator(dataset)
+
+        result = model.make_prediction(dataset, 0, proba=True)
+
+        assert result.shape == (1, 2)
+        assert result.columns.tolist() == ["Probability Class 0", "Probability Class 1"]
+
+    def test_make_prediction_errors_if_asked_for_proba_without_predict_proba_method(
+        self, train_iris_dataset
+    ):
+        with pytest.raises(
+            MLToolingError, match="LinearRegression does not have a `predict_proba`"
+        ):
+            model = Model(LinearRegression())
+            model.train_estimator(train_iris_dataset)
+            model.make_prediction(train_iris_dataset, 5, proba=True)
+
+    @pytest.mark.parametrize("use_index, expected_index", [(False, 0), (True, 5)])
+    def test_make_prediction_returns_prediction_if_proba_is_false(
+        self,
+        classifier: Model,
+        use_index: bool,
+        expected_index: int,
+        train_iris_dataset,
+    ):
+        results = classifier.make_prediction(
+            train_iris_dataset, 5, proba=False, use_index=use_index
+        )
+        assert isinstance(results, pd.DataFrame)
+        assert 2 == results.ndim
+        assert np.all((results == 1) | (results == 0))
+        assert np.all(np.sum(results, axis=1) == 0)
+        assert results.index == pd.RangeIndex(
+            start=expected_index, stop=expected_index + 1, step=1
+        )
+
+    @pytest.mark.parametrize("use_index, expected_index", [(False, 0), (True, 5)])
+    def test_make_prediction_returns_proba_if_proba_is_true(
+        self,
+        classifier: Model,
+        use_index: bool,
+        expected_index: int,
+        train_iris_dataset,
+    ):
+        results = classifier.make_prediction(
+            train_iris_dataset, 5, proba=True, use_index=use_index
+        )
+        assert isinstance(results, pd.DataFrame)
+        assert 2 == results.ndim
+        assert np.all((results <= 1) & (results >= 0))
+        assert np.all(np.sum(results, axis=1) == 1)
+        assert results.index == pd.RangeIndex(
+            start=expected_index, stop=expected_index + 1, step=1
+        )
+
+    def test_make_prediction_uses_cache_if_set(self, classifier, iris_df):
+        mock_dataset = MagicMock(spec=Dataset)
+        mock_dataset.x = iris_df.head(5).drop(columns="target")
+
+        result = classifier.make_prediction(mock_dataset, use_cache=True)
+        assert len(result) == 5
+        mock_dataset._load_prediction_data.assert_not_called()

pip install -e .[dev]
parse_log_pytest
requirements.txt
pytest
3.7
requirements.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
alabaster==0.7.13
attrs==19.3.0
Babel==2.14.0
cachetools==5.5.2
certifi @ file:///croot/certifi_1671487769961/work/certifi
cfgv==3.3.1
chardet==5.2.0
charset-normalizer==3.4.1
colorama==0.4.6
coverage==7.2.7
cycler==0.10.0
distlib==0.3.9
docutils==0.19
exceptiongroup==1.2.2
filelock==3.12.2
identify==2.5.24
idna==3.10
imagesize==1.4.1
importlib-metadata==6.7.0
iniconfig==2.0.0
Jinja2==3.1.6
joblib==0.14.1
kiwisolver==1.1.0
MarkupSafe==2.1.5
matplotlib==3.2.0
-e git+https://github.com/andersbogsnes/ml_tooling.git@d028b0fb4c1bdfadc6270cf716ea231bd01aea30#egg=ml_tooling
nodeenv==1.9.1
numpy==1.18.1
packaging==24.0
pandas==1.0.1
platformdirs==4.0.0
pluggy==1.2.0
pre-commit==2.21.0
Pygments==2.17.2
pyparsing==2.4.6
pyproject-api==1.5.3
pytest==7.4.4
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3
requests==2.31.0
scikit-learn==0.22.2.post1
scipy==1.4.1
six==1.14.0
snowballstemmer==2.2.0
Sphinx==5.3.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
SQLAlchemy==1.3.13
tomli==2.0.1
tox==4.8.0
typing_extensions==4.7.1
urllib3==2.0.7
virtualenv==20.26.6
zipp==3.15.0

name: ml_tooling
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - certifi=2022.12.7=py37h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=1.1.1w=h7f8727e_0
  - pip=22.3.1=py37h06a4308_0
  - python=3.7.16=h7a1cb2a_0
  - readline=8.2=h5eee18b_0
  - setuptools=65.6.3=py37h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - wheel=0.38.4=py37h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - alabaster==0.7.13
      - attrs==19.3.0
      - babel==2.14.0
      - cachetools==5.5.2
      - cfgv==3.3.1
      - chardet==5.2.0
      - charset-normalizer==3.4.1
      - colorama==0.4.6
      - coverage==7.2.7
      - cycler==0.10.0
      - distlib==0.3.9
      - docutils==0.19
      - exceptiongroup==1.2.2
      - filelock==3.12.2
      - identify==2.5.24
      - idna==3.10
      - imagesize==1.4.1
      - importlib-metadata==6.7.0
      - iniconfig==2.0.0
      - jinja2==3.1.6
      - joblib==0.14.1
      - kiwisolver==1.1.0
      - markupsafe==2.1.5
      - matplotlib==3.2.0
      - nodeenv==1.9.1
      - numpy==1.18.1
      - packaging==24.0
      - pandas==1.0.1
      - platformdirs==4.0.0
      - pluggy==1.2.0
      - pre-commit==2.21.0
      - pygments==2.17.2
      - pyparsing==2.4.6
      - pyproject-api==1.5.3
      - pytest==7.4.4
      - python-dateutil==2.8.1
      - pytz==2019.3
      - pyyaml==5.3
      - requests==2.31.0
      - scikit-learn==0.22.2.post1
      - scipy==1.4.1
      - six==1.14.0
      - snowballstemmer==2.2.0
      - sphinx==5.3.0
      - sphinxcontrib-applehelp==1.0.2
      - sphinxcontrib-devhelp==1.0.2
      - sphinxcontrib-htmlhelp==2.0.0
      - sphinxcontrib-jsmath==1.0.1
      - sphinxcontrib-qthelp==1.0.3
      - sphinxcontrib-serializinghtml==1.1.5
      - sqlalchemy==1.3.13
      - tomli==2.0.1
      - tox==4.8.0
      - typing-extensions==4.7.1
      - urllib3==2.0.7
      - virtualenv==20.26.6
      - zipp==3.15.0
prefix: /opt/conda/envs/ml_tooling

tests/test_baseclass.py::TestMakePrediction::test_make_prediction_uses_cache_if_set
tests/test_baseclass.py::TestBaseClass::test_is_properties_works tests/test_baseclass.py::TestBaseClass::test_instantiate_model_with_non_estimator_pipeline_fails tests/test_baseclass.py::TestBaseClass::test_instantiate_model_with_other_object_fails tests/test_baseclass.py::TestBaseClass::test_default_metric_getter_works_as_expected_classifier tests/test_baseclass.py::TestBaseClass::test_default_metric_getter_works_as_expected_regressor tests/test_baseclass.py::TestBaseClass::test_default_metric_works_as_expected_without_pipeline tests/test_baseclass.py::TestBaseClass::test_default_metric_works_as_expected_with_pipeline tests/test_baseclass.py::TestBaseClass::test_regression_model_can_be_saved tests/test_baseclass.py::TestBaseClass::test_regression_model_filename_is_generated_correctly tests/test_baseclass.py::TestBaseClass::test_save_model_saves_pipeline_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_saves_logging_dir_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_with_prod_flag_saves_correctly tests/test_baseclass.py::TestBaseClass::test_save_estimator_uses_default_storage_if_no_storage_is_passed tests/test_baseclass.py::TestBaseClass::test_can_load_production_estimator tests/test_baseclass.py::TestBaseClass::test_log_context_manager_works_as_expected tests/test_baseclass.py::TestBaseClass::test_log_context_manager_logs_when_scoring_model tests/test_baseclass.py::TestBaseClass::test_test_models_logs_when_given_dir tests/test_baseclass.py::TestBaseClass::test_dump_serializes_correctly_without_pipeline tests/test_baseclass.py::TestBaseClass::test_dump_serializes_correctly_with_pipeline tests/test_baseclass.py::TestBaseClass::test_to_dict_serializes_correctly_with_feature_union tests/test_baseclass.py::TestBaseClass::test_from_yaml_serializes_correctly_with_feature_union tests/test_baseclass.py::TestBaseClass::test_can_load_serialized_model_from_pipeline tests/test_baseclass.py::TestBaseClass::test_can_load_serialized_model_from_estimator tests/test_baseclass.py::TestTrainEstimator::test_train_model_sets_result_to_none tests/test_baseclass.py::TestTrainEstimator::test_train_model_followed_by_score_model_returns_correctly tests/test_baseclass.py::TestTrainEstimator::test_train_model_errors_correctly_when_not_scored tests/test_baseclass.py::TestTrainEstimator::test_can_score_estimator_with_no_y_value tests/test_baseclass.py::TestScoreEstimator::test_score_estimator_fails_if_no_train_test_data_available tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_specified_metric tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_default_metric tests/test_baseclass.py::TestScoreEstimator::test_can_score_estimator_with_multiple_metrics tests/test_baseclass.py::TestModelSelection::test_model_selection_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_works_with_default_metric tests/test_baseclass.py::TestModelSelection::test_model_selection_works_with_multiple_metrics tests/test_baseclass.py::TestModelSelection::test_model_selection_with_nonstandard_metric_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_with_pipeline_works_as_expected tests/test_baseclass.py::TestModelSelection::test_model_selection_refits_final_model tests/test_baseclass.py::TestGridSearch::test_gridsearch_model_returns_as_expected tests/test_baseclass.py::TestGridSearch::test_gridsearch_model_does_not_fail_when_run_twice tests/test_baseclass.py::TestGridSearch::test_prepare_gridsearch_estimators_has_different_parameters tests/test_baseclass.py::TestGridSearch::test_prepare_gridsearch_estimators_in_pipeline_has_different_parameters tests/test_baseclass.py::TestGridSearch::test_gridsearch_uses_default_metric tests/test_baseclass.py::TestGridSearch::test_gridsearch_can_take_multiple_metrics tests/test_baseclass.py::TestGridSearch::test_gridsearch_can_log_with_context_manager tests/test_baseclass.py::TestMakePrediction::test_make_prediction_errors_when_model_is_not_fitted tests/test_baseclass.py::TestMakePrediction::test_make_prediction_with_regression_sqldataset_works_as_expected tests/test_baseclass.py::TestMakePrediction::test_make_prediction_with_classification_sqldataset_works_as_expected tests/test_baseclass.py::TestMakePrediction::test_make_prediction_errors_if_asked_for_proba_without_predict_proba_method tests/test_baseclass.py::TestMakePrediction::test_make_prediction_returns_prediction_if_proba_is_false[False-0] tests/test_baseclass.py::TestMakePrediction::test_make_prediction_returns_prediction_if_proba_is_false[True-5] tests/test_baseclass.py::TestMakePrediction::test_make_prediction_returns_proba_if_proba_is_true[False-0] tests/test_baseclass.py::TestMakePrediction::test_make_prediction_returns_proba_if_proba_is_true[True-5]
src/ml_tooling/baseclass.py src/ml_tooling/data/sql.py CHANGELOG.md
src/ml_tooling/baseclass.py src/ml_tooling/data/sql.py CHANGELOG.md