We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
ARIMA and AutoARIMA should return pandas.NA when value cannot be predicted, e.g., first index in-sample for start=0, d=1
Update 2021-09-15 - re-opened since a recurring issue.

Not a bug, but an interface convention - currently ARIMA, AutoARIMA and other learners fail when asked for a prediction that is undefined, e.g., difference parameter d=1 and first in-sample index (start=0).

It would be more sensible if `pandas.NA` is returned instead for such indices, perhaps a warning should be raised too that `pandas.NA` are produced.

In first instance, we should update ARIMA and AutoARIMA with this convention.

---

**Describe the bug**

When using `Imputer` with `forecaster = AutoARIMA()`  the error "ValueError: In-sample predictions undefined for start=0 when d=1" is raised.

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-2d05f4822fd6> in <module>()
      4 
      5 transformer = Imputer(method="forecaster", forecaster=AutoARIMA()).fit(Z=df.value)
----> 6 y_hat = transformer.transform(Z=df.value)

/usr/local/lib/python3.7/dist-packages/sktime/transformations/series/impute.py in transform(self, Z, X)
    102             z_pred = forecaster.fit(
    103                 z.fillna(method="ffill").fillna(method="backfill")
--> 104             ).predict(fh=fh_ins)
    105             # fill with trend values
    106             z = z.fillna(value=z_pred)

/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/_sktime.py in predict(self, fh, X, return_pred_int, alpha)
    226         self.check_is_fitted()
    227         self._set_fh(fh)
--> 228         return self._predict(self.fh, X, return_pred_int=return_pred_int, alpha=alpha)
    229 
    230     def compute_pred_int(self, y_pred, alpha=DEFAULT_ALPHA):

/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/adapters/_pmdarima.py in _predict(self, fh, X, return_pred_int, alpha)
     58         # all values are in-sample
     59         elif fh.is_all_in_sample(self.cutoff):
---> 60             return self._predict_in_sample(fh_ins, **kwargs)
     61 
     62         # both in-sample and out-of-sample values

/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/adapters/_pmdarima.py in _predict_in_sample(self, fh, X, return_pred_int, alpha)
     80             X=X,
     81             return_conf_int=return_pred_int,
---> 82             alpha=alpha,
     83         )
     84 

/usr/local/lib/python3.7/dist-packages/pmdarima/utils/metaestimators.py in <lambda>(*args, **kwargs)
     51 
     52         # lambda, but not partial, allows help() to work with update_wrapper
---> 53         out = (lambda *args, **kwargs: self.fn(obj, *args, **kwargs))
     54         # update the docstring of the returned function
     55         update_wrapper(out, self.fn)

/usr/local/lib/python3.7/dist-packages/pmdarima/arima/auto.py in predict_in_sample(self, X, start, end, dynamic, return_conf_int, alpha, typ, **kwargs)
    242             return_conf_int=return_conf_int,
    243             alpha=alpha,
--> 244             typ=typ,
    245         )
    246 

/usr/local/lib/python3.7/dist-packages/pmdarima/arima/arima.py in predict_in_sample(self, X, start, end, dynamic, return_conf_int, alpha, **kwargs)
    584         if start is not None and start < d:
    585             raise ValueError("In-sample predictions undefined for start={0} "
--> 586                              "when d={1}".format(start, d))
    587 
    588         # if we fit with exog, make sure one was passed:

ValueError: In-sample predictions undefined for start=0 when d=1
```

**To Reproduce**

Data:
[df.csv](https://github.com/alan-turing-institute/sktime/files/6426660/df.csv) 

```python
import pandas as pd
from sktime.transformations.series.impute import Imputer
from sktime.forecasting.arima import AutoARIMA

df = pd.read_csv('df.csv')  
df['timestamp'] = pd.DatetimeIndex(df['timestamp']).to_period('D')
df = df.set_index('timestamp')

transformer = Imputer(method="forecaster", forecaster=AutoARIMA()).fit(Z=df.value)
y_hat = transformer.transform(Z=df.value)
```

**Expected behavior**

`transformer.transform(Z)` should return `Z` with NAn values replaced by AutoArima predictions.

**Additional context**

I think the problem is in `/usr/local/lib/python3.7/dist-packages/sktime/transformations/series/impute.py` in `transform(self, Z, X)`, line 100, because all the points are being passed to `fh_ins` instead of just the points with NaN.

```python
100             fh_ins = -np.arange(len(z))
101             # fill NaN before fitting with ffill and backfill (heuristic)
102             z_pred = forecaster.fit(
103                 z.fillna(method="ffill").fillna(method="backfill")
104             ).predict(fh=fh_ins)
105             # fill with trend values
106             z = z.fillna(value=z_pred)

```

**Versions**
<details>

```
System:
    python: 3.7.10 (default, Feb 20 2021, 21:17:23)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 19.3.1
   setuptools: 56.0.0
      sklearn: 0.24.2
       sktime: 0.6.0
  statsmodels: 0.12.2
        numpy: 1.19.5
        scipy: 1.4.1
       Cython: 0.29.22
       pandas: 1.1.5
   matplotlib: 3.2.2
       joblib: 1.0.1
        numba: 0.51.2
     pmdarima: 1.8.2
      tsfresh: None
```

</details>


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `sktime/forecasting/base/adapters/_pmdarima.py`
Content:
```
1 # -*- coding: utf-8 -*-
2 # !/usr/bin/env python3 -u
3 # copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
4 """Implements adapter for pmdarima forecasters to be used in sktime framework."""
5 
6 __author__ = ["Markus Löning", "Hongyi Yang"]
7 __all__ = ["_PmdArimaAdapter"]
8 
9 import pandas as pd
10 
11 from sktime.forecasting.base._base import DEFAULT_ALPHA
12 from sktime.forecasting.base import BaseForecaster
13 
14 
15 class _PmdArimaAdapter(BaseForecaster):
16     """Base class for interfacing pmdarima."""
17 
18     _tags = {
19         "ignores-exogeneous-X": True,
20         "capability:pred_int": True,
21         "requires-fh-in-fit": False,
22         "handles-missing-data": False,
23     }
24 
25     def __init__(self):
26         self._forecaster = None
27         super(_PmdArimaAdapter, self).__init__()
28 
29     def _instantiate_model(self):
30         raise NotImplementedError("abstract method")
31 
32     def _fit(self, y, X=None, fh=None, **fit_params):
33         """Fit to training data.
34 
35         Parameters
36         ----------
37         y : pd.Series
38             Target time series to which to fit the forecaster.
39         fh : int, list, np.array or ForecastingHorizon, optional (default=None)
40             The forecasters horizon with the steps ahead to to predict.
41         X : pd.DataFrame, optional (default=None)
42             Exogenous variables are ignored
43 
44         Returns
45         -------
46         self : returns an instance of self.
47         """
48         self._forecaster = self._instantiate_model()
49         self._forecaster.fit(y, X=X, **fit_params)
50         return self
51 
52     def _predict(self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA):
53         # distinguish between in-sample and out-of-sample prediction
54         fh_oos = fh.to_out_of_sample(self.cutoff)
55         fh_ins = fh.to_in_sample(self.cutoff)
56 
57         kwargs = {"X": X, "return_pred_int": return_pred_int, "alpha": alpha}
58 
59         # all values are out-of-sample
60         if fh.is_all_out_of_sample(self.cutoff):
61             return self._predict_fixed_cutoff(fh_oos, **kwargs)
62 
63         # all values are in-sample
64         elif fh.is_all_in_sample(self.cutoff):
65             return self._predict_in_sample(fh_ins, **kwargs)
66 
67         # both in-sample and out-of-sample values
68         else:
69             y_ins = self._predict_in_sample(fh_ins, **kwargs)
70             y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)
71             return y_ins.append(y_oos)
72 
73     def _predict_in_sample(
74         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA
75     ):
76         if isinstance(alpha, (list, tuple)):
77             raise NotImplementedError("multiple `alpha` values are not yet supported")
78 
79         # for in-sample predictions, pmdarima requires zero-based
80         # integer indicies
81         start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]
82         result = self._forecaster.predict_in_sample(
83             start=start,
84             end=end,
85             X=X,
86             return_conf_int=return_pred_int,
87             alpha=alpha,
88         )
89 
90         fh_abs = fh.to_absolute(self.cutoff)
91         fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)
92         if return_pred_int:
93             # unpack and format results
94             y_pred, pred_int = result
95             y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)
96             pred_int = pd.DataFrame(
97                 pred_int[fh_idx, :], index=fh_abs, columns=["lower", "upper"]
98             )
99             return y_pred, pred_int
100 
101         else:
102             return pd.Series(result[fh_idx], index=fh_abs)
103 
104     def _predict_fixed_cutoff(
105         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA
106     ):
107         # make prediction
108         n_periods = int(fh.to_relative(self.cutoff)[-1])
109         result = self._forecaster.predict(
110             n_periods=n_periods,
111             X=X,
112             return_conf_int=return_pred_int,
113             alpha=alpha,
114         )
115 
116         fh_abs = fh.to_absolute(self.cutoff)
117         fh_idx = fh.to_indexer(self.cutoff)
118         if return_pred_int:
119             y_pred, pred_int = result
120             y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)
121             pred_int = pd.DataFrame(
122                 pred_int[fh_idx, :], index=fh_abs, columns=["lower", "upper"]
123             )
124             return y_pred, pred_int
125         else:
126             return pd.Series(result[fh_idx], index=fh_abs)
127 
128     def get_fitted_params(self):
129         """Get fitted parameters.
130 
131         Returns
132         -------
133         fitted_params : dict
134         """
135         self.check_is_fitted()
136         names = self._get_fitted_param_names()
137         params = self._get_fitted_params()
138         fitted_params = {name: param for name, param in zip(names, params)}
139 
140         if hasattr(self._forecaster, "model_"):  # AutoARIMA
141             res = self._forecaster.model_.arima_res_
142         elif hasattr(self._forecaster, "arima_res_"):  # ARIMA
143             res = self._forecaster.arima_res_
144         else:
145             res = None
146 
147         for name in ["aic", "aicc", "bic", "hqic"]:
148             fitted_params[name] = getattr(res, name, None)
149 
150         return fitted_params
151 
152     def _get_fitted_params(self):
153         # Return parameter values under `arima_res_`
154         if hasattr(self._forecaster, "model_"):  # AutoARIMA
155             return self._forecaster.model_.arima_res_._results.params
156         elif hasattr(self._forecaster, "arima_res_"):  # ARIMA
157             return self._forecaster.arima_res_._results.params
158         else:
159             raise NotImplementedError()
160 
161     def _get_fitted_param_names(self):
162         # Return parameter names under `arima_res_`
163         if hasattr(self._forecaster, "model_"):  # AutoARIMA
164             return self._forecaster.model_.arima_res_._results.param_names
165         elif hasattr(self._forecaster, "arima_res_"):  # ARIMA
166             return self._forecaster.arima_res_._results.param_names
167         else:
168             raise NotImplementedError()
169 
170     def summary(self):
171         """Summary of the fitted model."""
172         return self._forecaster.summary()
173
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/sktime/forecasting/base/adapters/_pmdarima.py b/sktime/forecasting/base/adapters/_pmdarima.py
--- a/sktime/forecasting/base/adapters/_pmdarima.py
+++ b/sktime/forecasting/base/adapters/_pmdarima.py
@@ -7,7 +7,6 @@
 __all__ = ["_PmdArimaAdapter"]
 
 import pandas as pd
-
 from sktime.forecasting.base._base import DEFAULT_ALPHA
 from sktime.forecasting.base import BaseForecaster
 
@@ -66,9 +65,18 @@
 
         # both in-sample and out-of-sample values
         else:
-            y_ins = self._predict_in_sample(fh_ins, **kwargs)
-            y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)
-            return y_ins.append(y_oos)
+            if return_pred_int:
+                y_ins_pred, y_ins_pred_int = self._predict_in_sample(fh_ins, **kwargs)
+                y_oos_pred, y_oos_pred_int = self._predict_fixed_cutoff(
+                    fh_oos, **kwargs
+                )
+                return y_ins_pred.append(y_oos_pred), y_ins_pred_int.append(
+                    y_oos_pred_int
+                )
+            else:
+                y_ins = self._predict_in_sample(fh_ins, **kwargs)
+                y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)
+                return y_ins.append(y_oos)
 
     def _predict_in_sample(
         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA
@@ -76,9 +84,34 @@
         if isinstance(alpha, (list, tuple)):
             raise NotImplementedError("multiple `alpha` values are not yet supported")
 
-        # for in-sample predictions, pmdarima requires zero-based
-        # integer indicies
+        if hasattr(self, "order"):
+            diff_order = self.order[1]
+        else:
+            diff_order = self._forecaster.model_.order[1]
+
+        # Initialize return objects
+        fh_abs = fh.to_absolute(self.cutoff).to_numpy()
+        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)
+        y_pred = pd.Series(index=fh_abs)
+        pred_int = pd.DataFrame(index=fh_abs, columns=["lower", "upper"])
+
+        # for in-sample predictions, pmdarima requires zero-based integer indicies
         start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]
+        if start < 0:
+            # Can't forecasts earlier to train starting point
+            raise ValueError("Can't make predictions earlier to train starting point")
+        elif start < diff_order:
+            # Can't forecasts earlier to arima's differencing order
+            # But we return NaN for these supposedly forecastable points
+            start = diff_order
+            if end < start:
+                # since we might have forced `start` to surpass `end`
+                end = diff_order
+            # get rid of unforcastable points
+            fh_abs = fh_abs[fh_idx >= diff_order]
+            # reindex accordingly
+            fh_idx = fh_idx[fh_idx >= diff_order] - diff_order
+
         result = self._forecaster.predict_in_sample(
             start=start,
             end=end,
@@ -87,19 +120,16 @@
             alpha=alpha,
         )
 
-        fh_abs = fh.to_absolute(self.cutoff)
-        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)
         if return_pred_int:
-            # unpack and format results
-            y_pred, pred_int = result
-            y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)
-            pred_int = pd.DataFrame(
-                pred_int[fh_idx, :], index=fh_abs, columns=["lower", "upper"]
+            # unpack results
+            y_pred.loc[fh_abs], pred_int.loc[fh_abs] = (
+                result[0][fh_idx],
+                result[1][fh_idx, :],
             )
             return y_pred, pred_int
-
         else:
-            return pd.Series(result[fh_idx], index=fh_abs)
+            y_pred.loc[fh_abs] = result[fh_idx]
+            return y_pred
 
     def _predict_fixed_cutoff(
         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA

{"golden_diff": "diff --git a/sktime/forecasting/base/adapters/_pmdarima.py b/sktime/forecasting/base/adapters/_pmdarima.py\n--- a/sktime/forecasting/base/adapters/_pmdarima.py\n+++ b/sktime/forecasting/base/adapters/_pmdarima.py\n@@ -7,7 +7,6 @@\n __all__ = [\"_PmdArimaAdapter\"]\n \n import pandas as pd\n-\n from sktime.forecasting.base._base import DEFAULT_ALPHA\n from sktime.forecasting.base import BaseForecaster\n \n@@ -66,9 +65,18 @@\n \n         # both in-sample and out-of-sample values\n         else:\n-            y_ins = self._predict_in_sample(fh_ins, **kwargs)\n-            y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)\n-            return y_ins.append(y_oos)\n+            if return_pred_int:\n+                y_ins_pred, y_ins_pred_int = self._predict_in_sample(fh_ins, **kwargs)\n+                y_oos_pred, y_oos_pred_int = self._predict_fixed_cutoff(\n+                    fh_oos, **kwargs\n+                )\n+                return y_ins_pred.append(y_oos_pred), y_ins_pred_int.append(\n+                    y_oos_pred_int\n+                )\n+            else:\n+                y_ins = self._predict_in_sample(fh_ins, **kwargs)\n+                y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)\n+                return y_ins.append(y_oos)\n \n     def _predict_in_sample(\n         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n@@ -76,9 +84,34 @@\n         if isinstance(alpha, (list, tuple)):\n             raise NotImplementedError(\"multiple `alpha` values are not yet supported\")\n \n-        # for in-sample predictions, pmdarima requires zero-based\n-        # integer indicies\n+        if hasattr(self, \"order\"):\n+            diff_order = self.order[1]\n+        else:\n+            diff_order = self._forecaster.model_.order[1]\n+\n+        # Initialize return objects\n+        fh_abs = fh.to_absolute(self.cutoff).to_numpy()\n+        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)\n+        y_pred = pd.Series(index=fh_abs)\n+        pred_int = pd.DataFrame(index=fh_abs, columns=[\"lower\", \"upper\"])\n+\n+        # for in-sample predictions, pmdarima requires zero-based integer indicies\n         start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]\n+        if start < 0:\n+            # Can't forecasts earlier to train starting point\n+            raise ValueError(\"Can't make predictions earlier to train starting point\")\n+        elif start < diff_order:\n+            # Can't forecasts earlier to arima's differencing order\n+            # But we return NaN for these supposedly forecastable points\n+            start = diff_order\n+            if end < start:\n+                # since we might have forced `start` to surpass `end`\n+                end = diff_order\n+            # get rid of unforcastable points\n+            fh_abs = fh_abs[fh_idx >= diff_order]\n+            # reindex accordingly\n+            fh_idx = fh_idx[fh_idx >= diff_order] - diff_order\n+\n         result = self._forecaster.predict_in_sample(\n             start=start,\n             end=end,\n@@ -87,19 +120,16 @@\n             alpha=alpha,\n         )\n \n-        fh_abs = fh.to_absolute(self.cutoff)\n-        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)\n         if return_pred_int:\n-            # unpack and format results\n-            y_pred, pred_int = result\n-            y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)\n-            pred_int = pd.DataFrame(\n-                pred_int[fh_idx, :], index=fh_abs, columns=[\"lower\", \"upper\"]\n+            # unpack results\n+            y_pred.loc[fh_abs], pred_int.loc[fh_abs] = (\n+                result[0][fh_idx],\n+                result[1][fh_idx, :],\n             )\n             return y_pred, pred_int\n-\n         else:\n-            return pd.Series(result[fh_idx], index=fh_abs)\n+            y_pred.loc[fh_abs] = result[fh_idx]\n+            return y_pred\n \n     def _predict_fixed_cutoff(\n         self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n", "issue": "ARIMA and AutoARIMA should return pandas.NA when value cannot be predicted, e.g., first index in-sample for start=0, d=1\nUpdate 2021-09-15 - re-opened since a recurring issue.\r\n\r\nNot a bug, but an interface convention - currently ARIMA, AutoARIMA and other learners fail when asked for a prediction that is undefined, e.g., difference parameter d=1 and first in-sample index (start=0).\r\n\r\nIt would be more sensible if `pandas.NA` is returned instead for such indices, perhaps a warning should be raised too that `pandas.NA` are produced.\r\n\r\nIn first instance, we should update ARIMA and AutoARIMA with this convention.\r\n\r\n---\r\n\r\n**Describe the bug**\r\n\r\nWhen using `Imputer` with `forecaster = AutoARIMA()`  the error \"ValueError: In-sample predictions undefined for start=0 when d=1\" is raised.\r\n\r\n```\r\n---------------------------------------------------------------------------\r\nValueError                                Traceback (most recent call last)\r\n<ipython-input-11-2d05f4822fd6> in <module>()\r\n      4 \r\n      5 transformer = Imputer(method=\"forecaster\", forecaster=AutoARIMA()).fit(Z=df.value)\r\n----> 6 y_hat = transformer.transform(Z=df.value)\r\n\r\n/usr/local/lib/python3.7/dist-packages/sktime/transformations/series/impute.py in transform(self, Z, X)\r\n    102             z_pred = forecaster.fit(\r\n    103                 z.fillna(method=\"ffill\").fillna(method=\"backfill\")\r\n--> 104             ).predict(fh=fh_ins)\r\n    105             # fill with trend values\r\n    106             z = z.fillna(value=z_pred)\r\n\r\n/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/_sktime.py in predict(self, fh, X, return_pred_int, alpha)\r\n    226         self.check_is_fitted()\r\n    227         self._set_fh(fh)\r\n--> 228         return self._predict(self.fh, X, return_pred_int=return_pred_int, alpha=alpha)\r\n    229 \r\n    230     def compute_pred_int(self, y_pred, alpha=DEFAULT_ALPHA):\r\n\r\n/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/adapters/_pmdarima.py in _predict(self, fh, X, return_pred_int, alpha)\r\n     58         # all values are in-sample\r\n     59         elif fh.is_all_in_sample(self.cutoff):\r\n---> 60             return self._predict_in_sample(fh_ins, **kwargs)\r\n     61 \r\n     62         # both in-sample and out-of-sample values\r\n\r\n/usr/local/lib/python3.7/dist-packages/sktime/forecasting/base/adapters/_pmdarima.py in _predict_in_sample(self, fh, X, return_pred_int, alpha)\r\n     80             X=X,\r\n     81             return_conf_int=return_pred_int,\r\n---> 82             alpha=alpha,\r\n     83         )\r\n     84 \r\n\r\n/usr/local/lib/python3.7/dist-packages/pmdarima/utils/metaestimators.py in <lambda>(*args, **kwargs)\r\n     51 \r\n     52         # lambda, but not partial, allows help() to work with update_wrapper\r\n---> 53         out = (lambda *args, **kwargs: self.fn(obj, *args, **kwargs))\r\n     54         # update the docstring of the returned function\r\n     55         update_wrapper(out, self.fn)\r\n\r\n/usr/local/lib/python3.7/dist-packages/pmdarima/arima/auto.py in predict_in_sample(self, X, start, end, dynamic, return_conf_int, alpha, typ, **kwargs)\r\n    242             return_conf_int=return_conf_int,\r\n    243             alpha=alpha,\r\n--> 244             typ=typ,\r\n    245         )\r\n    246 \r\n\r\n/usr/local/lib/python3.7/dist-packages/pmdarima/arima/arima.py in predict_in_sample(self, X, start, end, dynamic, return_conf_int, alpha, **kwargs)\r\n    584         if start is not None and start < d:\r\n    585             raise ValueError(\"In-sample predictions undefined for start={0} \"\r\n--> 586                              \"when d={1}\".format(start, d))\r\n    587 \r\n    588         # if we fit with exog, make sure one was passed:\r\n\r\nValueError: In-sample predictions undefined for start=0 when d=1\r\n```\r\n\r\n**To Reproduce**\r\n\r\nData:\r\n[df.csv](https://github.com/alan-turing-institute/sktime/files/6426660/df.csv) \r\n\r\n```python\r\nimport pandas as pd\r\nfrom sktime.transformations.series.impute import Imputer\r\nfrom sktime.forecasting.arima import AutoARIMA\r\n\r\ndf = pd.read_csv('df.csv')  \r\ndf['timestamp'] = pd.DatetimeIndex(df['timestamp']).to_period('D')\r\ndf = df.set_index('timestamp')\r\n\r\ntransformer = Imputer(method=\"forecaster\", forecaster=AutoARIMA()).fit(Z=df.value)\r\ny_hat = transformer.transform(Z=df.value)\r\n```\r\n\r\n**Expected behavior**\r\n\r\n`transformer.transform(Z)` should return `Z` with NAn values replaced by AutoArima predictions.\r\n\r\n**Additional context**\r\n\r\nI think the problem is in `/usr/local/lib/python3.7/dist-packages/sktime/transformations/series/impute.py` in `transform(self, Z, X)`, line 100, because all the points are being passed to `fh_ins` instead of just the points with NaN.\r\n\r\n```python\r\n100             fh_ins = -np.arange(len(z))\r\n101             # fill NaN before fitting with ffill and backfill (heuristic)\r\n102             z_pred = forecaster.fit(\r\n103                 z.fillna(method=\"ffill\").fillna(method=\"backfill\")\r\n104             ).predict(fh=fh_ins)\r\n105             # fill with trend values\r\n106             z = z.fillna(value=z_pred)\r\n\r\n```\r\n\r\n**Versions**\r\n<details>\r\n\r\n```\r\nSystem:\r\n    python: 3.7.10 (default, Feb 20 2021, 21:17:23)  [GCC 7.5.0]\r\nexecutable: /usr/bin/python3\r\n   machine: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic\r\n\r\nPython dependencies:\r\n          pip: 19.3.1\r\n   setuptools: 56.0.0\r\n      sklearn: 0.24.2\r\n       sktime: 0.6.0\r\n  statsmodels: 0.12.2\r\n        numpy: 1.19.5\r\n        scipy: 1.4.1\r\n       Cython: 0.29.22\r\n       pandas: 1.1.5\r\n   matplotlib: 3.2.2\r\n       joblib: 1.0.1\r\n        numba: 0.51.2\r\n     pmdarima: 1.8.2\r\n      tsfresh: None\r\n```\r\n\r\n</details>\r\n\n", "before_files": [{"content": "# -*- coding: utf-8 -*-\n# !/usr/bin/env python3 -u\n# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)\n\"\"\"Implements adapter for pmdarima forecasters to be used in sktime framework.\"\"\"\n\n__author__ = [\"Markus L\u00f6ning\", \"Hongyi Yang\"]\n__all__ = [\"_PmdArimaAdapter\"]\n\nimport pandas as pd\n\nfrom sktime.forecasting.base._base import DEFAULT_ALPHA\nfrom sktime.forecasting.base import BaseForecaster\n\n\nclass _PmdArimaAdapter(BaseForecaster):\n    \"\"\"Base class for interfacing pmdarima.\"\"\"\n\n    _tags = {\n        \"ignores-exogeneous-X\": True,\n        \"capability:pred_int\": True,\n        \"requires-fh-in-fit\": False,\n        \"handles-missing-data\": False,\n    }\n\n    def __init__(self):\n        self._forecaster = None\n        super(_PmdArimaAdapter, self).__init__()\n\n    def _instantiate_model(self):\n        raise NotImplementedError(\"abstract method\")\n\n    def _fit(self, y, X=None, fh=None, **fit_params):\n        \"\"\"Fit to training data.\n\n        Parameters\n        ----------\n        y : pd.Series\n            Target time series to which to fit the forecaster.\n        fh : int, list, np.array or ForecastingHorizon, optional (default=None)\n            The forecasters horizon with the steps ahead to to predict.\n        X : pd.DataFrame, optional (default=None)\n            Exogenous variables are ignored\n\n        Returns\n        -------\n        self : returns an instance of self.\n        \"\"\"\n        self._forecaster = self._instantiate_model()\n        self._forecaster.fit(y, X=X, **fit_params)\n        return self\n\n    def _predict(self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA):\n        # distinguish between in-sample and out-of-sample prediction\n        fh_oos = fh.to_out_of_sample(self.cutoff)\n        fh_ins = fh.to_in_sample(self.cutoff)\n\n        kwargs = {\"X\": X, \"return_pred_int\": return_pred_int, \"alpha\": alpha}\n\n        # all values are out-of-sample\n        if fh.is_all_out_of_sample(self.cutoff):\n            return self._predict_fixed_cutoff(fh_oos, **kwargs)\n\n        # all values are in-sample\n        elif fh.is_all_in_sample(self.cutoff):\n            return self._predict_in_sample(fh_ins, **kwargs)\n\n        # both in-sample and out-of-sample values\n        else:\n            y_ins = self._predict_in_sample(fh_ins, **kwargs)\n            y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)\n            return y_ins.append(y_oos)\n\n    def _predict_in_sample(\n        self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n    ):\n        if isinstance(alpha, (list, tuple)):\n            raise NotImplementedError(\"multiple `alpha` values are not yet supported\")\n\n        # for in-sample predictions, pmdarima requires zero-based\n        # integer indicies\n        start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]\n        result = self._forecaster.predict_in_sample(\n            start=start,\n            end=end,\n            X=X,\n            return_conf_int=return_pred_int,\n            alpha=alpha,\n        )\n\n        fh_abs = fh.to_absolute(self.cutoff)\n        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)\n        if return_pred_int:\n            # unpack and format results\n            y_pred, pred_int = result\n            y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)\n            pred_int = pd.DataFrame(\n                pred_int[fh_idx, :], index=fh_abs, columns=[\"lower\", \"upper\"]\n            )\n            return y_pred, pred_int\n\n        else:\n            return pd.Series(result[fh_idx], index=fh_abs)\n\n    def _predict_fixed_cutoff(\n        self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n    ):\n        # make prediction\n        n_periods = int(fh.to_relative(self.cutoff)[-1])\n        result = self._forecaster.predict(\n            n_periods=n_periods,\n            X=X,\n            return_conf_int=return_pred_int,\n            alpha=alpha,\n        )\n\n        fh_abs = fh.to_absolute(self.cutoff)\n        fh_idx = fh.to_indexer(self.cutoff)\n        if return_pred_int:\n            y_pred, pred_int = result\n            y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)\n            pred_int = pd.DataFrame(\n                pred_int[fh_idx, :], index=fh_abs, columns=[\"lower\", \"upper\"]\n            )\n            return y_pred, pred_int\n        else:\n            return pd.Series(result[fh_idx], index=fh_abs)\n\n    def get_fitted_params(self):\n        \"\"\"Get fitted parameters.\n\n        Returns\n        -------\n        fitted_params : dict\n        \"\"\"\n        self.check_is_fitted()\n        names = self._get_fitted_param_names()\n        params = self._get_fitted_params()\n        fitted_params = {name: param for name, param in zip(names, params)}\n\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            res = self._forecaster.model_.arima_res_\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            res = self._forecaster.arima_res_\n        else:\n            res = None\n\n        for name in [\"aic\", \"aicc\", \"bic\", \"hqic\"]:\n            fitted_params[name] = getattr(res, name, None)\n\n        return fitted_params\n\n    def _get_fitted_params(self):\n        # Return parameter values under `arima_res_`\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            return self._forecaster.model_.arima_res_._results.params\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            return self._forecaster.arima_res_._results.params\n        else:\n            raise NotImplementedError()\n\n    def _get_fitted_param_names(self):\n        # Return parameter names under `arima_res_`\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            return self._forecaster.model_.arima_res_._results.param_names\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            return self._forecaster.arima_res_._results.param_names\n        else:\n            raise NotImplementedError()\n\n    def summary(self):\n        \"\"\"Summary of the fitted model.\"\"\"\n        return self._forecaster.summary()\n", "path": "sktime/forecasting/base/adapters/_pmdarima.py"}], "after_files": [{"content": "# -*- coding: utf-8 -*-\n# !/usr/bin/env python3 -u\n# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)\n\"\"\"Implements adapter for pmdarima forecasters to be used in sktime framework.\"\"\"\n\n__author__ = [\"Markus L\u00f6ning\", \"Hongyi Yang\"]\n__all__ = [\"_PmdArimaAdapter\"]\n\nimport pandas as pd\nfrom sktime.forecasting.base._base import DEFAULT_ALPHA\nfrom sktime.forecasting.base import BaseForecaster\n\n\nclass _PmdArimaAdapter(BaseForecaster):\n    \"\"\"Base class for interfacing pmdarima.\"\"\"\n\n    _tags = {\n        \"ignores-exogeneous-X\": True,\n        \"capability:pred_int\": True,\n        \"requires-fh-in-fit\": False,\n        \"handles-missing-data\": False,\n    }\n\n    def __init__(self):\n        self._forecaster = None\n        super(_PmdArimaAdapter, self).__init__()\n\n    def _instantiate_model(self):\n        raise NotImplementedError(\"abstract method\")\n\n    def _fit(self, y, X=None, fh=None, **fit_params):\n        \"\"\"Fit to training data.\n\n        Parameters\n        ----------\n        y : pd.Series\n            Target time series to which to fit the forecaster.\n        fh : int, list, np.array or ForecastingHorizon, optional (default=None)\n            The forecasters horizon with the steps ahead to to predict.\n        X : pd.DataFrame, optional (default=None)\n            Exogenous variables are ignored\n\n        Returns\n        -------\n        self : returns an instance of self.\n        \"\"\"\n        self._forecaster = self._instantiate_model()\n        self._forecaster.fit(y, X=X, **fit_params)\n        return self\n\n    def _predict(self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA):\n        # distinguish between in-sample and out-of-sample prediction\n        fh_oos = fh.to_out_of_sample(self.cutoff)\n        fh_ins = fh.to_in_sample(self.cutoff)\n\n        kwargs = {\"X\": X, \"return_pred_int\": return_pred_int, \"alpha\": alpha}\n\n        # all values are out-of-sample\n        if fh.is_all_out_of_sample(self.cutoff):\n            return self._predict_fixed_cutoff(fh_oos, **kwargs)\n\n        # all values are in-sample\n        elif fh.is_all_in_sample(self.cutoff):\n            return self._predict_in_sample(fh_ins, **kwargs)\n\n        # both in-sample and out-of-sample values\n        else:\n            if return_pred_int:\n                y_ins_pred, y_ins_pred_int = self._predict_in_sample(fh_ins, **kwargs)\n                y_oos_pred, y_oos_pred_int = self._predict_fixed_cutoff(\n                    fh_oos, **kwargs\n                )\n                return y_ins_pred.append(y_oos_pred), y_ins_pred_int.append(\n                    y_oos_pred_int\n                )\n            else:\n                y_ins = self._predict_in_sample(fh_ins, **kwargs)\n                y_oos = self._predict_fixed_cutoff(fh_oos, **kwargs)\n                return y_ins.append(y_oos)\n\n    def _predict_in_sample(\n        self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n    ):\n        if isinstance(alpha, (list, tuple)):\n            raise NotImplementedError(\"multiple `alpha` values are not yet supported\")\n\n        if hasattr(self, \"order\"):\n            diff_order = self.order[1]\n        else:\n            diff_order = self._forecaster.model_.order[1]\n\n        # Initialize return objects\n        fh_abs = fh.to_absolute(self.cutoff).to_numpy()\n        fh_idx = fh.to_indexer(self.cutoff, from_cutoff=False)\n        y_pred = pd.Series(index=fh_abs)\n        pred_int = pd.DataFrame(index=fh_abs, columns=[\"lower\", \"upper\"])\n\n        # for in-sample predictions, pmdarima requires zero-based integer indicies\n        start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]\n        if start < 0:\n            # Can't forecasts earlier to train starting point\n            raise ValueError(\"Can't make predictions earlier to train starting point\")\n        elif start < diff_order:\n            # Can't forecasts earlier to arima's differencing order\n            # But we return NaN for these supposedly forecastable points\n            start = diff_order\n            if end < start:\n                # since we might have forced `start` to surpass `end`\n                end = diff_order\n            # get rid of unforcastable points\n            fh_abs = fh_abs[fh_idx >= diff_order]\n            # reindex accordingly\n            fh_idx = fh_idx[fh_idx >= diff_order] - diff_order\n\n        result = self._forecaster.predict_in_sample(\n            start=start,\n            end=end,\n            X=X,\n            return_conf_int=return_pred_int,\n            alpha=alpha,\n        )\n\n        if return_pred_int:\n            # unpack results\n            y_pred.loc[fh_abs], pred_int.loc[fh_abs] = (\n                result[0][fh_idx],\n                result[1][fh_idx, :],\n            )\n            return y_pred, pred_int\n        else:\n            y_pred.loc[fh_abs] = result[fh_idx]\n            return y_pred\n\n    def _predict_fixed_cutoff(\n        self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA\n    ):\n        # make prediction\n        n_periods = int(fh.to_relative(self.cutoff)[-1])\n        result = self._forecaster.predict(\n            n_periods=n_periods,\n            X=X,\n            return_conf_int=return_pred_int,\n            alpha=alpha,\n        )\n\n        fh_abs = fh.to_absolute(self.cutoff)\n        fh_idx = fh.to_indexer(self.cutoff)\n        if return_pred_int:\n            y_pred, pred_int = result\n            y_pred = pd.Series(y_pred[fh_idx], index=fh_abs)\n            pred_int = pd.DataFrame(\n                pred_int[fh_idx, :], index=fh_abs, columns=[\"lower\", \"upper\"]\n            )\n            return y_pred, pred_int\n        else:\n            return pd.Series(result[fh_idx], index=fh_abs)\n\n    def get_fitted_params(self):\n        \"\"\"Get fitted parameters.\n\n        Returns\n        -------\n        fitted_params : dict\n        \"\"\"\n        self.check_is_fitted()\n        names = self._get_fitted_param_names()\n        params = self._get_fitted_params()\n        fitted_params = {name: param for name, param in zip(names, params)}\n\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            res = self._forecaster.model_.arima_res_\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            res = self._forecaster.arima_res_\n        else:\n            res = None\n\n        for name in [\"aic\", \"aicc\", \"bic\", \"hqic\"]:\n            fitted_params[name] = getattr(res, name, None)\n\n        return fitted_params\n\n    def _get_fitted_params(self):\n        # Return parameter values under `arima_res_`\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            return self._forecaster.model_.arima_res_._results.params\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            return self._forecaster.arima_res_._results.params\n        else:\n            raise NotImplementedError()\n\n    def _get_fitted_param_names(self):\n        # Return parameter names under `arima_res_`\n        if hasattr(self._forecaster, \"model_\"):  # AutoARIMA\n            return self._forecaster.model_.arima_res_._results.param_names\n        elif hasattr(self._forecaster, \"arima_res_\"):  # ARIMA\n            return self._forecaster.arima_res_._results.param_names\n        else:\n            raise NotImplementedError()\n\n    def summary(self):\n        \"\"\"Summary of the fitted model.\"\"\"\n        return self._forecaster.summary()\n", "path": "sktime/forecasting/base/adapters/_pmdarima.py"}]}