title: use smoothing-spline effectively for predictions
body: <p>I have a time series given by two vectors</p>
  
  <pre><code>x_1 = np.array([119., 122., 129., 142., 159., 162., 179., 182., 189., 194., 197.,
                199., 202., 204., 209., 214., 217., 219., 222., 224., 229., 234.,
                237., 242., 244., 257., 269., 274., 277., 279., 299., 327., 339.])
  
   y_1 = np.array([0.70, 0.78, 0.80, 0.84, 0.84, 0.83, 0.15, 0.16, 0.26, 0.42,
                 0.52, 0.53, 0.60, 0.64, 0.69, 0.56, 0.58, 0.57, 0.60, 0.77,
                 0.77, 0.72, 0.77, 0.78 , 0.75, 0.19, 0.24, 0.27, 0.28, 0.29,
                 0.31, 0.28, 0.26])
  </code></pre>
  
  <p>I use smoothing spline to fit a model to the time series:</p>
  
  <pre><code>x_basis = cr(x_1, df=freedom_df, constraints='center')
  
  # Fit model to the data
  model = LinearRegression().fit(x_basis, y_1)
  
  # Get estimates
  y_prediction = model.predict(x_basis)
  </code></pre>
  
  <p>so far so good.</p>
  
  <p>In fact I have so many different time series. Each have different length. so, the <code>x_basis</code>
  and consequently the <code>y_prediction</code> will have different length.
  since my goal is to feed them into a machine learning model later, they all, as an input, must have the same length. I want them to be every two weeks:</p>
  
  <pre><code>X_const = np.arange(start=1, stop=365, step=14, dtype=None)
  </code></pre>
  
  <p>How can I make a good prediction using the trained smoothing spline model for <code>X_const</code>?</p>
  
  <p>I do not get a good result When I do the following (the <code>model.predict</code> expects a 2D input, so, I cannot just stick <code>X_const</code> in there.):</p>
  
  <pre><code>x_const_basis = cr(X_const, df=freedom_df, constraints='center')
  x_const_basis = pd.DataFrame(x_const_basis)
  y_const_pred = model.predict(x_const_basis)
  </code></pre>
  
  <p>I want the black dots to be closer to red/actual values. any suggestion?
  <a href="https://i.stack.imgur.com/2SILh.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2SILh.png" alt="the resulting plot"></a></p>
  
  <h2>If we do a simple linear regression, we get sth like y=1.5x+10; then we can use it to predict y for any desirable x. But with this spline thing, it is a little bit confusing. we have to generate a basis like so: <code>x_const_basis = cr(X_const, df=freedom_df, constraints='center')</code>, and even if there are some <code>x</code> values in <code>X_const</code> that are close original <code>x</code> values in <code>x_1</code>, the predictions are too different.</h2>
  
  <p>The packet imported are:</p>
  
  <pre><code>import csv
  import numpy as np
  import pandas as pd
  import geopandas as gpd
  import sys
  from IPython.display import Image
  from shapely.geometry import Point, Polygon
  from math import factorial
  import datetime
  import time
  import scipy
  import os, os.path
  
  from statsmodels.sandbox.regression.predstd import wls_prediction_std
  from sklearn.linear_model import LinearRegression
  from patsy import cr
  
  from pprint import pprint
  import matplotlib.pyplot as plt
  import seaborn as sb
  </code></pre>