How can I weigh observations differently that were provided for a time horizon?
Therefore, the year of the observation was considered as a predictor variable which had a positive impact on the dependent variable. However, the year itself doesn’t have any effect on the dependent variable per se; rather, it is the other factors occurring in the same time period which result in improvements.


This sounds like a challenge for trees with sufficient interaction depth, as you've found that the year interacts with other factors that results in improvements. Ordinary least squares regression here do not capture that type of interaction well.

I would suggest setting up the following regression model:


Transform the date column into (CurrentYear - YearOfDateStamp) AS NumOfYearsAway. I'd recommend leaving this as a numerical feature rather than a categorical one. This would allow the tree based model to select cuts like NumOfYearsAway >= 5.5 instead of NumOfYearsAway in (6,7,10). This could also be more helpful when NumOfYearsAway = 0 occurs in your scoring dataset, where you might not have training data for current year data set.
Fit a tree based model, I'd pick XGBoost, with the usual CV to tune the hyperparameters such as interaction depth.


The drawback of picking XGBoost in your application however, is that the interpretation of the impact of a particular variable on the target variable is not obvious. You'd need partial dependence plot to observe how the target variable vary with the bespoke input variable. If interpretability is very important, one could pick a single tree regression model like rpart.