I am trying to train a CNN for regression on a dataset where most of the points lie around a similar output value. There are however a few outliers that are very important but they are less represented and the trained network thus tends to predict all output values close to the mean of the whole dataset (underfitting). This leads to a somwhat small error (and good precision) because the vast majority of points lie in that range, but the error is way higher for points even slightly outside the "normal" case.

But since this regressor would be most useful to predict the output for outliers (quality-control use case), it's currently pretty much useless.

Is there a way to prevent this kind of behavior and have a CNN that is giving a greater weight to outliers and extrema, in order to avoid underfitting?

To some extent, the Random Forest method, although much better at predicting the output for outliers, still exhibits a higher error for points in the extrema, while the error around the mean is very small. The "low" points are predicted with too high a value, and the "high" points are predicted with too low a value (each time closer to the mean). So any idea for that case would be great too!

Thanks a lot

  • $\begingroup$ What non-linearity do you use? $\endgroup$
    – Media
    Dec 1 '18 at 19:20
  • $\begingroup$ Do you mean that the inputs or outputs are outliers? $\endgroup$
    – Dave
    Aug 30 '20 at 6:07
  • $\begingroup$ @Dave the output values are outliers. In the end I could mitigate this effect a little by implementing SMOTE for Regression. $\endgroup$
    – beeb
    Sep 18 '20 at 7:32

I'm not sure what you are trying to do. CNNs are good for image-related tasks, as they attempt to extract spatially local features from the input data. They can be used for regression problems but only as long as the input resembles an image.

Random Forests on the other hand are bad at image-related tasks, unless some sort of feature extraction has been performed beforehand.

Does your dataset consist of images? If not don't use CNNs!

  • 1
    $\begingroup$ Hello! Thanks for the reply. The data is multi-channel time-related graphs (1d time versus metric, for 6 metrics). Think sensor output, for a finite duration. I saw that CNN were used for natural language processing so it gave me hope that they could be used for such a task. $\endgroup$
    – beeb
    Aug 3 '18 at 11:57
  • $\begingroup$ Mind you the random forest regression is not used directly on the time data, features are extracted beforehand and the decision trees are applied on the extracted features $\endgroup$
    – beeb
    Aug 3 '18 at 12:05
  • $\begingroup$ Oh, I'm not aware of any methods CNN have to prevent outliers in such data, however in theory dropout might help as it might make use of all the features... $\endgroup$
    – user50384
    Aug 3 '18 at 23:45
  • $\begingroup$ Thanks for the answer. In the case of the Random Forest, simply duplicating records below the 25% and above the 75% percentiles increased the accuracy in those regions dramatically. A bit dangerous but in this case it works. I'll try with the CNN soon as well. $\endgroup$
    – beeb
    Aug 8 '18 at 7:32
  • $\begingroup$ Hey there! Just coming back to say that we achieved good results with a 1D CNN in the end. I'm not at liberty to discuss the details, but in principle it works for our multi-channel timeseries. So I'd like to mitigate your statement about not using CNNs if the input is not an image. $\endgroup$
    – beeb
    Dec 3 '18 at 13:15

Your Answer

