Test dataset with categorical variable value not present in train dataset & transformer I want to replace values of a categorical variable ( named 'six' ) by the mean of my target variable ( named 'target' ). 

I am fitting a transformer doing just that on a train dataset df and then transform the test dataset df2.

How do I deal with a value appearing solely in the test dataset ? 

When fitted on the train dataset the transformer received no mean value of the target variable on that value.

For example :

myarray = np.array([ [ 1 , 1 , 3 , 'v' , 'a' , 'x' , 0 ] , 
                   [ 2 , 2 , 2 , 'v' , 'b' , 'y' , 1 ] ,
                   [ 4 , 5 , 1 , 'w' , 'c' , 'z' , 1 ] ,
                   [ 2 , 1 , 9 , 'w' , 'c' , 'x' , 1 ] , 
                   [ 1 , 0 , 4 , 'w' , 'b' , 'y' , 1 ] ,
                   [ 2 , 2 , 3 , 'v' , 'b' , 'z' , 0 ] ] )

colnames = [ 'one', 'two', 'three' , 'four' , 'five' , 'six' , 'target' ]

df = pd.DataFrame( myarray , columns = colnames )

myarray2 = np.array([ [ 2 , 7 , 3 , 'v' , 'a' , 'x' , 0 ] , 
                      [ 9 , 2 , 2 , 'v' , 'a' , 'y' , 0 ] ,
                      [ 4 , 5 , 1 , 'w' , 'c' , 'k' , 1 ] ]  )

colnames2 = [ 'one', 'two', 'three' , 'four' , 'five' , 'six' , 'target' ]

df2 = pd.DataFrame( myarray2 , columns = colnames2 )


df is my train dataset, df2 my test dataset.

We can see variable 'six' has the k value not existing in the train dataset.

Next :

df[ 'target' ] = df[ 'target' ].astype( 'float64' )


Next ( my homemade transformer ) :

class Cat2TargetMean( BaseEstimator , TransformerMixin ) :

    def __init__( self , col2trans , tgt_col ) :
        self._col2trans = col2trans
        self._tgt_col = tgt_col

    def fit( self, X, y = None ) :
        self._dic_col_p = {}

        for col in self._col2trans :
            p = X.groupby( col ).mean()[ self._tgt_col ]
            self._dic_col_p.update( { col : p.to_dict() } )
        return self 

    def transform( self , X , y = None ) :
        for col , dic_p in self._dic_col_p.items() : 
            X.replace( { col : dic_p } , inplace = True )  
        return X


Then :

 tsf = Cat2TargetMean( [ 'four' , 'five' , 'six' ] , 'target' )

 tsf.fit( df )

 tsf.transform( df )

 tsf.transform( df2 )


Result :

    one two three   four    five    six target
    0   2.0 7.0 3.0 0.333333    0.0 0.5 0.0
    1   9.0 2.0 2.0 0.333333    0.0 1   0.0
    2   4.0 5.0 1.0 1.000000    1.0 k   1.0


'k' value of column 'six' has not been transformed.
I usually replace unseen and NaN values with the global target mean.

There are also already implemented transformers for target encoding that you could use that gives you some options such as smoothing: scikit contrib - target encoder.