Feature engineering decrease my cross validation
You might be dealing with overfitting, since your model does not generalize that well. It should be noted that Light GBM is sensitive to overfitting in small datasets.

To solve this, you can check for features with correlation and drop some of them. Another solution can be the implementation of a feature selection method. Lastly, since GBM is a tree algorithm, you can try an early termination.

If you are dealing with highly imbalance data, you can consider using a stratified method rather than CV. In sklearn, there are StratifiedKFold and StratifiedShuffleSplit, which ensure that relative class frequencies are approximately preserved in both train and validation phases.