Is there a rule of thumb when designing neural network in deep reinforcement learning? In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used in deep reinforcement learning? We can assess reinforcement learning algorithm's performance itself with rewards and so on, but how can we be sure that neural network used in reinforcement learning algorithm is good or bad ?
The process of reinforcement learning already implies that you have a base model to work from, that's what you're reinforcing. So, presumably, that underlying model is already good, otherwise you wouldn't be using it, right? 

The whole point of reinforcement learning is to introduce your (functioning) model to new information and/or changing conditions. Reinforcement learning isn't going to take a bad model and turn it into a good one. If your base model is not performing to your satisfaction, then you should go back to a model design/selection phase; reinforcement learning is not going to help you in that scenario.