How does the initialization of the value function and definition of the reward function affect the performance of the RL agent?
There seem to be two different ideas in this question here:


What's the impact / importance of our choice for reward values?
What's the impact / importance of our choice for initial value estimates (how do we initialise our table of $Q(s, a)$ values in the case of a simple, tabular RL algorithm like Sarsa or $Q$-learning)?


The reward values are typically assumed to be a part of the problem definition - something we shouldn't modify if we're using an existing problem definition as a benchmark. But if we're in charge of defining the problem ourselves, we can of course also pick the reward values. Modifying them may indeed have a huge impact on the speed with which RL algorithms are able to learn a task - but it may also intrinsically changes the task, it changes the objective of the problem, it may change which policies are optimal.



As for initialisation of our table of value approximations: by default, we normally assume an all-$0$ initialisation. However, it is a fairly common trick (in tabular RL algorithms, without function approximation) to initialise value estimates optimistically; pick higher initial $Q(s, a)$ value estimates than are likely (or even pick values higher than a known upper bound on what the true value possibly could be). This is often beneficial - also in large gridworlds with sparse rewards (e.g. a single distant goal somewhere) and negative rewards (i.e. costs) incurred for every step taken - because it incentivises exploration of state-action pairs that have not yet been tried.

Suppose you have your gridworld with negative rewards associated with every time step, and the optimal policy being one that takes you to a distant goal as soon as possible. If all $Q(s, a)$ are initialised to $0$ (or worse, to negative values), your agent may quickly learn that everything it does is equally bad anyway, and get stuck near the starting position. If all $Q(s, a)$ values are initialised optimistically (to large, positive values), your agent during the learning process will still have optimistic expectations of what it can achieve if it just tries to navigate to unexplored parts of the state-action space.