Can we exchange the permutation of a sponge construction? Part of a sponge construction (like SHA3 uses) is a fixed permutation $p$; which is clearly not one-way.

Could we, theoretically, exchange the permutation $p$ with any other permutation? What basic characteristics should such a permutation have – or would, for example, a simple LFSR already represent a valid replacement, assuming it spans the whole bit-range ($r+c$)?
The security of the sponge construction rely on two parts:


the size of the capacity.
and the strength of the permutation used in the construction.


This permutation is expected to have at least the following requirement:


provide a strong diffusion (in Keccak this is provided by $\rho$ and $\pi$).
provide confusion ($\theta$ and $\chi$).


In the case of Keccak, $\theta$ is an operation mainly columns oriented.


Which is why $\pi$ will ensure that every bits of a column is evenly spread in the slice. This prevent the creation of patterns.

$\chi$ is the main ingredient in Keccak-$f$. It is the only part that is not linear. Without it Keccak would be super weak to cryptanalysis.


propagation of a difference through $\chi$

Lastly, Keccak-$f$ provides a weak alignment (resistance to truncated differential cryptanalysis). The idea is to make sure that the differences are not constrained by a subdivision of the state (byte for AES or a group of 5 bits in the case of Keccak). However, due to the weak alignment of Keccak-f, finding the lower security bounds of the algorithm is harder.

If another permutation provide such characteristics (NORX permutation? I'll let  Richie Frame answer that part. He loves NORX). Then I guess it would be another decent choice.

I haven't studied LFSR.

TL;DR: The candidate permutation must provide strong diffusion, confusion and if possible a weak alignment.