Is it smart to just create wallet and then delete it, writing down the address? What are possible downsides of practice where I download a bitcoin wallet on my phone, write down the seed and the address, purge the app into nonexistence and continue my happy life, sharing my address, using it like, for example, a savings account?

I would then check the balance every few years.
If you only generate a limited set of addresses before you delete your wallet, it is likely you will run out of fresh addresses to receive funds on, meaning multiple spendable transactions will be linked together via the receiving address. This is a privacy disadvantage.

Secondly, your mnemonic key phrase back-up follows an off-chain deterministic key derivation standard (BIP32/43/44), and even if all consensus rules do not chain, there is no guarantee that wallets continue to implement the same key generation method that they currently do (depending on your timeframe). If you cannot use a wallet application with the same key derivation standard later, you will have to derive your used keys yourself using the BIP32/43/44 standards.