The Taiwanese accent: are pairs "种，总", "四，是", "忘，万" pronounced EXACTLY the same?
The key to this question is which accent of Taiwan you're talking about. There is a large difference between Standard Taiwan Mandarin (標準台灣國語) and the various accents commonly found across Taiwan.
There certainly are accents where there is absolutely no distinction between ㄕ (sh) vs ㄙ (s), ㄓ (zh) vs ㄗ (z), ㄔ (ch) vs ㄘ (c), ㄖ (r) vs ㄌ (l). The latter of each pair is the only phonetic realisation, and because of the loss of the distinction, indeed one must rely on context. This is actually the most widespread geographically across Taiwan, and is characteristic of casual speech in less urbanised areas, as reported by this 2019 study. See this interview from the early 2010s, as well as this TEDx talk from 2019.
There are also accents where there is no distinction, but the resultant merger lies between the two versions. Sometimes it is a consistent postalveolar [ʃ] / [tʃ] / [tʃʰ] (although ㄖ (r) remains merged into ㄌ (l), in these accents). But more often not than not, it varies between completely alveolar [s] and completely postalveolar [ʃ] or alveolo-palatal [ɕ], with rounded vowels and front vowels leading to more palatalisation (e.g. one 2012 study compared sa, sha, su, shu across Beijing and Taiwan listeners). This last realisation generally does not lead to merger with the &quot;true&quot; palatal series (Pinyin x-, j- y-) because the vowels that follow are different.
Then you have the accents where the distinction is maintained, even if it is &quot;softened&quot; to an postalveolar vs alveolar distinction. This is the one that is most associated with formal media, e.g. news reports, and is the one that is standardised by the Ministry of Education of the Republic. Thus, as Zhuyin Fuhao was designed for this standard, there is no variation in the Zhuyin, although it can be used to describe variation. See this commentary of some Taiwan-based celebrities, selected for the 標準ness of their Mandarin.
Finally, there are those with inconsistent or non-standard distributions of the alveolar vs postalveolar distinction. This is characteristic of the Taipei-centric urban Taiwanese accent, and is perhaps the most confusing for learners of Mandarin. See this TEDx talk for an example (compare 事情 and 世界 across the talk).
The loss or reduction of the retroflex vs alveolar distinction is widespread and indeed strongly associated with the whole of Taiwan, as well as being common in most of the southern provinces of China as well as Singapore and Malaysia. However, the merger of final -n vs -ng (usually into a general nasalisation of the vowel) is more strongly stigmatised. There are further aspects to accents that are considered even less standard, e.g. the fronting of the ㄭ central+null vowel, phonemically /ɨ/, realised as [u~ɯ~y], and non-standard aspiration.
As a summary, please see this short debate, where:

The news anchor and the reporter speak in a standard Taiwan accent that distinguishes postalveolar sh- from alveolar s-; it retains many features that are characteristic of standard Taiwan Mandarin.
The first man is of a Taiwanese Hokkien-speaking background, and has a typical ㄉㄞˇ ㄨㄢ ㄍㄛˊ ㄧˇ  (Dǎiwān góyǐ) accent. This includes the lack of -n and -ng distinction, limited use of ü /y/ (being replaced by i /i/) and the Pinyin &quot;central i&quot; /ɨ/ being fronted. He also mixes plenty of actual Taiwanese Hokkien into what he says.
The second man is of a 外省人 (post-1949 &quot;mainlander&quot;) family background and speaks with a fairly general pre-2000 Taiwanese accent. There is no retroflex distinction, but there is the -n vs -ng distinction. On the other hand, his Taiwanese Hokkien makes a tonal error which causes ripples of suppressed laughter.