Here, we apply S4 to chemical language modeling on SMILES strings and benchmark it on various tasks relevant to drug design -from learning bioactivity to chemical space exploration and natural-product design. Moreover, we further corroborate the promise of S4 via the prospective de novo design of kinase inhibitors, validated using molecular dynamics simulations. Our results show the promise of S4 for chemical language modeling, especially in capturing bioactivity and complex molecular properties. To the best of our knowledge, this is the first time that state space models have been applied to molecular tasks, and we expect their relevance for chemical language modeling to increase in the future. S4s are an extension of discrete state-space models, which are widely adopted in control engineering . Discrete state-space models map an input sequence u to an output sequence y, through the learnable parameters