If SegWit only increases the capacity to only 2-4x only. Wouldn't we have the same issue in a few years again? If I understand correctly. SegWit is one of the solutions to the full 1MB block problem that doesn't require a hard fork. But I've read this only increases the 1MB to maybe about 2-4x only. If that is the case, wouldn't we have the same problem down the line when transactions reach that level again?
Just like a hard-fork blocksize increase SegWit  will not solve the scalability issue by itself. Either provides chiefly a fixed factor increase in capacity. 

SegWit is also a way to kick the can down the road, but in comparison to the hard-fork blocksize increase, it also solves other issues, doesn't require a hard-fork, and makes blocksize increases less problematic: SegWit resolves an issue where transactions require quadratic effort for verification. The problem was highlighted by the "megatransaction" created by f2pool last July. While capacity only increases linearly with bigger blocks, effort to verify blocks increases at a super-linear rate. For SegWit transactions the transaction verification takes linear effort, making bigger blocks cause less of an advantage for the author of the latest block. Hence, SegWit helps reduces centralizing pressure on mining in the face of capacity increases.

On-chain scalability is always limited to linear growth anyway: For every transaction that you resolve on-chain, you need to provide the storage space on-chain. While we will be able to go down that route for a bit, it is unsustainable for large amounts of transactions and we wouldn't be able to reach e.g. the transaction counts corresponding to the payment counts of credit card companies.

Scalability improvements need to make better than linear use of the blockchain capacity.
Unlike a hard-fork blocksize increase, SegWit sets the stage for further improvements in the future. One promising proposal is the Lightning Network, which compounds numerous payments at the cost of two or three on-chain transactions. Another exciting improvement is the introduction of Schnorr signatures. Schnorr signatures are smaller than ECDSA signatures, but due to their homomorphic nature allow for both signature aggregation and key aggregation. In addition, Schnorr signatures enable the use of adaptor signatures that allow for a wide range of smartcontracts to be created with Scriptless Scripts.

Last but not least, SegWit introduces Script versioning. This allows to specify which version of Script to use to interpret a transaction, allowing multiple versions of Script to coexist safely at the same time in the network. Script versioning allows much easier deployment of Script improvements to expunge bugs and vulnerabilities, or to add new features like Schnorr signatures.