iPhone SE Random contacts
Open the contacts app and tap "groups" in the upper left.  This should list all potential sources of new contacts.  For instance, I only have groups listed under "iCloud" and "Facebook".  If you see a source that you're not expecting you might be able to narrow down where these contacts are coming from.

It might also help to restrict what apps have access to your contacts and perhaps disable some. To check go to settings... privacy... contacts.