pg_ctl says "no server running" when I'm connected to postgres and running queries
pg_ctl doesn't track your database.  It interacts with a specific directory, either specified by -D, or through the PGDATA env variable.  If neither are set, then it throws an error.
$ unset PGDATA
$ pg_ctl status
pg_ctl: no database directory specified and environment variable PGDATA unset
Try &quot;pg_ctl --help&quot; for more information.

If your database truly did successfully start up, then it either must be configured to run on a non-default port, or it must be configured not to use a port at all but only a unix domain socket.  Neither of these configurations will be set up for you automatically by initdb, so I think you may be mistake that it started up in the first place.  Or maybe you did something beyond what you have described to us.
The Unix domain socket is named with a leading ., which makes it a hidden file.  You will only see it in /var/run/postgresql with ls if you give the -a option to it.
What pg_ctl start does behind the scenes is call postgres, so the database is always started via postgres.  It might be directly, or indirectly, or even more indirectly, but eventually that is how it is started.
It is not clear what it is you want to do.  If you want to run out of your own directory, then you probably want to disable the start up script that runs the other one automatically.  On the other hand, why not just use the one that is already configured and managed for you by the packaging system?  You can have both running side by side, buy why?