Why should foreign keys be implemented in my relational database?
I don't have enough points to comment, so an answer.

OMG. I feel your pain ;-)

Yeah, people find foreign key constraints a real pain. But without them, you have to then audit the data for widows and orphans, otherwise you're entirely likely to have bad summary results. At the very least, you don't really know the validity of your results. Are they off by 0.001%? By 2%? It's hard to say...you have go go in and figure out the level of garbage data.

On a human level, okay, I get it, constraints can be a pain. It's an acute pain as it blocks you right at the point of insertion. (Kind of the whole point.) With sloppy rules, then you get the pain, but it's deferred. And speaking of which, since I see a Postgres tag on here, a couple of things to know about. First up deferrable constraint. This is an enormously valuable feature for this very situation. It really can be a pain to insert a child only to get it kicked back because the parent doesn't exist yet when you're going to add that parent shortly, within the same transaction. It's kind of fussy and tedious to sort that out. (Or it can feel that way.) With a deferred constraint, the FOREIGN KEY is still enforced, but not until the end of the transaction. So, you can add your records in less strict order, if that's a help. Here's a piece on the subject that looks pretty thorough, you can find many others if you look:

https://begriffs.com/posts/2017-08-27-deferrable-sql-constraints.html

And, secondly, and for what it's worth, you've also got ON UPDATE CASCADE support, although needing that is often a sign of a problem in the design. 

Oh, and for bullet points, a formally defined constant goes into the meta data about your system. So, any automatic tool that knows how to interpret that data can do much, much more for you. For example:


Class and code generators.
Structure diagrams.
Test cases. 
Query constructors.
Etc.


Adding an edit here with a suggestion that could help. Opinions are all well and good, but evidence sometimes is better. To that end, what about running some exploratory queries to find out the state of play in the data today? You may find that there are no problems in the data, even without server-side constraints. Some people are just freakishly good at not screwing up. Then again, there might be a total mess. Here's a query off the top of my head which, I'm sure, can be improved:

     select count(*) 
       from child
tablesample bernoulli(5) -- If you're only after an estimate, TABLESAMPLE is your friend.
  left join parent on (child.parent_id = parent.id)
      where parent.id is null


P.S. I lost this very same argument once, largely because I was unaware of the deferred option at the time.