Is my index too expensive to maintain?
Is there a magic ratio where you can safely say it's not worth it to maintain such an index ?

There is not a magic number that exist. You might have an index that is used rarely (compare to number of updates) but supporting that query is important for your business/users.

It is a call that only you can make based on your business/user requirement.

If you want to test, do not drop the index. You can disable the index that way your metadata about the index is preserved. Here is a good post about that.

Disabling vs. Dropping Indexes by Jes Borland.