Allow only unique combination of values for a composite primary key
Since you use MariaDB 10.2, you can add a CHECK constraint, enforcing that the first id is less than the second (id_1 &lt; id_2). CHECK constraints were added in version 10.2.1.

This has the limitation that you'll have to provide the ids for the INSERT statements in correct order (you won't be able to insert the (3,2) combination, even if (2,3) is not there):

ALTER TABLE compare
  ADD CONSTRAINT ids_unique_combination_chk
    CHECK (id_1 &lt; id_2) ;


Tested at dbfiddle.uk.



If you don't want to have to hassle with your INSERT statements and procedures, you can use two VIRTUAL (or PERSISTENT) columns and a UNIQUE constraint.

Virtual columns are available in MariaDB since version 5.2 and several limitations were lifted in 10.2.1:

ALTER TABLE compare
  ADD small_id BIGINT AS (LEAST(id_1, id_2)) VIRTUAL,
  ADD big_id   BIGINT AS (GREATEST(id_1, id_2)) VIRTUAL,
  ADD CONSTRAINT ids_unique_combination_uq
    UNIQUE (small_id, big_id);


Tested at dbfiddle.uk, too: fiddle-2.