why is my trigger called even when the underlying table didn't change?
This is the expected behaviour of a statement trigger. It executes every time a statement (INSERT in this case) is executed and once (per statement, not per row.) The behaviour is similar in other DBMS (like Oracle.)

The MSDN says it in the first paragraph in the CREATE TRIGGER page:


  These triggers fire when any valid event is fired, regardless of whether or not any table rows are affected.


I don't think there is a way to change this. There are ways to check though - inside the trigger, once it fires - whether there are rows affected or not and act accordingly.

For example you can use

IF ( @@ROWCOUNT = 0 )
  RETURN;


Or a somewhat more robust alternative in the presence of MERGE

IF NOT EXISTS(SELECT *
              FROM   DELETED)
   AND NOT EXISTS(SELECT *
                  FROM   INSERTED)
  RETURN;