SQL Server De-duplication Trigger
I think the condition you are checking will always be true as for every insert query executed there will be an entry in the dbo.Table and inserted as well so it will rollback the transaction.

From MSDN


  The inserted table stores copies of the affected rows during INSERT
  and UPDATE statements. During an insert or update transaction, new
  rows are added to both the inserted table and the trigger table. The
  rows in the inserted table are copies of the new rows in the trigger
  table.


I would advice you to check the duplicate before insert as

IF NOT EXISTS (Select a.* FROM [dbo].[Table] a WHERE..............)
  INSERT INTO [dbo].[Table] .......


More Readings : http://msdn.microsoft.com/en-in/library/ms191300.aspx