Is it a good practice to build in as many foreign keys as possible?
Another approach is to add player_id to instances table as a foreign
  key. Now players can queried directly from instances table. Shorter
  query, but one more column for a foreign key.


IMO, this will be bad approach.This is kind of circular reference  .

Since you have one to many relation so mapping table will be separate,dungeon,instance and player each will have master table and mapping table defining their one-to-many relation.

    CREATE TABLE instances (
    instance_id INT PRIMARY KEY
    ,instance VARCHAR(100) NOT NULL
    )

CREATE TABLE dungeons (
    dungeon_id INT PRIMARY KEY
    ,dungeon VARCHAR(100) NOT NULL
    )

CREATE TABLE player (
    playerid INT PRIMARY KEY
    ,player VARCHAR(100) NOT NULL
    )


CREATE TABLE dungeon_instance (
    dungeon_instance_id INT identity(1, 1)
    ,dungeon_id INT NOT NULL
    ,instance_id INT NOT NULL CONSTRAINT FK_instances FOREIGN KEY (instance_id) REFERENCES instances(instance_id)
    ,CONSTRAINT FK_dungeon FOREIGN KEY (dungeon_id) REFERENCES dungeons(dungeon_id)
    ,PRIMARY KEY (
        dungeon_id
        ,instance_id
        )
    )

CREATE TABLE player_dungeon (
    player_dungeon_id INT NOT NULL
    ,playerid INT NOT NULL
    ,dungeon_id INT NOT NULL
    ,CONSTRAINT FK_player FOREIGN KEY (playerid) REFERENCES player(playerid)
    ,CONSTRAINT FK_dungeon FOREIGN KEY (dungeon_id) REFERENCES dungeons(dungeon_id)
    ,PRIMARY KEY (
        playerid
        ,dungeon_id
        )
    )


if you have very frequent query between player and instance where there is no dungeon table require in output and volumne of data is high then you can go for Denormalisation.

Whether to create player_instance mapping table or not depend upon above requirement.

But it desirable to achieve Denormalisation via index view