How to represent the relationship between artists and albums in an entity-relationship diagram
Replace ALBUM with SONG, keep ALBUM but add a "many to many" relationship between the two.

There are still a lot of complexities you are missing (eg "Artist formerly known as Prince" vs "Prince", cover songs, etc)