When I am logged-in the page doesn't look the same as when I am logged-off
What you see / notice is pretty standard Drupal ... The link like "Add content" is something you typically only want to be shown for logged in user (and maybe even limited to users with roles such as "content editor" (or whatever you'd call it). Also, the small icons in the upper left (above "logo" and "home") are typically also things you only should be able to use if you are logged in.

Should you want to change any of such things (eg: anonymous users should also be able to "add content"), then it's just a matter of finding out which permission it is that actually governs the specific function (such as "add content"), and then indicate in your "permissions" page within admin that is related to it that you want "role" anonymous to have the very same permission.