How do I directly access a memory mapped register of AVR with C
It's worth exploring how PORTD is defined in the actual AVR header file.  The relevant file is iom328p.h, and it's defined as:

#define PORTD   _SFR_IO8(0x0B)


_SFR_IO8 is defined as

#define _SFR_IO8(io_addr) _MMIO_BYTE((io_addr) + __SFR_OFFSET)


and

#define _MMIO_BYTE(mem_addr) (*(volatile uint8_t *)(mem_addr))


This shows that 0x0b is not the address of the PORTD register, but its offset from the address __SFR_OFFSET, which is 0x20.  Indeed, if you look at the ATMEGA328 datasheet, the register summary shows that PORTD is at offset 0x2b.  So that is the address that you would want to access via a pointer.

Note that this also shows the type of the pointer, if we fully expand the original definition of PORTD we get:

#define PORTD   _SFR_IO8(0x0B)
#define PORTD _MMIO_BYTE((0x0B) + __SFR_OFFSET)
#define PORTD _MMIO_BYTE((0x0B) + 0x20)
#define PORTD (*(volatile uint8_t *)(0x2B))


This syntax takes the integer 0x2B, casts it to type pointer to volatile uint8_t, and dereferences that pointer so you can assign to or read from the location the pointer references.  The volatile qualifier is important because it prevents the compiler from optimizing out accesses to memory.  Without it, the compiler may store the value to a register instead, or not store it at all, which would mean your IO port would remain unchanged or appear to report an incorrect value.

To go back to the title of your question, you have asked how to "directly access a memory mapped register in C", well the answer is you do that exactly the way the header file defines for you: 

(*(volatile uint8_t *)(0x2B) = 0xFF


aka

PORTD = 0xff


There is no more direct way than this.  The compiler sees a literal address and writes to it.  What you've written is very different (adjusting to use the correct address and type:

volatile uint8_t * portd = 0x2b;
 *portd = 0xFF;


This allocates a location in memory symbolized by portd, which holds the value 0x2B.  When you write *portd = foo, the MCU has to access the location where portd is stored, take the value found there as an address to find the location you actually want to write to.  This method requires that you allocate a place in memory to hold portd and requires an extra memory access to be able to read or write to the port register.  This method is therefore indirect.

In reality, if optimization is enabled, the compiler may optimize the pointer away and replace it with the same literal access as in the PORTD case, however this depends on the program structure, and in particular the qualification and scope of the pointer.  Limiting the scope of portd as much as possible and/or declaring it as volatile uint8_t * const portd = 0x2b (which indicates that the pointer's value is constant even if the value pointed to is volatile) will increase the likelihood of the pointer being optimized away.



The reason the AVR headers define the IO port registers this way is because in addition to the standard load/store instructions that can access the entire data memory address space and take three clock cycles to complete, the IO registers are special, and can also be accessed using IN/OUT instructions, which complete in two cycles.  The load/store instructions use the memory address for the IO registers (starting at offset 0x20), whereas the IN/OUT instructions use the IO address, which starts at 0x00, but corresponds to 0x20 in the memory address space.  From the datasheet:


So when you access an IO location, the optimizer should recognize that it can issue a more efficient instruction than the standard load/store, and it will replace the load/store to 0x2b with an in/out to 0x0b.