What exactly are the differences between a USB host and device
What exactly are the differences between a USB host and device?

The host initiates all communication on the bus, the device only responds when asked by the host. For Details see the specs on usb.org.

When two devices connect [...]

One must behave as the host and the other as the device. Details can be found in the USB OTG Specs.

Can one device acts as a host as well as a device?

For example, many Android phones and tablets can (requires Android 3+). The &quot;magic&quot; is in the On-the-Go cable/adapter, which switches the phone into host mode by pulling the ID pin to GND.