WhatsApp end-to-end encryption WhatsApp offers end-to-end encryption. However, when I start using it on a second device such as a PC running Windows, I can decrypt incoming messages without providing any private key into my second device or application. 

When a party sends a message to my account which has two clients receiving the incoming message, on PC and mobile device, they receive the same message perfectly.

If my private key is generated by my first device and stored on my first device, how is the private key being passed to my second device without any security flaw?
If by "using them on a second device such as a PC running Windows" you mean that you open Whatsapp web there, then no you cannot decrypt your messages there.

Whatsapp web is a client that connects to your phone (usually using whatsapp servers as proxy).

The client has shared keys with the phone (you scanned them as QR code) and the phone uses this to encrypt the already (on the phone) decrypted messages.

+----------------------------------+                     +----------------------------------------+
|                                  |                     |                                        |
|                                  +---------------------&gt;                                        |
|    Whatsapp Web Client           |     encrypted       |         Whatsapp Servers               |
|                                  &lt;---------------------+                                        |
|     shared key with phone        |                     |                                        |
+----------------------------------+                     +---+---------------------^--------------+
                                                             |                     |
                                                             |                     |
                                                             |                     |
                                                             |                     |
+-----------------------------------+                        |                     |
|         encryption secret keys    |                        |                     |
|                                   &lt;------------------------+                     |
|                                   |                                              |
|      Your phone                   |         encrypted                            |
|                                   |                                              |
|      shared key with web client   +----------------------------------------------+
+-----------------------------------+


Basically the Web client is just a GUI for your phone client, connected (securely) over the web.

You can try the following: Turn off your phone while using whatsapp web. You will see, that you cannot send and receive messages anymore.