What exactly is this App Data that is being backed up to Google? Anyone know exactly what "app data" consist off? I'm refering to the new App Data-backup adapter that Google pushed out a while ago. 
I have a notes application. From what I can see, it don't save the notes to the memory card unless I choose to export them. Question is, are the notes saved in the data folder and are my (very private) notes therefor somewhere within the Google cloud now?

(This backup happened behind my back - I have all sync adapters turned off, but this one was installed silently and of course enabled by default. I'm not comfortable at all with this.)
Google has documented details of the Android Backup Service and of the Data Backup Framework on its Android Developers website. The descriptions are targeted to developers writing applications but are still helpful:

Android's backup service allows you to copy your persistent application data to remote &quot;cloud&quot; storage, in order to provide a restore point for the application data and settings. If a user performs a factory reset or converts to a new Android-powered device, the system automatically restores your backup data when the application is re-installed. This way, your users don't need to reproduce their previous data or application settings. This process is completely transparent to the user and does not affect the functionality or user experience in your application.

They also specifically address user privacy:

Google securely transmits backup data to and from Google servers in order to provide backup and restore features. Google treats this data as personal information in accordance with Google's Privacy Policy.
In addition, users can disable data backup functionality through the Android system's privacy settings. When a user disables backup, Android Backup Service deletes all saved backup data. A user can re-enable backup on the device, but Android Backup Service will not restore any previously deleted data.

TL/DR: If the Notes app is configured to use the Android Backup Service and has defined your notes as data to include in the backup, then yes, your data is being stored on a Google server. Toggling Backup my data will delete currently saved personal information from Google's servers that is associated with your Google account. This will affect all of your current apps' backed up data.