How do I stop Blender from automatically creating floating windows?
I assume you're looking for fullscreen by default.

Open a new blank file. Go to Window > Toggle Fullscreen Window. Then, go to File > Defaults > Save Startup File

Edit:

To stop renders and file browsers from popping up in new windows, go to Edit > Preferences.

Then select Interface > Editors > Temporary Windows.

Keep User Interface will not make any changes to the user UI when you press render. If there is an Image Editor set to Render Preview open, it'll display in there.

Fullscreen will replace your current window with a fullscreen render preview/file explorer.

Image Editor will replace the current editing window to an Image Editor set to render preview (for ex. your default 3D viewport will be switched to Image Editor it that was open)



EDIT 2

If you want to keep the current interface without any floating windows all you need to do is change the editor type or Toggle the active window, by selecting it on the top left, or selecting the shortcut of the editor you want. The file browser is Shift+F1. To go back to the 3D viewport press Shift+F5. Rendered images will be in the Image Editor or Shift+F10



To toggle the current active window into the preferences window you need create a new shortcut (version 2.8 has no pre-assigned default).

To edit the preferences and add a new shortcut:
Open the preferences window>keymap open the window section. Press on Add New to a new shortcut :


Type screen_space_type_set_or_cycle and select a combination of keys of your choice (in previous versions of blender it was Ctrl+Alt+U) and select the preferences window as "type" on the bottom left.

Then Click on the top right of the screen to show the option to save the preferences to make the change permanent.