how does disabling 'password at startup' works before file-based encryption? Android 6 (before file based encryption):

after encrypting device and setting screen lock, i can choose an option 'require the PIN, password, or pattern at startup' or can i choose 'no thanks'. in the latter case the phone will boot correctly without asking for password but will ask for password just for screen unlock. 

how does it work? how can android boot from fully encrypted disk without getting password (when the screenlock password is set)? does it effectively disable encryption (by using default password)?
Actually Android 6.0 had Full Disk Encryption as specified in the Android 6.0 Compatibility Definition Document - Section 9.9 Full Disk Encryption with exceptions if the hardware was insufficient to perform the operation. 

So your case of a not specified lockscreen password is actually called out:


  If the user has not specified a lockscreen passcode or has disabled use of the passcode for encryption, the system SHOULD use a default passcode to wrap the encryption key. If the device provides a hardware-backed keystore, the password stretching algorithm MUST be cryptographically bound to that keystore. The encryption key MUST NOT be sent off the device (even when wrapped with the user passcode and/or hardware bound key). The upstream Android Open Source project provides a preferred implementation of this feature based on the Linux kernel feature dm-crypt.


So there 'SHOULD' be a default passcode in your scenario, however individual manufacturers may avoid it, most probably on hardware with poor encryption performance which were released with a prior version of Android.

If you are asking what happened before Full Disk Encryption was introduced in Android 5.0, the answer is that the manufacturer would have had to modify Android to support encryption. Samsung would be an example as they have their own Knox API for device management going back to Gingerbread/Honeycomb (2.x/3.x)

Update regarding boot sequence for Full Disk Encryption

I think your question is

If the disk is encrypted, how does it boot?

The Android framework documentation on Full Disk Encryption does actually cover the on boot behavior:


  In order to encrypt, decrypt or wipe /data, /data must not be mounted. However, in order to show any user interface (UI), the framework must start and the framework requires /data to run. To resolve this conundrum, a temporary filesystem is mounted on /data. This allows Android to prompt for passwords, show progress, or suggest a data wipe as needed. It does impose the limitation that in order to switch from the temporary filesystem to the true /data filesystem, the system must stop every process with open files on the temporary filesystem and restart those processes on the real /data filesystem.


From what I recall, for devices with Full Disk Encryption, upon boot once the user password/pin was entered, the device took a while for everything to get setup. This is implied by the fact that all of the running processes had to switch from a temp filesystem to the encrypted filesystem.