Can I change default folder to save startup file (startup.blend and userpref.blend)?
To save Blender user preferences in Blender's own installation folder, also known as a Portable Install, start by navigating to the place where you installed or unpacked the Blender archive.
Method 1 - Easy and simple
Once there open the Blender folder and search for the version number folder (commonly 2.## ) inside the root directory, and inside it create a new directory called Config such that it stays Blender\2.##\Config.
If you now run Blender from this location and press Ctrl+U startup.blend, userpref.blend, bookmarks.txt, and recent-files.txt will be placed under this config folder by default.
New addons will now be installed alongside default ones currently shipped with the installation Blender\2.##\scripts\addons or Blender\2.##\scripts\addons_contrib
If you already had any previously customized config files under your Windows profile you can manually copy them from the default profile location (generally under C:\Users\\\$USERNAME$\AppData\Blender Foundation\Blender\2.##\) to this new location to bring back your previously saved preferences.
Method 2 - Safe and maintainable
Blender supports several launch variables, a few of them are BLENDER_USER_CONFIG and BLENDER_USER_SCRIPTSwhich allow overriding the default folders for user configuration (where startup files are saved) and the folder for users scripts and addons, where third party installed addons, presets, keyconfigs and scripts are installed.
The trick is to launch Blender from a batch file that sets this to your custom desired location.
In a plain text editor like Notepad paste the below lines of code and save them to the same folder as Blender.exe, give it a any name like &quot;Blender_Portable&quot; but make sure you specify the file extension as .bat not .txt.
SET BLENDER_USER_CONFIG=%~dp0Portable_Data\Config
SET BLENDER_USER_SCRIPTS=%~dp0Portable_Data\Scripts
    
if not exist %BLENDER_USER_CONFIG% mkdir %BLENDER_USER_CONFIG%
if not exist %BLENDER_USER_SCRIPTS% mkdir %BLENDER_USER_SCRIPTS%
    
start Blender.exe

This will ensure any custom addons and scripts installed will be placed in a separate directory from default shipped ones, making this &quot;portable install&quot; easier to update for future releases without loosing your custom installed addons or having to install them and configure again.
Notes
When using any of these methods make sure Blender is not &quot;formally installed&quot; into one of the default protected system locations like C:\Program Files or C:\Program Files (x86) or any other place in the file system with restricted access that may require administrator privileges to write to, otherwise saving preferences may trigger UAC prompts or fail entirely.
These method are more suitable if you install Blender in some secondary internal drive, removable media like a thumbdrive, external HDD, USB device, memory cards, or similar, and useful if you jump around computers frequently (between work, home or school). This way you can keep you settings and user preferences with you, and carry them around wherever you go, without changing system installed or local defaults.