Is there a way to change the order in which the finder alphabetizes?
Unfortunately, no.



Since Mac OS X 10.0 "Cheetah", when sorting items by filename in Finder, files are sorted using the Unicode Collation Algorithm, with an additional specification that numeric substrings are sorted by numeric value, i.e. if multiple files exist whose filenames are of the form xny, where x is a susbtring that does not end with a digit, n is a substring which only consists of digits, and y is a substring that does not begin with a digit, then each such n is treated as a numeric value rather than a sequence of bytes, and the files are sorted accordingly. There is no option to disable this "natural canonical sorting", as some refer to it. This is stated officially in Apple's File System Programming Guide for macOS (§&nbsp;File System Details):


  The Finder
  
  Because the Finder is the user’s main access to the file system in macOS, it helps to understand a little about how the Finder presents and works with files.
  
  Filename Sorting Rules
  
  The Finder’s sort order for file and directory names is based on the Unicode Collation Algorithm (Technical Standard UTS #10) defined by the Unicode Consortium. That standard provides a complete and unambiguous sort ordering for all Unicode characters and is available on the Unicode Consortium website (http://www.unicode.org). The Finder alters the default sorting behavior of this algorithm slightly by taking advantage of some sanctioned alternatives, specifically:
  
  
  Punctuation and symbols are significant for sorting.
  Substrings of digits are sorted according to their numeric value, as opposed to sorting the actual characters in the number.
  Case is not considered during sorting.
  


Note that this behaviour is only exhibited by Finder itself. Other programs, such as the plain old BSD ls that comes packaged with macOS, behave in their own ways; typically, such BSD utilities abide by the Unix-style convention of sorting "lexicographically", i.e. by character encoding value.



As for sorting such that folders appear first, this can be done using the "Arrange/Group by Kind" option, but this then applies to other categories of files as well, not just folders. There is no way to sort a directory in Finder so that it is grouped into folders and non-folders.



As a workaround to both of these issues, you can use an alternative file browser such as Path Finder, which was partially designed to cater to these specific use cases.