how to run bert's pretrained model word embeddings faster?
There are several things that could make your code faster:

Stop using Pandas. Pandas is not designed for large-scale, fast text processing. It would be better to switch to something like Apache Arrow which is designed for efficient analytic operations on modern hardware.

Refactor code to avoid casting into memory-inefficient data types. progress_notes['vectors'].to_list() converts to a Python list which will use a lot of memory.

Refactor code to avoid casting back and forth between data types. Replace the list comprehensions and ''.join with complied regex. Complied regex processing will be much faster than equivalent Python code.

Refactor code to replace functions with inline code. Python creates a new stack for each function call. Inline code will be more memory efficient.