Is the Java function AES/CBC/PKCS5Padding vulnerable to padding oracle? I have an application that stores data on the devices. Currently AES/CBC/PKCS5Padding from javax.crypto.Cipher(see reference) is used to encrypt the stored data.

The padding isn't verified manually. Does that make the process vulnerable to Padding Oracle attacks?

Would the process benefit from using AES/GCM?
The problem with CBC mode is the padding. When there is a padding error, the server must respond a message back to you so that you can send the message back again or encrypt the message from the beginning.
The padding oracle attack is solely based on this idea. The attacker changes the byte and looks at the response of the server to execute the attack.
As we can see the attacker needs an oracle to execute the padding oracle attack. Data on-rest or encrypted databases has no oracle. Therefore they are not vulnerable to padding oracle attacks.

CBC mode has no integrity and authentication and removed from TLS 1.3.
AES-GCM which is in TLS 1.3 has an authenticated encryption scheme which provides both integrity and authentication. In the internal GCM mode uses CTR mode for encryption that requires no padding. Therefore padding oracle is not applicable.
So in short, in GCM mode you will have

Integrity
Authentication, and
No padding which is vulnerable to padding oracles.

Since you are going to use AES/GCM your next issue will be the IV. Instead of random IV, you can use incremental IV so that you can mitigate IV Reuse catastrophe. For more information on IV recommendations see section 8 of NIST-800-38D
In Java, you can use AES/GCM/NoPadding from  javax.crypto.Cipher package.