Disable MEMORY_TARGET after being locked out of database?
Find the name of the spfile on the filesystem, it should be something like this:

/u01/app/oracle/product/11.2.0/xe/dbs/spfileXE.ora


You should not edit the spfile directly as it contains binary data, but you can create a pure, text-based parameterfile from it:

sqlplus /nolog
create pfile='/tmp/initXE.ora' from spfile='/u01/app/oracle/product/11.2.0/xe/dbs/spfileXE.ora';
exit


Edit /tmp/initXE.ora, modify the values you want, then recreate the spfile from it:

sqlplus /nolog
create spfile='/u01/app/oracle/product/11.2.0/xe/dbs/spfileXE.ora' from pfile='/tmp/initXE.ora';
exit


Once it's done, try restarting the database.