Should I put time on my Vanilla ANN for classifying MNIST Dataset
Some quick notes:

A mini-batch size of 4096 is rather large, try training with sizes like 16, 32, or even 1 to see what happens.

Your test accuracy seems to be around random since there are 9/10 classes to predict. Does your model predict randomly (i.e. every class gets a similar score) or is it just wrong often?

is it necessary to implement it from scratch?

Necessary for who? Unless it's required for your coursework, nobody is forcing you to implement this from scratch.

P.S.: Does it create any problem if a DL model is not written using OOP?

If an implementation is correct, it's correct. Whether it's spaghetti code, whether it's a single file, whether it's beautiful, tested, and reviewed OOP software.

OOP is there to help human write more maintainable, robust, and readable software, but under the hood, the computer really couldn't care less.