Can we understand SVM without knowledge of Machine Learning
I don't think there is anything that should stop you. A couple of tips to set you on the right path:


The basic principles of Machine Learning are very simple. They are often formalized in a way that makes it look very complicated, but this is only necessary if you want to get into the details of why things work. The general framework of what makes a classifier or what makes a clustering algorithm is very simple. Make sure you understand things on that level, before you dig into the details.
The title question (can SVMs be understood without ML knowledge) is slightly non-sensical. Understanding the basic ideas of ML is a minor investment and understanding SMVs takes much longer. There's no reason not to get a basic idea of what machine learning is before you dig into SVMs.
Make sure you have a good intuitive undertanding of the following principles and distinctions: features and instances, clustering and classification, supervised and unsupervised learning. After that, SVMs are just one particular way of solving a particular problem.
If you understand what a classifier is, you can start using SVMs without knowing anything about how they work or what they do. This may not give you the best result, but it's a good way to start learning by doing.


I can't really give you any references to start reading, since most of what's out there is far to formal for the level you'd want to start with. I recommend browsing around. If something is easy to read, then continue on, if it's too full of math, try something else first.