Stochastic gradient descent based on vector operations?
First of all, word "sample" is normally used to describe subset of population, so I will refer to the same thing as "example". 

Your SGD implementation is slow because of this line: 

for each training example i:


Here you explicitly use exactly one example for each update of model parameters. By definition, vectorization is a technique for converting operations on one element into operations on a vector of such elements. Thus, no, you cannot process examples one by one and still use vectorization. 

You can, however, approximate true SGD by using mini-batches. Mini-batch is a small subset of original dataset (say, 100 examples). You calculate error and parameter updates based on mini-batches, but you still iterate over many of them without global optimization, making the process stochastic. So, to make your implementation much faster it's enough to change previous line to: 

batches = split dataset into mini-batches
for batch in batches: 


and calculate error from batch, not from a single example. 

Though pretty obvious, I should also mention vectorization on per-example level. That is, instead of something like this: 

theta = np.array([...])  # parameter vector
x = np.array([...])      # example
y = 0                    # predicted response
for i in range(len(example)):
    y += x[i] * theta[i]
error = (true_y - y) ** 2  # true_y - true value of response


you should definitely do something like this: 

error = (true_y - sum(np.dot(x, theta))) ** 2


which, again, easy to generalize for mini-batches:

true_y = np.array([...])     # vector of response values
X = np.array([[...], [...]]) # mini-batch
errors = true_y - sum(np.dot(X, theta), 1)
error = sum(e ** 2 for e in errors)