Does taking a long time to graduate impact my Ms/PhD admission?
As a rule of thumb, the time you took wont matter, but the failing grade will.

However, if your goal is to get into a masters course in Brazil, many of them won't look at your undergraduate marks (I've served on graduate committees on two top brazilian math departments).

In these cases, you can be admitted by doing well on an admitting summer course, or on an admission exam (many of them nowadays use the Extramuros examination, or a local exam).

Even for institutions that will judge your undergraduate marks, it's not uncommon that a student with a failing mark get accepted. Usually they would indicate in their statement why they failed and how they did well on this course on the next time. The fact that you had to work is enough justification, if that's a localized problem and you have strong marks otherwise.

After doing well in your masters, your undergraduate marks will have a negligible impact on your PhD admission.