It backed plans to issue fines initially, and only resort to criminal charges for repeat offenders.
Selling, buying and producing the drug will remain illegal and the move must still be ratified by parliament.
According to the UN Office on Drugs and Crime, almost 9% of Israelis use cannabis, though some experts believe the figure to be higher.
The move follows recommendations by a committee set up to study the issue, and moves by a number of US states and European nations to decriminalise use of the drug.
"On the one hand we are opening ourselves up to the future. On the other hand, we understand the dangers and will try to balance the two," Prime Minister Benjamin Netanyahu told his cabinet ahead of the vote.
Israeli Justice Minister Ayelet Shaked said: "Israel cannot shut its eyes to the changes being made across the world in respect to marijuana consumption and its effects."
Meanwhile, Public Security Minister Gilad Erdan added that the move was "an important step on the way to implementing a new policy that will emphasise education and treatment instead of criminal enforcement".
Under the new system, first-time personal users who are caught and confess will be fined 1,000 shekels (Â£220; $270), with this doubling on the second occasion.
Probation will apply the third time and only a fourth case would lead to criminal charges.
Israel is one of the world leaders in research into medical use of marijuana.
The Israeli government has taken steps to reduce the penalties for personal marijuana use.