Here are five foods that can cause kidney stones because of a high oxalate content. 1  Sweet potatoes, baked or canned–These vegetables which are often advised as being better for you then regular white potatoes which are very high in oxates.  Kiwi fruit–This is one of of the fruits highest in oxalates.