The Bottom Line Inflammatory diet patterns that are high in sugar, refined carbs, unhealthy fats and processed foods can contribute to impaired memory and learning, as well as increase your risk of diseases such as Alzheimer's and dementia. Several other substances in food are dangerous for your brain too.