Celery (stalks and leaves) is a great vegetable for guinea pigs to eat a couple times a week but in small quantities due to the high level of oxalates and low vitamin C content. ... Mix a small piece of celery in with plenty of leafy vegetables for a good varied diet and never give one guinea pig a whole celery stick.