Measuring the Mix. Use 4 ounces of plain, low-fat yogurt, a cup of skim milk and a medium banana, and your smoothie will have 259 calories. Swap the skim milk for almond milk, and you'll save 23 calories. But if you use whole milk, the calories in your smoothie increase by 66.epending on the ingredients, they can also be high in calories. You can make a basic banana smoothie with yogurt, milk and banana, or add other ingredients to increase the nutrient content and vary the flavor. Choose low-fat versions of dairy products to limit the saturated fat and calories in your smoothie.