How to show missing rows with default values
I have updated your fiddle with:


add one more category for to see the query is correct when there is more than 1 category
alter amounts making category amount sets different and altering zero values for to see the records inserted additionally.


Edited fiddle.

The final query is

With 
Categories AS (
  SELECT DISTINCT Category
  FROM Category
  ),
MonthCategories AS (
  SELECT m.MonthNameLong, cat.Category
  FROM Month m, Categories cat
  )

select mc.Category, mc.MonthNameLong, coalesce(cat.Amount, 0.0) as Amount
FROM MonthCategories mc
LEFT JOIN Category cat ON cat.Month = mc.MonthNameLong
                      AND cat.Category = mc.Category;


Of course you may "pack" the query combining CTEs to one and/or moving CTEs to subquery.

PS. If there is only one category in the table you may simply replace s.Category with MAX(s.Category) OVER (ORDER BY s.Category) as Category in output list of your initial query.