Create Table With Aggregated Dates
You can use different formatters for the exact same field when you are in a different context. So, while the full node is shown, you can use the full date (as set in CT's Manage Display). When displaying the date in views, you can use the time ago. For correct grouping, add the date field once more, exclude it from display, then set the formatter to a custom one which only shows the year and finally use that rendered output to group the view's rows.