Linq query performance that use ToList()
You can indeed improve the query by changing the strategy. 

If you first GroupBy file extension and Count them inside the GroupBy element selector you can remove ToList and the query will be executed on the server and it won't be necessary to download all the data to group it later on your machine. You will then only get a few groups and their number of elements.

Now for the few file extensions it's very easy and quick to get the category names:

var result =
    Files
    .GroupBy(x =&gt; x.Extension, (ext, fs) =&gt; new 
    { 
        Extension = ext, 
        Category = Categories.Single(c =&gt; c.Extension == ext).Name, 
        FileCount = fs.Count() 
    })
    // the second GroupBy sums categories with multiple file extensions like pdf, doc etc.
    .GroupBy(x =&gt; x.Category, (cat, counts) =&gt; new
    {
        Category = cat,
        FileCount = counts.Sum(y =&gt; y.FileCount)
    });


This will produce the following query:

SELECT SUM([t3].[value2]) AS [FileCount], [t3].[value] AS [Category]
FROM (
    SELECT (
        SELECT [t2].[Name]
        FROM [Category] AS [t2]
        WHERE [t2].[Extension] = [t1].[Extension]
        ) AS [value], [t1].[value] AS [value2]
    FROM (
        SELECT COUNT(*) AS [value], [t0].[Extension]
        FROM [File] AS [t0]
        GROUP BY [t0].[Extension]
        ) AS [t1]
    ) AS [t3]
GROUP BY [t3].[value]






I've tested it with 1.000.000 rows: 



and compared with the ToList version:



In the last rows of the statistics you can see that short version is about 3 times faster then the original one.