Getting the top 10 stores within a category
There are some things that you don't need here

SELECT u.id, 
       u.avatar, 
       u.storeName,
       (SELECT COUNT(o.id) FROM orders o WHERE o.storeId = u.id) AS orders
FROM users u
WHERE (SELECT c.id
       FROM categories c, products p
       WHERE c.id = p.categoryId
       AND p.userId = u.id
       GROUP BY p.categoryId
       ORDER BY COUNT(p.id) DESC
       LIMIT 1) = 9
ORDER BY orders DESC
LIMIT 10


like the ORDER BY and the LIMIT 1 and the GROUP BY in the Select statement that is nested in your where clause.

SELECT u.id, 
       u.avatar, 
       u.storeName,
       (SELECT COUNT(o.id) FROM orders o WHERE o.storeId = u.id) AS orders
FROM users u
WHERE (SELECT c.id
       FROM categories c, products p
       WHERE c.id = p.categoryId
       AND p.userId = u.id
       ) = 9
ORDER BY orders DESC
LIMIT 10


You should also rename your column orders to orderCount

one more thing that I would suggest is to take it all out of the WHERE clause, this is slow

do it like this

SELECT u.id, 
       u.avatar, 
       u.storeName,
       (SELECT COUNT(o.id) FROM orders o WHERE o.storeId = u.id) AS orderCount
FROM users u INNER JOIN products p ON u.id = p.userId 
WHERE p.categoryID = 9
ORDER BY orderCount DESC
LIMIT 10