Identical aggregation
Let $x_i(p, w_i)$ be consumer $i$'s (Marshallian) demand and $y_j(p)$ be firm $j$'s profit-maximizing supply at price $p$.
A competitive equilibrium is a price $p$ such that
$$
\sum_{i=1}^I x_i(p, w_i(p, e_i)) = \sum_{j=1}^J y_j(p).
$$

By strict concavity of $u_i$'s, consumer demands are functions, rather than correspondences. Assume also that $y_j(p)$ are functions.
There can be multiple goods---e.g. consumption good/time(labor)/etc.
Agent's wealth $w_i$ may depend on price $p$ and endowment $e_i$---e.g. in an Edgeworth box where demands are Slutsky demands at the endowments---or wealth transfer $w_i$ can be specified as part of equilibrium, subject to
$$
\sum_i w_i = p^T (\sum_j y_j(p) + \sum_i e_i),
$$
i.e. aggregate wealth equals total value of goods.

Aggregating Consumers
Under the assumption that $u_i$'s are strictly concave (and differentiable, say), one can define a representative consumer via a linear social welfare function:
$$
u_{rep}(x) = \max_{\sum_{i} x_i \leq x}\sum_i \frac{1}{I} u_i(x_i).
$$
Then $u_{rep}$ is also strictly concave. Let $u_{rep, l}$ denotes marginal utility with respect to good $l$, then by the Envelope and Lagrange Theorems,
$$
\frac{u_{rep, l}}{u_{rep, k}} = \frac{u_{i, l}}{u_{i, k}}, \;\; \mbox{for all $i$}.
$$
At a competitive equilibrium price $p$, the MRS of $u_{rep}$ at the aggregate
demand $\sum_i x_i(p)$ is the ratio of prices:
$$
\frac{u_{rep, l} (\sum_i x_i(p),w_i)}{u_{rep, k}(\sum_i x_i(p), w_i)} =  \frac{p_l}{p_k}. \quad(*)
$$
By concavity of $u_{rep}$, the FOC $(*)$ implies that the aggregate demand at $p$ is the optimal bundle for the representative consumer $u_{rep}$ at $p$:
$$
x_{rep}(p, \sum_i w_i) = \sum_i x_i(p, w_i).
$$

As you already started to say in the question, this is a general construction of a (local) representative agent under concavity assumption. By the First Welfare Theorem, CE is Pareto. Under concavity of $u_i$'s, Pareto allocations must arise from a linear SWF, which implies existence of a representative consumer.
In this specific instance, consumers receive equal social weights $\alpha_i = \frac{1}{I}$ because equilibrium wealth is equal across agents. In general, $\alpha_i = \frac{1}{\lambda_i}$ where $\lambda_i$ is consumer $i$'s Lagrange multipler. In particular, $\alpha_i$ would be increasing in agent $i$'s share of aggregate wealth.
If $u_i = u$ is homothetic, with possibly different levels of wealth across agents, one can simply take $u_{rep} = u$ trivially. (Concavity assumption not needed.)
Gorman aggregation, alluded to in the comments, is a slightly different context, I believe. Gorman aggregation says that if all wealth expansion curves are parallel at all wealth distributions, then aggregate demand is a function of aggregate wealth and one has a global representative agent. The notion of Gorman aggregation is independent of equilibrium or Pareto efficiency. Here in your question, the local representative agent depends on the initial CE allocation---e.g. different endowments distributions would lead to different social weights.

Aggregating Firms
As pointed out in the comments, to aggregate firms one just uses the CRS assumption and take $F_{rep} = F$. For a CRS firm, doubling the input means output also doubles, with the same marginal products. In equilibrium, the firm still makes zero profit. For any one of the $J$ firms, aggregate production $\sum_j y_j(p)$ would also be optimal at the same equilibrium price $p$.
At the given equilibrium price $p$, $p = \nabla F(y_j(p))$, the gradient vector of marginal products of firm $j$ for all $j$. By the CRS/homogeneous-of-degree-one property,
$$
\nabla F_{rep}(\sum_j y_j(p)) = \nabla F(y_1(p)) = p,
$$
which tells you $y_{rep}(p) = \sum_j y_j(p)$.
Putting it together, given CE with allocation
$\{(x_i(p))_i, (y_j(p))_j \}$ and price $p$,
the allocation $\{\sum_i x_i(p), \sum_j y_j(p) \}$ and price $p$
is a CE for the economy with one agent $u_{rep}$ and one firm $F_{rep}$.