Recently, machine learning has emerged as a unique and powerful tool for the development of bottom-up coarse-graining methods. It has shown remarkable potential in improving essential components of coarse-graining methods, including determining optimal coarse-grained mapping schemes, facilitating back-mapping processes, refining coarse-grained force fields, and optimizing associated parameters. Moreover, an increasingly noteworthy technique garnering attention is the differentiable simulations (Diff-Sim). DiffSim built on automatic differentiation has been utilized to accelerate quantum calculations, generate protein structures, and computational fluid dynamics. In molecular simulations, Greener et al. used DiffSim to learn parameters of simple force-field functionals for coarse-grained simulations of small proteins. Gómez-Bombarelli and co-workers improved the memory efficiency of DiffSim using the adjoint ordinary differentiation equation. This augmented DiffSim is successfully applied to learn pairwise neural network potentials in simple condensed systems and free energy barriers in systems with rare events. Although it is significant for learning dynamics and non-equilibrium behaviors, differentiating through a (long) simulation trajectory may generate unreasonable gradients due to the chaotic nature of the dynamical system of molecular dynamics. To alleviate this issue, Differentiable Trajectory Reweighting (DiffTre) provides a promising alternative that bypasses differentiation through MD simulation for time-independent observables via combinations of automatic differentiation with statistical reweighting. The utilization of DiffSim has proven to be remarkably successful in developing CG mod-els of simple liquids in condensed phase. Building upon this foundation, we introduce a novel structural coarse-graining approach, namely, differentiable coarse-graining (DiffCG). This novel method extends the capabilities of DiffSim to coarse-graining complex systems in condensed phases, such as polymer melts. To alleviate the instability of gradients from differentiating through a long trajectory in DiffSim, DiffTre is employed to obtain stable gradients of CG potentials. Notably, the extension to more complex condensed-phase systems is achieved through the integration of multi-objective optimization techniques. Specifically, three multi-objective optimization algorithms are selected, which are popular in machine learning community. These algorithms are meticulously benchmarked using a representative system, i.e., mono-disperse polystyrene (PS) melts, which has been widely used for the evaluation of coarse-grained models. The conventional CG force fields (bond, angle, dihedral and pairwise interactions) are employed to describe CG-PS system and they are optimized to simultaneously match multiple target properties, encompassing the corresponding structural correlations and pressure characteristics. More importantly, leveraging the capability of multi-objective optimization in DiffCG, we have developed a temperature-transferable CG model of PS melts using (non-linear) temperature-dependent potentials. The effectiveness of DiffCG in developing CG models of PS melts is comprehensively examined through a systematic evaluation in terms of the reproduction of target properties, transferability across a wide range of temperatures and molecular weights, and the optimization efficiency.