The construction industry is one of the risky workplaces attributable to its complexity and dynamic nature of construction activities. High rise building construction remains predominant for high accident rates counted yearly. A site-specific assessment tool that considers the characteristics and changeable conditions of the currently managed construction site is necessary to precisely assess safety risks. Therefore, this study aims to address the significant safety risks and principal factors associated with the high rise building construction projects in Malaysia. Responses obtained within the construction management personnel are evaluated using factor analysis to understand the latent critical risk factors. Preliminary findings suggest that all safety risks are significant and working environment is identified as the most critical principal factor which consists of diverse underlying safety risks with high loading factors. Successful investigations of the study will lead to the development of a high rise building construction safety and health risk model. From the practical view, these findings should assist the high rise construction project participants to be more attentive to health and safety issues.