Health-related quality of life (HRQOL) represents an individual's perception of physical, mental, and social well-being and is a strong predictor of health status. Few studies have examined associations of sedentary behavior (SB) and moderate-to-vigorous physical activity (MVPA) with HRQOL in the general population. This study aimed to determine combined associations of objectively measured SB and MVPA on the risk of poor HRQOL in the general US population, after controlling for potential confounding factors. We analyzed data from 5359 adults from the National Health and Nutrition Examination Survey 2003-2006. HRQOL was measured using a HRQOL-4 consisting of four questions focused on the self-perception of general health, physical health, mental health, and activity limitation. We dichotomized each HRQOL-4 component as good versus poor and defined poor overall HRQOL when participants had any poor HRQOL components. SB and MVPA were measured using an accelerometer. Survey logistic models were examined to estimate the odds ratio (OR) and 95% confidence interval (CI) for poor HRQOL by SB and MVPA as tertiles. When examined as a combination of SB and MVPA, a substantial decrease in the risk of poor overall HRQOL was found in individuals with low SB/high MVPA [OR 0.69 (95% CI 0.51-0.94) and moderate SB/high MVPA (OR 0.56 (95% CI 0.40-0.78)], but no significant decrease was found in individuals with high SB/high MVPA (vs. high SB/low MVPA). Our findings suggest that both increasing MVPA and reducing time spent in SB may be useful strategies to improve HRQOL.