In this paper, we introduce a novel three-dimensional (3D) reconstruction framework for ultrasound images using a piecewise smooth Markov random field (MRF) model from irregularly spaced B-scan images obtained by freehand scanning. Freehand 3D ultrasound imaging is a useful system for various clinical applications, including image-guided surgeries and interventions, as well as diagnoses, due to the variety of its scan ranges and relatively low cost. The reconstruction process performs a key role in this system because its sampling irregularities may cause undesired artifacts, and ultrasound images generally suffer from noise and distortions. However, traditional approaches are based on simple geometric interpolations, such as pixel-based or distance-weighted methods, which are sensitive to sampling density and speckle noise. These approaches generally have an additional limitation of smoothing objects boundaries. To reduce speckle noise and preserve boundaries, we devised a piecewise smooth (PS) MRF model and developed its optimization algorithm. In our framework, we can easily apply an individual noise level for each image pixel, which is specified by the characteristics of an ultrasound probe, and possibly, the lateral and axial positions of an image. As a result, the reconstructed volume has sharp object boundaries with reduced speckle noise and artifacts. Our PS-MRF model provides simple segmentation results within a reconstruction framework that is useful for various purposes, such as clear visualization. The corresponding optimization methods have also been developed, and we tested a virtual phantom and a physical phantom model. Experimental results show that our method outperforms existing methods in terms of interpolation and segmentation accuracy. With this method, all computations can be performed with practical time consumption and with an appropriate resolution, via parallel computing using graphic processing units. (C) 2016 Elsevier Inc. All rights reserved.