Applications of remote sensing using unmanned aerial vehicle (UAV) in agriculture has proved to be an effective and efficient way of obtaining field information. In this study, we validated the feasibility of utilizing multi-temporal color images acquired from a low altitude UAV-camera system to monitor real-time wheat growth status and to map within-field spatial variations of wheat yield for smallholder wheat growers, which could serve as references for site-specific operations. Firstly, eight orthomosaic images covering a small winter wheat field were generated to monitor wheat growth status from heading stage to ripening stage in Hokkaido, Japan. Multi-temporal orthomosaic images indicated straightforward sense of canopy color changes and spatial variations of tiller densities. Besides, the last two orthomosaic images taken from about two weeks prior to harvesting also notified the occurrence of lodging by visual inspection, which could be used to generate navigation maps guiding drivers or autonomous harvesting vehicles to adjust operation speed according to specific lodging situations for less harvesting loss. Subsequently orthomosaic images were geo-referenced so that further study on stepwise regression analysis among nine wheat yield samples and five color vegetation indices (CVI) could be conducted, which showed that wheat yield correlated with four accumulative CVIs of visible-band difference vegetation index (VDVI), normalized green-blue difference index (NGBDI), green-red ratio index (GRRI), and excess green vegetation index (ExG), with the coefficient of determination and RMSE as 0.94 and 0.02, respectively. The average value of sampled wheat yield was 8.6 t/ha. The regression model was also validated by using leave-one-out cross validation (LOOCV) method, of which root-mean-square error of predication (RMSEP) was 0.06. Finally, based on the stepwise regression model, a map of estimated wheat yield was generated, so that within-field spatial variations of wheat yield, which was usually seen as general information on soil fertility, water potential, tiller density, etc., could be better understood for applications of site-specific or variable-rate operations. Average yield of the studied field was also calculated according to the map of wheat yield as 7.2 t/ha.