Gilman, Taylor County, Wisconsin. Gilman is a village in Taylor County, Wisconsin, United States. The population was 410 at the 2010 census. The village is located between the towns of Aurora and Ford.