Jamal Hakrush starts his job as deputy commissioner after months of violence between Israelis and Palestinians.
He will oversee policing in Arab communities where there is a longstanding distrust of the police.
A fifth of Israel's population is Arab and they often complain that areas in which they live are not so well policed and have poorer public services.
Their grievances have been supported by Human Rights Watch which in recent years has published several reports highlighting the discrimination which it is argued the Arab population faces.
Deputy Commissioner Hakrush, from the Galilee village of Kafr Kanna, will be in charge of a newly-created police division established to improve policing in Arab communities, The Times of Israel reported.
Deputy Commissioner Hakrush was formally appointed into his new position on Wednesday at a ceremony attended by Police Commissioner Roni Alsheich. One of his main responsibilities will be to stop illegal weapons falling into the hands of the Arab community.
Mr Alsheich also wants to reduce domestic violence, murder rates and other crimes in the Arab sector.
He and the government wants to recruit 1,300 new officers and construct several new stations in Arab population centres.
An Arab police officer has been promoted by Israel to the highest rank ever attained by a Muslim in the force.