How to create new type element to form API?
You need to implement hook_element_info(). See the 
form_example_elements.inc file from the Example: Form API module, and the Creating Custom Elements documentation guide.

Note that the Elements module already provides a form element type for HTML5 mail inputs.