<h3 dir="auto">Describe the issue:</h3>
<p dir="auto">The type <code class="notranslate">numpy.typing.DTypeLike</code> cannot be validated in <a href="https://pydantic-docs.helpmanual.io/usage/models/" rel="nofollow">pydantic models</a>, leading to a <code class="notranslate">TypeError</code></p>
<p dir="auto">This is possibly a duplicate of <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="996588319" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/19875" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/19875/hovercard" href="https://github.com/numpy/numpy/issues/19875">#19875</a>; however, that issue is very broad and this is a lot more specific and narrowly scoped, so I thought I would raise it. Feel free to close if you decide this is in fact a duplicate.</p>
<h3 dir="auto">Reproduce the code example:</h3>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from pydantic import BaseModel
from numpy.typing import DTypeLike
      
class Model(BaseModel):
    dtype: DTypeLike
    
    class Config:
            arbitrary_types_allowed = True"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">pydantic</span> <span class="pl-k">import</span> <span class="pl-v">BaseModel</span>
<span class="pl-k">from</span> <span class="pl-s1">numpy</span>.<span class="pl-s1">typing</span> <span class="pl-k">import</span> <span class="pl-v">DTypeLike</span>
      
<span class="pl-k">class</span> <span class="pl-v">Model</span>(<span class="pl-v">BaseModel</span>):
    <span class="pl-s1">dtype</span>: <span class="pl-v">DTypeLike</span>
    
    <span class="pl-k">class</span> <span class="pl-v">Config</span>:
            <span class="pl-s1">arbitrary_types_allowed</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span></pre></div>
<h3 dir="auto">Error message:</h3>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_323/1951530187.py in &lt;module&gt;
      3 
      4 
----&gt; 5 class Model(BaseModel):
      6     dtype: DTypeLike
      7 

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.ModelMetaclass.__new__()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.infer()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.__init__()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.prepare()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField._type_analysis()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField._create_sub_type()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.__init__()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.prepare()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.populate_validators()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/validators.cpython-38-x86_64-linux-gnu.so in find_validators()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/validators.cpython-38-x86_64-linux-gnu.so in pydantic.validators.make_typeddict_validator()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/annotated_types.cpython-38-x86_64-linux-gnu.so in pydantic.annotated_types.create_model_from_typeddict()

TypeError: You should use `typing_extensions.TypedDict` instead of `typing.TypedDict`. Without it, there is no way to differentiate required and optional fields when subclassed."><pre class="notranslate">---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_323/1951530187.py <span class="pl-k">in</span> <span class="pl-k">&lt;</span>module<span class="pl-k">&gt;</span>
      3 
      4 
----<span class="pl-k">&gt;</span> 5 class Model(BaseModel):
      6     dtype: DTypeLike
      7 

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.main.ModelMetaclass.__new__</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.infer</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.__init__</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.prepare</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField._type_analysis</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField._create_sub_type</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.__init__</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.prepare</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.fields.ModelField.populate_validators</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/validators.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">find_validators</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/validators.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.validators.make_typeddict_validator</span>()

/srv/conda/envs/notebook/lib/python3.8/site-packages/pydantic/annotated_types.cpython-38-x86_64-linux-gnu.so <span class="pl-k">in</span> <span class="pl-en">pydantic.annotated_types.create_model_from_typeddict</span>()

TypeError: You should use <span class="pl-s"><span class="pl-pds">`</span>typing_extensions.TypedDict<span class="pl-pds">`</span></span> instead of <span class="pl-s"><span class="pl-pds">`</span>typing.TypedDict<span class="pl-pds">`</span></span>. Without it, there is no way to differentiate required and optional fields when subclassed.</pre></div>
<p dir="auto">Note that this error message is very helpful and suggests a fix, which would be made here:</p>
<p dir="auto"></p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/numpy/numpy/blob/874e4afce3cf81ff65feb74c9018e518ba0cd639/numpy/typing/_dtype_like.py#L10">numpy/numpy/typing/_dtype_like.py</a>
    </p>
    <p class="mb-0 color-fg-muted">
         Line 10
      in
      <a data-pjax="true" class="commit-tease-sha" href="/numpy/numpy/commit/874e4afce3cf81ff65feb74c9018e518ba0cd639">874e4af</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L10" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="10"></td>
          <td id="LC10" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-v">TypedDict</span>, </td>
        </tr>
    </tbody></table>
  </div>
</div>
<p></p>
<h3 dir="auto">NumPy/Python version information:</h3>
<ul dir="auto">
<li>numpy 1.21.2</li>
<li>pydantic 1.8.2</li>
<li>python 3.8.12</li>
</ul>
<p dir="auto">EDITS following different comments:</p>
<ul dir="auto">
<li>(<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="216096571" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8810" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/8810/hovercard?comment_id=288431404&amp;comment_type=issue_comment" href="https://github.com/numpy/numpy/issues/8810#issuecomment-288431404">#8810 (comment)</a>): include <code class="notranslate">keepdims</code>. See also previous discussion in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="46065008" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/5197" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/5197/hovercard" href="https://github.com/numpy/numpy/issues/5197">#5197</a> and on <a href="https://mail.scipy.org/pipermail/numpy-discussion/2014-October/071455.html" rel="nofollow">https://mail.scipy.org/pipermail/numpy-discussion/2014-October/071455.html</a> (which independently suggested more or less the same syntax).</li>
<li>see <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="216570678" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8819" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/8819/hovercard" href="https://github.com/numpy/numpy/pull/8819">#8819</a> for the implementation of <code class="notranslate">axes</code>; adjusted implementation notes accordingly.</li>
</ul>
<h2 dir="auto">Rationale</h2>
<p dir="auto">Following the <a href="https://github.com/numpy/numpy/issues/8797#issuecomment-287919544" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/8797/hovercard">suggestion</a> that, ideally, <code class="notranslate">np.median</code> and similar functions would be <code class="notranslate">gufuncs</code> so that they could be overridden with <code class="notranslate">__array_ufunc__</code> (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="187577313" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8247" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/8247/hovercard" href="https://github.com/numpy/numpy/pull/8247">#8247</a>), it was realised that the main show-stopper is the absence of an <code class="notranslate">axis</code> argument for gufuncs (also annoying in the context of the new <code class="notranslate">all_equal</code> gufuncs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="202936542" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8528" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/8528/hovercard" href="https://github.com/numpy/numpy/pull/8528">#8528</a>). Furthermore, for functions like <code class="notranslate">argmax</code>, etc., a <code class="notranslate">keepdims</code> would be very useful (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="210642534" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8710" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/8710/hovercard" href="https://github.com/numpy/numpy/issues/8710">#8710</a>).</p>
<h2 dir="auto">Specification</h2>
<p dir="auto">In practice, the new <code class="notranslate">axis</code> argument would use underneath a more general <code class="notranslate">axes</code> argument, which is a list of tuples of axes for each of the core dimensions. For normal <code class="notranslate">ufunc</code>, <code class="notranslate">axes</code> obviously has to be empty (or absent); for <code class="notranslate">gufunc</code>, defaults would be <code class="notranslate">-1</code>, <code class="notranslate">-2</code>, etc., as needed. For the output, the tuple would be extended by one (or possibly more) from that implied by the signature if <code class="notranslate">keepdims</code> is set . Short-cuts may be possible, e.g., for gufuncs with only one index, a single number (or perhaps even <code class="notranslate">None</code> for ravelling all dimensions); one would distinguish the general case of <code class="notranslate">axes</code> with the simple case of <code class="notranslate">axis</code>.</p>
<h2 dir="auto">Implementation</h2>
<p dir="auto">Looking at the <a href="https://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html" rel="nofollow">reference</a>, it would seem that the C-api would not need to change, but the routine actually calling the underlying iteration machinery in <code class="notranslate">umath/ufunc_object.c</code> would need to do appropriate transposes of the input and output arrays. Going through the call sequence to see places where change would be necessary (not marked with <code class="notranslate">X</code>):</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> <code class="notranslate">ufunc_generic_call</code> (all OK)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> <code class="notranslate">PyUFunc_CheckOverride</code> (OK, <code class="notranslate">axis</code> should already be in <code class="notranslate">kwds</code>)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> <code class="notranslate">PyUFunc_GenericFunction</code>: signal that non-empty <code class="notranslate">axis</code> should error.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> <code class="notranslate">PyUFunc_GeneralizedFunction</code>: do a remap of axes for the checks and before passing on to the iterator;</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> <code class="notranslate">get_ufunc_arguments</code>: interpret <code class="notranslate">axes</code> argument (new return argument); error if not needed. <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="216570678" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/8819" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/8819/hovercard" href="https://github.com/numpy/numpy/pull/8819">#8819</a></li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Implement the simple <code class="notranslate">axis</code> one for just a single axis. <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="318955660" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/11018" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/11018/hovercard" href="https://github.com/numpy/numpy/pull/11018">#11018</a></li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> repeat above for <code class="notranslate">keep_dims</code>; <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="318955660" data-permission-text="Title is private" data-url="https://github.com/numpy/numpy/issues/11018" data-hovercard-type="pull_request" data-hovercard-url="/numpy/numpy/pull/11018/hovercard" href="https://github.com/numpy/numpy/pull/11018">#11018</a></li>
</ul>
<h2 dir="auto">Excluded</h2>
<p dir="auto">In principle, it might be nice to allow <code class="notranslate">axis</code> to refer to multiple axes or all of them (<code class="notranslate">None</code>), but this is substantially more complicated, and it is not obvious it isn't better to let wrapping code deal with this. Indeed, this is done for <code class="notranslate">np.median</code>, which ravels any requested axes as needed before passing the array on to <code class="notranslate">partition</code> (which can handle only a single axis).</p>