<p dir="auto">Hello everyone.<br>
I have a question.<br>
I have a simple scrapy project.<br>
I run my project like:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="scrapy crawl mp -a city=&quot;Toronto, ON&quot;"><pre class="notranslate">scrapy crawl mp -a city=<span class="pl-s"><span class="pl-pds">"</span>Toronto, ON<span class="pl-pds">"</span></span></pre></div>
<p dir="auto">In</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def __init__(self, *args, **kwargs):"><pre class="notranslate"><code class="notranslate">def __init__(self, *args, **kwargs):
</code></pre></div>
<p dir="auto">I get argument value like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="self.city = kwargs[&quot;city&quot;]"><pre class="notranslate"><code class="notranslate">self.city = kwargs["city"]
</code></pre></div>
<p dir="auto">In <code class="notranslate">seetings.py</code> I can add a line like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="FEED_FORMAT = &quot;csv&quot;
FEED_URI = &quot;output/some_city.csv&quot;"><pre class="notranslate"><code class="notranslate">FEED_FORMAT = "csv"
FEED_URI = "output/some_city.csv"
</code></pre></div>
<p dir="auto">And information about need city will be saved in <code class="notranslate">output/some_city.csv</code> file.</p>
<h5 dir="auto">Problem</h5>
<p dir="auto">But if I will be pass different cities like arguments to my spider, like:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="scrapy crawl mp -a city=&quot;New York, NY&quot;"><pre class="notranslate">scrapy crawl mp -a city=<span class="pl-s"><span class="pl-pds">"</span>New York, NY<span class="pl-pds">"</span></span></pre></div>
<p dir="auto">then</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="scrapy crawl mp -a city=&quot;Toronto, ON&quot;"><pre class="notranslate">scrapy crawl mp -a city=<span class="pl-s"><span class="pl-pds">"</span>Toronto, ON<span class="pl-pds">"</span></span></pre></div>
<p dir="auto">then</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="scrapy crawl mp -a city=&quot;Chicago, IL&quot;"><pre class="notranslate">scrapy crawl mp -a city=<span class="pl-s"><span class="pl-pds">"</span>Chicago, IL<span class="pl-pds">"</span></span></pre></div>
<p dir="auto">etc<br>
All information will be saved in  <code class="notranslate">output/some_city.csv</code> file.</p>
<h5 dir="auto">What I want</h5>
<p dir="auto">I want to find a decision, when I pass <code class="notranslate">"Chicago, IL"</code>  - data will be saved in <code class="notranslate">Chicago_IL.csv</code> if I pass <code class="notranslate">"Toronto, ON"</code> in <code class="notranslate">Toronto_ON.csv</code><br>
I just need to change <code class="notranslate">FEED_URI</code> in my spider.<br>
But how?</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="self.settings.attributes['FEED_URI']"><pre class="notranslate"><code class="notranslate">self.settings.attributes['FEED_URI']
</code></pre></div>
<p dir="auto">I can get my <code class="notranslate">FEED_URI</code> in spider use this command, but how change this data?<br>
I read this <a href="https://stackoverflow.com/questions/42134570/update-scrapy-settings-based-on-spider-property/42135262" rel="nofollow">link</a> but I wrote a solution and it does not work.<br>
I can use <code class="notranslate">-s</code> but it not conveniently.<br>
Do you know how I can resolve it?<br>
Thank you very much.</p>
<h3 dir="auto">Description</h3>
<p dir="auto">The static variables set in <code class="notranslate">scrapy.Items</code> class are not registered in the internal <code class="notranslate">_values</code> dict and therefore I cannot access them via <code class="notranslate">self['field_name']</code>, i.e., the <code class="notranslate">__getattr__</code> function.</p>
<h3 dir="auto">Steps to Reproduce</h3>
<p dir="auto">In my items_error.py file I have this. (the actual code is larger but I isolate the code causing the error)</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from scrapy import Item, Field

class BaseItem(Item):
    url=Field()
    id=Field()
    type=Field()

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if not self['type']: #&lt;--- error here. Raises KeyError
            raise NotImplementedError('type of ad must be set as static class variable')
        #use url hash as id to avoid duplicates
        self['id']=hash(self.get('url'))

class ChildItem(BaseItem):
    type='estate'
    other_fields=Field()"><pre class="notranslate"><code class="notranslate">from scrapy import Item, Field

class BaseItem(Item):
    url=Field()
    id=Field()
    type=Field()

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if not self['type']: #&lt;--- error here. Raises KeyError
            raise NotImplementedError('type of ad must be set as static class variable')
        #use url hash as id to avoid duplicates
        self['id']=hash(self.get('url'))

class ChildItem(BaseItem):
    type='estate'
    other_fields=Field()
</code></pre></div>
<p dir="auto">When I ran it this raises:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="C:\Users\ivangonzalez\Documents\Code\scrady&gt;python
Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] on win32
Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
&gt;&gt;&gt; from scrady.items_error import *
&gt;&gt;&gt; ChildItem()
Traceback (most recent call last):
  File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
  File &quot;C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py&quot;, line 10, in __init__
    if not self['type']: #&lt;--- error here. Raises KeyError
  File &quot;C:\Users\ivangonzalez\Anaconda3\lib\site-packages\scrapy\item.py&quot;, line 91, in __getitem__
    return self._values[key]
KeyError: 'type'
&gt;&gt;&gt; BaseItem()
Traceback (most recent call last):
  File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
  File &quot;C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py&quot;, line 10, in __init__
    if not self['type']: #&lt;--- error here. Raises KeyError
  File &quot;C:\Users\ivangonzalez\.virtualenvs\scrady-d4yjsu4b\lib\site-packages\scrapy\item.py&quot;, line 91, in __getitem__       return self._values[key]
KeyError: 'type'
&gt;&gt;&gt;"><pre class="notranslate"><code class="notranslate">C:\Users\ivangonzalez\Documents\Code\scrady&gt;python
Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; from scrady.items_error import *
&gt;&gt;&gt; ChildItem()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py", line 10, in __init__
    if not self['type']: #&lt;--- error here. Raises KeyError
  File "C:\Users\ivangonzalez\Anaconda3\lib\site-packages\scrapy\item.py", line 91, in __getitem__
    return self._values[key]
KeyError: 'type'
&gt;&gt;&gt; BaseItem()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py", line 10, in __init__
    if not self['type']: #&lt;--- error here. Raises KeyError
  File "C:\Users\ivangonzalez\.virtualenvs\scrady-d4yjsu4b\lib\site-packages\scrapy\item.py", line 91, in __getitem__       return self._values[key]
KeyError: 'type'
&gt;&gt;&gt;
</code></pre></div>
<p dir="auto"><strong>Expected behavior:</strong> <code class="notranslate">self['field_name']</code> should return the <code class="notranslate">Field()</code> attribute.</p>
<p dir="auto"><strong>Actual behavior:</strong> static fields are not registered by <code class="notranslate">scrapy.Items</code>  in <code class="notranslate">scrapy.Items._values[key]</code>.</p>
<p dir="auto"><strong>Reproduces how often:</strong> Ran in pipenv, so 100%.</p>
<h3 dir="auto">Versions</h3>
<p dir="auto">Please paste here the output of executing <code class="notranslate">scrapy version --verbose</code> in the command line:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="C:\Users\ivangonzalez\Documents\Code\scrady&gt;scrapy version --verbose
Scrapy       : 1.8.0
lxml         : 4.4.2.0
libxml2      : 2.9.5
cssselect    : 1.1.0
parsel       : 1.5.2
w3lib        : 1.21.0
Twisted      : 19.10.0
Python       : 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)]
pyOpenSSL    : 19.1.0 (OpenSSL 1.1.1d  10 Sep 2019)
cryptography : 2.8
Platform     : Windows-10-10.0.18362-SP0"><pre class="notranslate"><code class="notranslate">C:\Users\ivangonzalez\Documents\Code\scrady&gt;scrapy version --verbose
Scrapy       : 1.8.0
lxml         : 4.4.2.0
libxml2      : 2.9.5
cssselect    : 1.1.0
parsel       : 1.5.2
w3lib        : 1.21.0
Twisted      : 19.10.0
Python       : 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)]
pyOpenSSL    : 19.1.0 (OpenSSL 1.1.1d  10 Sep 2019)
cryptography : 2.8
Platform     : Windows-10-10.0.18362-SP0
</code></pre></div>
<h3 dir="auto">Additional context</h3>
<p dir="auto">In scrapy/item.py : ItemMeta class the attributes should be registered by this code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="class ItemMeta(ABCMeta):
    &quot;&quot;&quot;Metaclass_ of :class:`Item` that handles field definitions.

    .. _metaclass: https://realpython.com/python-metaclasses
    &quot;&quot;&quot;

    def __new__(mcs, class_name, bases, attrs):
        classcell = attrs.pop('__classcell__', None)
        new_bases = tuple(base._class for base in bases if hasattr(base, '_class'))
        _class = super(ItemMeta, mcs).__new__(mcs, 'x_' + class_name, new_bases, attrs)

        fields = getattr(_class, 'fields', {})
        new_attrs = {}
        for n in dir(_class):
            v = getattr(_class, n)
            if isinstance(v, Field):
                fields[n] = v
            elif n in attrs:
                new_attrs[n] = attrs[n] #&lt;--- here

        new_attrs['fields'] = fields
        new_attrs['_class'] = _class
        if classcell is not None:
            new_attrs['__classcell__'] = classcell
        return super(ItemMeta, mcs).__new__(mcs, class_name, bases, new_attrs)"><pre class="notranslate"><code class="notranslate">class ItemMeta(ABCMeta):
    """Metaclass_ of :class:`Item` that handles field definitions.

    .. _metaclass: https://realpython.com/python-metaclasses
    """

    def __new__(mcs, class_name, bases, attrs):
        classcell = attrs.pop('__classcell__', None)
        new_bases = tuple(base._class for base in bases if hasattr(base, '_class'))
        _class = super(ItemMeta, mcs).__new__(mcs, 'x_' + class_name, new_bases, attrs)

        fields = getattr(_class, 'fields', {})
        new_attrs = {}
        for n in dir(_class):
            v = getattr(_class, n)
            if isinstance(v, Field):
                fields[n] = v
            elif n in attrs:
                new_attrs[n] = attrs[n] #&lt;--- here

        new_attrs['fields'] = fields
        new_attrs['_class'] = _class
        if classcell is not None:
            new_attrs['__classcell__'] = classcell
        return super(ItemMeta, mcs).__new__(mcs, class_name, bases, new_attrs)
</code></pre></div>
<p dir="auto">But they aren't. I don't know why.</p>