codecov[bot]: # [Codecov](https://codecov.io/gh/scrapy/scrapy/pull/4052?src=pr&el=h1) Report
> Merging [#4052](https://codecov.io/gh/scrapy/scrapy/pull/4052?src=pr&el=desc) into [master](https://codecov.io/gh/scrapy/scrapy/commit/7632e375c268d31c570904452902799db180c415?src=pr&el=desc) will **decrease** coverage by `2.74%`.
> The diff coverage is `100%`.


```diff
@@            Coverage Diff             @@
##           master    #4052      +/-   ##
==========================================
- Coverage   85.68%   82.94%   -2.75%     
==========================================
  Files         165      165              
  Lines        9732     9734       +2     
  Branches     1463     1463              
==========================================
- Hits         8339     8074     -265     
- Misses       1136     1394     +258     
- Partials      257      266       +9
```

| [Impacted Files](https://codecov.io/gh/scrapy/scrapy/pull/4052?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [scrapy/extensions/corestats.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvY29yZXN0YXRzLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [scrapy/linkextractors/sgml.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2xpbmtleHRyYWN0b3JzL3NnbWwucHk=) | `0% <0%> (-96.81%)` | :arrow_down: |
| [scrapy/linkextractors/regex.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2xpbmtleHRyYWN0b3JzL3JlZ2V4LnB5) | `0% <0%> (-95.66%)` | :arrow_down: |
| [scrapy/linkextractors/htmlparser.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2xpbmtleHRyYWN0b3JzL2h0bWxwYXJzZXIucHk=) | `0% <0%> (-92.07%)` | :arrow_down: |
| [scrapy/robotstxt.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L3JvYm90c3R4dC5weQ==) | `57.89% <0%> (-39.48%)` | :arrow_down: |
| [scrapy/extensions/statsmailer.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvc3RhdHNtYWlsZXIucHk=) | `0% <0%> (-30.44%)` | :arrow_down: |
| [scrapy/\_monkeypatches.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L19tb25rZXlwYXRjaGVzLnB5) | `54.54% <0%> (-18.19%)` | :arrow_down: |
| [scrapy/link.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L2xpbmsucHk=) | `86.36% <0%> (-13.64%)` | :arrow_down: |
| [scrapy/utils/gz.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L3V0aWxzL2d6LnB5) | `91.89% <0%> (-8.11%)` | :arrow_down: |
| [scrapy/utils/python.py](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree#diff-c2NyYXB5L3V0aWxzL3B5dGhvbi5weQ==) | `78.26% <0%> (-5.98%)` | :arrow_down: |
| ... and [14 more](https://codecov.io/gh/scrapy/scrapy/pull/4052/diff?src=pr&el=tree-more) | |

diff --git a/scrapy/extensions/corestats.py b/scrapy/extensions/corestats.py
index 8cc5e18ac..20adfbe4b 100644
--- a/scrapy/extensions/corestats.py
+++ b/scrapy/extensions/corestats.py
@@ -1,14 +1,16 @@
 """
 Extension for collecting core stats like items scraped and start/finish times
 """
-import datetime
+from datetime import datetime
 
 from scrapy import signals
 
+
 class CoreStats(object):
 
     def __init__(self, stats):
         self.stats = stats
+        self.start_time = None
 
     @classmethod
     def from_crawler(cls, crawler):
@@ -21,11 +23,12 @@ class CoreStats(object):
         return o
 
     def spider_opened(self, spider):
-        self.stats.set_value('start_time', datetime.datetime.utcnow(), spider=spider)
+        self.start_time = datetime.utcnow()
+        self.stats.set_value('start_time', self.start_time, spider=spider)
 
     def spider_closed(self, spider, reason):
-        finish_time = datetime.datetime.utcnow()
-        elapsed_time = finish_time - self.stats.get_value('start_time')
+        finish_time = datetime.utcnow()
+        elapsed_time = finish_time - self.start_time
         elapsed_time_seconds = elapsed_time.total_seconds()
         self.stats.set_value('elapsed_time_seconds', elapsed_time_seconds, spider=spider)
         self.stats.set_value('finish_time', finish_time, spider=spider)

Exception when using DummyStatsCollector
### Description

Using the DummyStatsCollector results in an exception:

```
2019-09-09 13:51:23 [scrapy.utils.signal] ERROR: Error caught on signal handler: <bound method CoreStats.spider_closed of <scrapy.extensions.corestats.CoreStats object at 0x7f86269cac18>>
Traceback (most recent call last):
  File ".../lib/python3.6/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File ".../lib/python3.6/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File ".../lib/python3.6/site-packages/scrapy/extensions/corestats.py", line 28, in spider_closed
    elapsed_time = finish_time - self.stats.get_value('start_time')
TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'
```

This problem has been introduced in aa46e1995cd5cb1099aba17535372b538bd656b3.

### Steps to Reproduce

Set `STATS_CLASS = "scrapy.statscollectors.DummyStatsCollector"` in the settings module as described in the documentation (https://docs.scrapy.org/en/latest/topics/stats.html#dummystatscollector).

**Expected behavior:** no exception
**Actual behavior:** exception thrown
**Reproduces how often:** always

### Versions

At least master as of 534de7395da3a53b5a2c89960db9ec5d8fdab60c

### Fix

A possible fix is to use the elapsed time as a default argument so that `get_value()` does not return None. I can prepare a PR if needed.

```diff
--- a/scrapy/extensions/corestats.py
+++ b/scrapy/extensions/corestats.py
@@ -25,7 +25,7 @@ class CoreStats(object):
 
     def spider_closed(self, spider, reason):
         finish_time = datetime.datetime.utcnow()
-        elapsed_time = finish_time - self.stats.get_value('start_time')
+        elapsed_time = finish_time - self.stats.get_value('start_time', finish_time)
         elapsed_time_seconds = elapsed_time.total_seconds()
         self.stats.set_value('elapsed_time_seconds', elapsed_time_seconds, spider=spider)
         self.stats.set_value('finish_time', finish_time, spider=spider)
```
diff --git a/tests/test_stats.py b/tests/test_stats.py
index 9f950ebc9..2033dbe07 100644
--- a/tests/test_stats.py
+++ b/tests/test_stats.py
@@ -1,10 +1,59 @@
+from datetime import datetime
 import unittest
 
+try:
+    from unittest import mock
+except ImportError:
+    import mock
+
+from scrapy.extensions.corestats import CoreStats
 from scrapy.spiders import Spider
 from scrapy.statscollectors import StatsCollector, DummyStatsCollector
 from scrapy.utils.test import get_crawler
 
 
+class CoreStatsExtensionTest(unittest.TestCase):
+
+    def setUp(self):
+        self.crawler = get_crawler(Spider)
+        self.spider = self.crawler._create_spider('foo')
+
+    @mock.patch('scrapy.extensions.corestats.datetime')
+    def test_core_stats_default_stats_collector(self, mock_datetime):
+        fixed_datetime = datetime(2019, 12, 1, 11, 38)
+        mock_datetime.utcnow = mock.Mock(return_value=fixed_datetime)
+        self.crawler.stats = StatsCollector(self.crawler)
+        ext = CoreStats.from_crawler(self.crawler)
+        ext.spider_opened(self.spider)
+        ext.item_scraped({}, self.spider)
+        ext.response_received(self.spider)
+        ext.item_dropped({}, self.spider, ZeroDivisionError())
+        ext.spider_closed(self.spider, 'finished')
+        self.assertEqual(
+            ext.stats._stats,
+            {
+                'start_time': fixed_datetime,
+                'finish_time': fixed_datetime,
+                'item_scraped_count': 1,
+                'response_received_count': 1,
+                'item_dropped_count': 1,
+                'item_dropped_reasons_count/ZeroDivisionError': 1,
+                'finish_reason': 'finished',
+                'elapsed_time_seconds': 0.0,
+            }
+        )
+
+    def test_core_stats_dummy_stats_collector(self):
+        self.crawler.stats = DummyStatsCollector(self.crawler)
+        ext = CoreStats.from_crawler(self.crawler)
+        ext.spider_opened(self.spider)
+        ext.item_scraped({}, self.spider)
+        ext.response_received(self.spider)
+        ext.item_dropped({}, self.spider, ZeroDivisionError())
+        ext.spider_closed(self.spider, 'finished')
+        self.assertEqual(ext.stats._stats, {})
+
+
 class StatsCollectorTest(unittest.TestCase):
 
     def setUp(self):

pip install -e .[dev]
parse_log_pytest
pytest
pytest pytest-cov pytest-xdist pytest-mock pytest-asyncio
apt-get update apt-get install -y gcc libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev
3.9
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
attrs==25.3.0
Automat==24.8.1
cffi==1.17.1
constantly==23.10.4
coverage==7.8.0
cryptography==44.0.2
cssselect==1.3.0
exceptiongroup @ file:///croot/exceptiongroup_1706031385326/work
execnet==2.1.1
hyperlink==21.0.0
idna==3.10
incremental==24.7.2
iniconfig @ file:///home/linux1/recipes/ci/iniconfig_1610983019677/work
jmespath==1.0.1
lxml==5.3.1
packaging @ file:///croot/packaging_1734472117206/work
parsel==1.10.0
pluggy @ file:///croot/pluggy_1733169602837/work
Protego==0.4.0
pyasn1==0.6.1
pyasn1_modules==0.4.2
pycparser==2.22
PyDispatcher==2.0.7
pyOpenSSL==25.0.0
pytest @ file:///croot/pytest_1738938843180/work
pytest-asyncio==0.26.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pytest-xdist==3.6.1
queuelib==1.7.0
-e git+https://github.com/scrapy/scrapy.git@74b4a5c77c0017106f6ba65b21b8bf2a1ed02f7b#egg=Scrapy
service-identity==24.2.0
six==1.17.0
tomli @ file:///opt/conda/conda-bld/tomli_1657175507142/work
Twisted==24.11.0
typing_extensions==4.13.0
w3lib==2.3.1
zope.interface==7.2

name: scrapy
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - exceptiongroup=1.2.0=py39h06a4308_0
  - iniconfig=1.1.1=pyhd3eb1b0_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - packaging=24.2=py39h06a4308_0
  - pip=25.0=py39h06a4308_0
  - pluggy=1.5.0=py39h06a4308_0
  - pytest=8.3.4=py39h06a4308_0
  - python=3.9.21=he870216_1
  - readline=8.2=h5eee18b_0
  - setuptools=75.8.0=py39h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - tomli=2.0.1=py39h06a4308_0
  - tzdata=2025a=h04d1e81_0
  - wheel=0.45.1=py39h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - attrs==25.3.0
      - automat==24.8.1
      - cffi==1.17.1
      - constantly==23.10.4
      - coverage==7.8.0
      - cryptography==44.0.2
      - cssselect==1.3.0
      - execnet==2.1.1
      - hyperlink==21.0.0
      - idna==3.10
      - incremental==24.7.2
      - jmespath==1.0.1
      - lxml==5.3.1
      - parsel==1.10.0
      - protego==0.4.0
      - pyasn1==0.6.1
      - pyasn1-modules==0.4.2
      - pycparser==2.22
      - pydispatcher==2.0.7
      - pyopenssl==25.0.0
      - pytest-asyncio==0.26.0
      - pytest-cov==6.0.0
      - pytest-mock==3.14.0
      - pytest-xdist==3.6.1
      - queuelib==1.7.0
      - service-identity==24.2.0
      - six==1.17.0
      - twisted==24.11.0
      - typing-extensions==4.13.0
      - w3lib==2.3.1
      - zope-interface==7.2
prefix: /opt/conda/envs/scrapy

tests/test_stats.py::CoreStatsExtensionTest::test_core_stats_default_stats_collector tests/test_stats.py::CoreStatsExtensionTest::test_core_stats_dummy_stats_collector
tests/test_stats.py::StatsCollectorTest::test_collector tests/test_stats.py::StatsCollectorTest::test_dummy_collector
scrapy/extensions/corestats.py
scrapy/extensions/corestats.py