mabelvj: I am having trouble with the tests since it seems we should start a crawling for getting the result from `start_requests` -> https://github.com/scrapy/scrapy/pull/4170/commits/96f7e976fe4a9302b05ab5cec8def480bf58a862.

Maybe just checking when trying to set the variable could be a good solution.
codecov[bot]: # [Codecov](https://codecov.io/gh/scrapy/scrapy/pull/4170?src=pr&el=h1) Report
> Merging [#4170](https://codecov.io/gh/scrapy/scrapy/pull/4170?src=pr&el=desc) into [master](https://codecov.io/gh/scrapy/scrapy/commit/78ad01632f16a97fb180d8c2972075b19f471380?src=pr&el=desc) will **decrease** coverage by `0.38%`.
> The diff coverage is `77.77%`.


```diff
@@            Coverage Diff            @@
##           master   #4170      +/-   ##
=========================================
- Coverage   83.69%   83.3%   -0.39%     
=========================================
  Files         165     165              
  Lines        9717    9616     -101     
  Branches     1445    1447       +2     
=========================================
- Hits         8133    8011     -122     
- Misses       1336    1355      +19     
- Partials      248     250       +2
```

| [Impacted Files](https://codecov.io/gh/scrapy/scrapy/pull/4170?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [scrapy/spiders/\_\_init\_\_.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L3NwaWRlcnMvX19pbml0X18ucHk=) | `95.52% <77.77%> (-2.9%)` | :arrow_down: |
| [scrapy/robotstxt.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L3JvYm90c3R4dC5weQ==) | `73.68% <0%> (-23.88%)` | :arrow_down: |
| [scrapy/core/downloader/tls.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2NvcmUvZG93bmxvYWRlci90bHMucHk=) | `79.06% <0%> (-6.98%)` | :arrow_down: |
| [scrapy/utils/trackref.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L3V0aWxzL3RyYWNrcmVmLnB5) | `83.78% <0%> (-2.71%)` | :arrow_down: |
| [scrapy/extensions/memdebug.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvbWVtZGVidWcucHk=) | `47.61% <0%> (-2.39%)` | :arrow_down: |
| [scrapy/extensions/debug.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvZGVidWcucHk=) | `44.73% <0%> (-1.42%)` | :arrow_down: |
| [scrapy/extensions/throttle.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvdGhyb3R0bGUucHk=) | `44.89% <0%> (-1.11%)` | :arrow_down: |
| [scrapy/pipelines/\_\_init\_\_.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L3BpcGVsaW5lcy9fX2luaXRfXy5weQ==) | `91.66% <0%> (-0.65%)` | :arrow_down: |
| [scrapy/extensions/telnet.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvdGVsbmV0LnB5) | `80.95% <0%> (-0.59%)` | :arrow_down: |
| [scrapy/extensions/memusage.py](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree#diff-c2NyYXB5L2V4dGVuc2lvbnMvbWVtdXNhZ2UucHk=) | `48.35% <0%> (-0.57%)` | :arrow_down: |
| ... and [59 more](https://codecov.io/gh/scrapy/scrapy/pull/4170/diff?src=pr&el=tree-more) | |

kmike: Hey! There are real-world spiders which use start_url option, as a way to customize things (found them using our internal code search). Is there a way to distinguish between intended and unintended usage? It is not good to show a warning for a perfectly valid use case.

I wonder if a better fix would be to have a warning in the default start_requests implementation: if start_urls is None, but start_url is present, then show a warning. There won't be a warning if start_requests function is using `start_url` attribute in this case, as it'd be overridden. 

It could make sense to check real spiders which use start_request attribute, to see how any change we make affect them.
mabelvj: You are right @kmike, that was one of my other options. Setting this warning in `start_requests` will allow overriding it when using `start_url`.
diff --git a/scrapy/spiders/__init__.py b/scrapy/spiders/__init__.py
index 94095bc27..517576ca6 100644
--- a/scrapy/spiders/__init__.py
+++ b/scrapy/spiders/__init__.py
@@ -58,6 +58,11 @@ class Spider(object_ref):
 
     def start_requests(self):
         cls = self.__class__
+        if not self.start_urls and hasattr(self, 'start_url'):
+            raise AttributeError(
+                "Crawling could not start: 'start_urls' not found "
+                "or empty (but found 'start_url' attribute instead, "
+                "did you miss an 's'?)")
         if method_is_overridden(cls, Spider, 'make_requests_from_url'):
             warnings.warn(
                 "Spider.make_requests_from_url method is deprecated; it "

Handle it gracefully when start_url is used instead of start_urls
Over the last year I’ve seen a few cases ([recent example](https://stackoverflow.com/q/58664004/939364)) of this, people missing the `s` at the end of the `start_urls`.

It may be nice to find a way to gracefully let the developer know where the issue is, why there is no crawling happening.
diff --git a/tests/test_spider.py b/tests/test_spider.py
index c0fccfdd6..3cebdf7ca 100644
--- a/tests/test_spider.py
+++ b/tests/test_spider.py
@@ -385,6 +385,14 @@ class CrawlSpiderTest(SpiderTest):
         self.assertTrue(hasattr(spider, '_follow_links'))
         self.assertFalse(spider._follow_links)
 
+    def test_start_url(self):
+        spider = self.spider_class("example.com")
+        spider.start_url = 'https://www.example.com'
+
+        with self.assertRaisesRegex(AttributeError,
+                                    r'^Crawling could not start.*$'):
+            list(spider.start_requests())
+
 
 class SitemapSpiderTest(SpiderTest):
 

pip install -e .[dev]
parse_log_pytest
requirements.txt
pytest pytest-cov pytest-xdist pytest-mock pytest-asyncio
apt-get update apt-get install -y gcc
3.9
tests/requirements-py3.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
asgiref==3.5.2
asttokens==3.0.0
attrs==25.3.0
Automat==24.8.1
blessed==1.20.0
bpython==0.25
Brotli==1.0.9
brotlipy==0.7.0
certifi==2025.1.31
cffi==1.17.1
charset-normalizer==3.4.1
click==8.1.8
constantly==23.10.4
coverage==7.8.0
cryptography==38.0.4
cssselect==1.3.0
curtsies==0.4.2
cwcwidth==0.1.10
decorator==5.2.1
exceptiongroup==1.2.2
execnet==2.1.1
executing==2.2.0
Flask==2.2.5
greenlet==3.1.1
h11==0.14.0
h2==4.2.0
hpack==4.1.0
hyperframe==6.1.0
hyperlink==21.0.0
idna==3.10
importlib_metadata==8.6.1
incremental==24.7.2
iniconfig==2.1.0
ipython==8.18.1
itsdangerous==2.2.0
jedi==0.19.2
Jinja2==3.1.6
jmespath==1.0.1
kaitaistruct==0.10
ldap3==2.9.1
lxml==5.3.1
MarkupSafe==3.0.2
matplotlib-inline==0.1.7
mitmproxy==9.0.1
mitmproxy_wireguard==0.1.23
msgpack==1.0.8
packaging==24.2
parsel==1.10.0
parso==0.8.4
passlib==1.7.4
pexpect==4.9.0
pluggy==1.5.0
prompt_toolkit==3.0.50
Protego==0.4.0
protobuf==4.25.6
ptyprocess==0.7.0
publicsuffix2==2.20191221
pure_eval==0.2.3
pyasn1==0.6.1
pyasn1_modules==0.4.2
pycparser==2.22
PyDispatcher==2.0.7
Pygments==2.19.1
pyOpenSSL==22.1.0
pyparsing==3.0.9
pyperclip==1.8.2
pytest==8.3.5
pytest-asyncio==0.26.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pytest-twisted==1.14.3
pytest-xdist==3.6.1
pyxdg==0.28
queuelib==1.7.0
requests==2.32.3
ruamel.yaml==0.17.40
ruamel.yaml.clib==0.2.12
-e git+https://github.com/scrapy/scrapy.git@82c01c261fc89b1ade289a1f07c4985854cd726b#egg=Scrapy
service-identity==24.2.0
six==1.17.0
sortedcontainers==2.4.0
stack-data==0.6.3
sybil==9.1.0
testfixtures==8.3.0
tomli==2.2.1
tornado==6.4.2
traitlets==5.14.3
Twisted==24.11.0
typing_extensions==4.13.0
urllib3==2.3.0
urwid==2.1.2
w3lib==2.3.1
wcwidth==0.2.13
Werkzeug==3.1.3
wsproto==1.2.0
zipp==3.21.0
zope.interface==7.2
zstandard==0.19.0

name: scrapy
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - pip=25.0=py39h06a4308_0
  - python=3.9.21=he870216_1
  - readline=8.2=h5eee18b_0
  - setuptools=75.8.0=py39h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - tzdata=2025a=h04d1e81_0
  - wheel=0.45.1=py39h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - asgiref==3.5.2
      - asttokens==3.0.0
      - attrs==25.3.0
      - automat==24.8.1
      - blessed==1.20.0
      - bpython==0.25
      - brotli==1.0.9
      - brotlipy==0.7.0
      - certifi==2025.1.31
      - cffi==1.17.1
      - charset-normalizer==3.4.1
      - click==8.1.8
      - constantly==23.10.4
      - coverage==7.8.0
      - cryptography==38.0.4
      - cssselect==1.3.0
      - curtsies==0.4.2
      - cwcwidth==0.1.10
      - decorator==5.2.1
      - exceptiongroup==1.2.2
      - execnet==2.1.1
      - executing==2.2.0
      - flask==2.2.5
      - greenlet==3.1.1
      - h11==0.14.0
      - h2==4.2.0
      - hpack==4.1.0
      - hyperframe==6.1.0
      - hyperlink==21.0.0
      - idna==3.10
      - importlib-metadata==8.6.1
      - incremental==24.7.2
      - iniconfig==2.1.0
      - ipython==8.18.1
      - itsdangerous==2.2.0
      - jedi==0.19.2
      - jinja2==3.1.6
      - jmespath==1.0.1
      - kaitaistruct==0.10
      - ldap3==2.9.1
      - lxml==5.3.1
      - markupsafe==3.0.2
      - matplotlib-inline==0.1.7
      - mitmproxy==9.0.1
      - mitmproxy-wireguard==0.1.23
      - msgpack==1.0.8
      - packaging==24.2
      - parsel==1.10.0
      - parso==0.8.4
      - passlib==1.7.4
      - pexpect==4.9.0
      - pluggy==1.5.0
      - prompt-toolkit==3.0.50
      - protego==0.4.0
      - protobuf==4.25.6
      - ptyprocess==0.7.0
      - publicsuffix2==2.20191221
      - pure-eval==0.2.3
      - pyasn1==0.6.1
      - pyasn1-modules==0.4.2
      - pycparser==2.22
      - pydispatcher==2.0.7
      - pygments==2.19.1
      - pyopenssl==22.1.0
      - pyparsing==3.0.9
      - pyperclip==1.8.2
      - pytest==8.3.5
      - pytest-asyncio==0.26.0
      - pytest-cov==6.0.0
      - pytest-mock==3.14.0
      - pytest-twisted==1.14.3
      - pytest-xdist==3.6.1
      - pyxdg==0.28
      - queuelib==1.7.0
      - requests==2.32.3
      - ruamel-yaml==0.17.40
      - ruamel-yaml-clib==0.2.12
      - service-identity==24.2.0
      - six==1.17.0
      - sortedcontainers==2.4.0
      - stack-data==0.6.3
      - sybil==9.1.0
      - testfixtures==8.3.0
      - tomli==2.2.1
      - tornado==6.4.2
      - traitlets==5.14.3
      - twisted==24.11.0
      - typing-extensions==4.13.0
      - urllib3==2.3.0
      - urwid==2.1.2
      - w3lib==2.3.1
      - wcwidth==0.2.13
      - werkzeug==3.1.3
      - wsproto==1.2.0
      - zipp==3.21.0
      - zope-interface==7.2
      - zstandard==0.19.0
prefix: /opt/conda/envs/scrapy

tests/test_spider.py::CrawlSpiderTest::test_start_url
tests/test_spider.py::CrawlSpiderTest::test_process_links tests/test_spider.py::CrawlSpiderTest::test_process_links_filter tests/test_spider.py::CrawlSpiderTest::test_process_links_generator tests/test_spider.py::CrawlSpiderTest::test_process_request tests/test_spider.py::CrawlSpiderTest::test_process_request_instance_method tests/test_spider.py::CrawlSpiderTest::test_process_request_instance_method_with_response tests/test_spider.py::CrawlSpiderTest::test_process_request_with_response tests/test_spider.py::CrawlSpiderTest::test_rule_without_link_extractor
tests/test_spider.py::SpiderTest::test_base_spider tests/test_spider.py::SpiderTest::test_closed_signal_call tests/test_spider.py::SpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::SpiderTest::test_from_crawler_init_call tests/test_spider.py::SpiderTest::test_log tests/test_spider.py::SpiderTest::test_logger tests/test_spider.py::SpiderTest::test_spider_args tests/test_spider.py::SpiderTest::test_spider_without_name tests/test_spider.py::SpiderTest::test_start_requests tests/test_spider.py::SpiderTest::test_update_settings tests/test_spider.py::InitSpiderTest::test_base_spider tests/test_spider.py::InitSpiderTest::test_closed_signal_call tests/test_spider.py::InitSpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::InitSpiderTest::test_from_crawler_init_call tests/test_spider.py::InitSpiderTest::test_log tests/test_spider.py::InitSpiderTest::test_logger tests/test_spider.py::InitSpiderTest::test_spider_args tests/test_spider.py::InitSpiderTest::test_spider_without_name tests/test_spider.py::InitSpiderTest::test_start_requests tests/test_spider.py::InitSpiderTest::test_update_settings tests/test_spider.py::XMLFeedSpiderTest::test_base_spider tests/test_spider.py::XMLFeedSpiderTest::test_closed_signal_call tests/test_spider.py::XMLFeedSpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::XMLFeedSpiderTest::test_from_crawler_init_call tests/test_spider.py::XMLFeedSpiderTest::test_log tests/test_spider.py::XMLFeedSpiderTest::test_logger tests/test_spider.py::XMLFeedSpiderTest::test_register_namespace tests/test_spider.py::XMLFeedSpiderTest::test_spider_args tests/test_spider.py::XMLFeedSpiderTest::test_spider_without_name tests/test_spider.py::XMLFeedSpiderTest::test_start_requests tests/test_spider.py::XMLFeedSpiderTest::test_update_settings tests/test_spider.py::CSVFeedSpiderTest::test_base_spider tests/test_spider.py::CSVFeedSpiderTest::test_closed_signal_call tests/test_spider.py::CSVFeedSpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::CSVFeedSpiderTest::test_from_crawler_init_call tests/test_spider.py::CSVFeedSpiderTest::test_log tests/test_spider.py::CSVFeedSpiderTest::test_logger tests/test_spider.py::CSVFeedSpiderTest::test_spider_args tests/test_spider.py::CSVFeedSpiderTest::test_spider_without_name tests/test_spider.py::CSVFeedSpiderTest::test_start_requests tests/test_spider.py::CSVFeedSpiderTest::test_update_settings tests/test_spider.py::CrawlSpiderTest::test_base_spider tests/test_spider.py::CrawlSpiderTest::test_closed_signal_call tests/test_spider.py::CrawlSpiderTest::test_follow_links_attribute_population tests/test_spider.py::CrawlSpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::CrawlSpiderTest::test_from_crawler_init_call tests/test_spider.py::CrawlSpiderTest::test_log tests/test_spider.py::CrawlSpiderTest::test_logger tests/test_spider.py::CrawlSpiderTest::test_spider_args tests/test_spider.py::CrawlSpiderTest::test_spider_without_name tests/test_spider.py::CrawlSpiderTest::test_start_requests tests/test_spider.py::CrawlSpiderTest::test_update_settings tests/test_spider.py::SitemapSpiderTest::test_alternate_url_locs tests/test_spider.py::SitemapSpiderTest::test_base_spider tests/test_spider.py::SitemapSpiderTest::test_closed_signal_call tests/test_spider.py::SitemapSpiderTest::test_from_crawler_crawler_and_settings_population tests/test_spider.py::SitemapSpiderTest::test_from_crawler_init_call tests/test_spider.py::SitemapSpiderTest::test_get_sitemap_body tests/test_spider.py::SitemapSpiderTest::test_get_sitemap_body_gzip_headers tests/test_spider.py::SitemapSpiderTest::test_get_sitemap_body_xml_url tests/test_spider.py::SitemapSpiderTest::test_get_sitemap_body_xml_url_compressed tests/test_spider.py::SitemapSpiderTest::test_get_sitemap_urls_from_robotstxt tests/test_spider.py::SitemapSpiderTest::test_log tests/test_spider.py::SitemapSpiderTest::test_logger tests/test_spider.py::SitemapSpiderTest::test_sitemap_filter tests/test_spider.py::SitemapSpiderTest::test_sitemap_filter_with_alternate_links tests/test_spider.py::SitemapSpiderTest::test_sitemapindex_filter tests/test_spider.py::SitemapSpiderTest::test_spider_args tests/test_spider.py::SitemapSpiderTest::test_spider_without_name tests/test_spider.py::SitemapSpiderTest::test_start_requests tests/test_spider.py::SitemapSpiderTest::test_update_settings tests/test_spider.py::DeprecationTest::test_crawl_spider tests/test_spider.py::DeprecationTest::test_make_requests_from_url_deprecated tests/test_spider.py::NoParseMethodSpiderTest::test_undefined_parse_method
scrapy/spiders/__init__.py
scrapy/spiders/__init__.py