<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues/5660" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/5660/hovercard">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="473592737" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5661" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/5661/hovercard" href="https://github.com/celery/celery/pull/5661">#5661</a></li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="473424725" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5660" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/5660/hovercard" href="https://github.com/celery/celery/issues/5660">#5660</a></li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="473424725" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5660" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/5660/hovercard" href="https://github.com/celery/celery/issues/5660">#5660</a></li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>:<br>
4.4.7</p>
<details>
<summary><b><code class="notranslate">celery report</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).
  warnings.warn(message, category=ElasticsearchDeprecationWarning)
[2020-08-18 09:14:06,421: ERROR/MainProcess] Pool callback raised exception: RequestError(400, 'illegal_argument_exception', {'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'mapper [result.args] of different type, current_type [long], merged_type [text]'}], 'type': 'illegal_argument_exception', 'reason': 'mapper [result.args] of different type, current_type [long], merged_type [text]'}, 'status': 400})
Traceback (most recent call last):
  File &quot;/usr/local/lib/python3.7/site-packages/billiard/pool.py&quot;, line 1796, in safe_apply_callback
    fun(*args, **kwargs)
  File &quot;/usr/local/lib/python3.7/site-packages/celery/worker/request.py&quot;, line 528, in on_failure
    store_result=self.store_errors,
  File &quot;/usr/local/lib/python3.7/site-packages/celery/backends/base.py&quot;, line 160, in mark_as_failure
    traceback=traceback, request=request)
  File &quot;/usr/local/lib/python3.7/site-packages/celery/backends/base.py&quot;, line 407, in store_result
    request=request, **kwargs)
  File &quot;/usr/local/lib/python3.7/site-packages/celery/backends/base.py&quot;, line 758, in _store_result
    self.set(self.get_key_for_task(task_id), self.encode(meta))
  File &quot;/usr/local/lib/python3.7/site-packages/celery/backends/elasticsearch.py&quot;, line 109, in set
    datetime.utcnow().isoformat()[:-3]
  File &quot;/usr/local/lib/python3.7/site-packages/celery/backends/elasticsearch.py&quot;, line 126, in _index
    **kwargs
  File &quot;/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py&quot;, line 139, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File &quot;/usr/local/lib/python3.7/site-packages/elasticsearch/client/__init__.py&quot;, line 380, in index
    body=body,
  File &quot;/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py&quot;, line 352, in perform_request
    timeout=timeout,
  File &quot;/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py&quot;, line 256, in perform_request
    self._raise_error(response.status, raw_data)
  File &quot;/usr/local/lib/python3.7/site-packages/elasticsearch/connection/base.py&quot;, line 288, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'mapper [result.args] of different type, current_type [long], merged_type [text]')"><pre class="notranslate"><code class="notranslate">types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).
  warnings.warn(message, category=ElasticsearchDeprecationWarning)
[2020-08-18 09:14:06,421: ERROR/MainProcess] Pool callback raised exception: RequestError(400, 'illegal_argument_exception', {'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'mapper [result.args] of different type, current_type [long], merged_type [text]'}], 'type': 'illegal_argument_exception', 'reason': 'mapper [result.args] of different type, current_type [long], merged_type [text]'}, 'status': 400})
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/billiard/pool.py", line 1796, in safe_apply_callback
    fun(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/celery/worker/request.py", line 528, in on_failure
    store_result=self.store_errors,
  File "/usr/local/lib/python3.7/site-packages/celery/backends/base.py", line 160, in mark_as_failure
    traceback=traceback, request=request)
  File "/usr/local/lib/python3.7/site-packages/celery/backends/base.py", line 407, in store_result
    request=request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/celery/backends/base.py", line 758, in _store_result
    self.set(self.get_key_for_task(task_id), self.encode(meta))
  File "/usr/local/lib/python3.7/site-packages/celery/backends/elasticsearch.py", line 109, in set
    datetime.utcnow().isoformat()[:-3]
  File "/usr/local/lib/python3.7/site-packages/celery/backends/elasticsearch.py", line 126, in _index
    **kwargs
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 139, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 380, in index
    body=body,
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 352, in perform_request
    timeout=timeout,
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 256, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 288, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'mapper [result.args] of different type, current_type [long], merged_type [text]')
</code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<h2 dir="auto">Required Dependencies</h2>

<ul dir="auto">
<li><strong>Minimal Python Version</strong>: 3.7</li>
<li><strong>Minimal Celery Version</strong>: 4.4.0</li>
<li><strong>Minimal OS and/or Kernel Version</strong>: ubuntu</li>
</ul>
<h3 dir="auto">Python Packages</h3>

<details>
<summary><b><code class="notranslate">pip freeze</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="uvicorn==0.11.3
fastapi&gt;=0.61.0
fastapi_permissions==0.2.6
python_multipart==0.0.5
email_validator==1.0.5
requests==2.23.0
celery==4.4.7"><pre class="notranslate"><code class="notranslate">uvicorn==0.11.3
fastapi&gt;=0.61.0
fastapi_permissions==0.2.6
python_multipart==0.0.5
email_validator==1.0.5
requests==2.23.0
celery==4.4.7
</code></pre></div>
<p dir="auto"></p>
</details>
<h3 dir="auto">Other Dependencies</h3>

<details>
<p dir="auto">
N/A
</p>
</details>
<h2 dir="auto">Minimally Reproducible Test Case</h2>

<details>
<p dir="auto">
</p><div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Expected Behavior</h1>
<p dir="auto">Expected to Save the result backend as Object to elastic search, So I will be able to use it in the dashboard.</p>
<h1 dir="auto">Actual Behavior</h1>
<p dir="auto">The result object is saved as text, and after trying all the solutions found after the search, I still have errors and not able to the result backend to elastic search.</p>
<h1 dir="auto">Checklist</h1>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Enhancement%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical enhancement to an existing feature.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22Issue+Type%3A+Enhancement%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed enhancements.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the if the same enhancement was already implemented in the<br>
master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues in this issue<br>
(If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>
<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="893932835" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/6773" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/6773/hovercard" href="https://github.com/celery/celery/pull/6773">#6773</a></li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h1 dir="auto">Brief Summary</h1>
<p dir="auto">Create a new major version (1.0.0) of the pytest-celery plugin.</p>
<h2 dir="auto">Motivation</h2>
<p dir="auto">We want to refactor the Celery integration test suite so that it will be up and running again. We will make the integration tests more readable and flexible and ergonomic to us.</p>
<p dir="auto">We also want to enable Celery users to test the integration easily and reliably, which is why the changes will eventually be available to all users via the pytest-celery plugin.</p>
<h2 dir="auto">Scope</h2>
<p dir="auto">The scope of this project is to migrate the Celery integration suite since it is broken.</p>
<p dir="auto">In this major version we will only address our use-cases. In the next one, we will begin planning and implementing features which could be important to our userbase as well.</p>
<h2 dir="auto">High Level Requirements</h2>
<h4 dir="auto">Memory and CPU efficiency</h4>
<p dir="auto">The plugin must support GH Actions as this is the CI service we use.</p>
<p dir="auto">It should support other commonly used CI services (such as Azure Pipelines, Circle CI or other similar CI services for open source).</p>
<p dir="auto">We will run dependencies in Docker using the testcontainers library.<br>
The containers our plugins runs will be reused from test to test for efficiency instead of tearing them down and re-running them in each test.<br>
When we require multiple instances of the same docker image (e.g. using the same container for the message broker and result backend), a separate virtual host will be created for each service our test requires.<br>
This provides logical separation which is good enough for testing purposes as usually security is not an issue in CI and therefore, we don't need to separate services by using stronger guarantees of isolation such as separate containers or even virtual machines.</p>
<h4 dir="auto">Resource leak prevention</h4>
<p dir="auto">The only resource that is opened for each test by default is a queue. Queues will be given an expiration date so that unexpected shutdowns and failures do not cause memory leaks.<br>
Containers are removed when the test suite is torn down. As a consequence, this will make all virtual hosts disappear.<br>
Note that if a test is forced to quit (if you hit ctrl+c twice or if the pytest process is killed for whatever reason) , its virtual hosts will not immediately be removed. However, each time the test suite is completed, it tears down all virtual hosts, so this is manageable.</p>
<h4 dir="auto">Run on the entire matrix of optional dependencies in parallel.</h4>
<p dir="auto">A test must be able to receive multiple options for each dependency.<br>
The test will run on the Cartesian product of all optional dependencies.</p>
<p dir="auto">This should also work in parallel with pytest-xdist.</p>
<h4 dir="auto">Meaningful error messages</h4>
<p dir="auto">Error messages must be meaningful and grouped.<br>
When the same test runs in parallel on several configurations, and some configurations fail for the same reason, we will batch those failures together.</p>
<p dir="auto">All relevant logs will be attached to pytest's report when something goes wrong, including the container logs and the Celery worker(s) logs.</p>
<h2 dir="auto">Timeline</h2>
<p dir="auto">ASAP: release version 0.1.0 to pytest-celery plugin.<br>
Celery 5.x (next minor release): deprecation pending for version 0.1.0<br>
Celery 6.0: pytest-celery version 1.0.0 released, version 0.1.0 deprecated<br>
Celery 6.1: pytest-celery version 0.2.0 released which includes the code in <code class="notranslate">celery/contrib/pytest.py</code><br>
Celery 7.x: delete code for version 0.1.0</p>
<h1 dir="auto">Design</h1>
<h2 dir="auto">Architectural Considerations</h2>

<p dir="auto">None</p>
<h2 dir="auto">Proposed Behavior</h2>

<h2 dir="auto">Proposed UI/UX</h2>

<p dir="auto">The API design will be added soon.</p>
<h2 dir="auto">Diagrams</h2>
<p dir="auto">N/A</p>
<h2 dir="auto">Alternatives</h2>
<p dir="auto">None</p>