We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Let's CDN cache most site-search queries
I downloaded the 1,000 most frequent URLs when people do a site-search in `en-US`. 
In en-US alone we have 7,586 searches per day. And that's based only on the top 1,000 search terms. Remember, this only counts people who have GA tracking not blocked. 

And I estimate that roughly 59% of all our `/$locale/docs/` traffic is `en-US`. That means that **we get 13.5k site-searches per day**. And remember, this is just for the top 1,000 search terms. And since site-searches today is never cached by the CDN, that's how many times Kuma has to process 13.5k Elasticsearch queries per day (and equally all the Django code that wraps Elasticsearch into a request/(JSON)response). Perhaps roughly 20k site-searches per when you include all search terms and the people who don't let GA track.
Also, each request that goes into Django also triggers the rate limiter which means it has to use Redis too. 

As of today, site-search triggers a full Django+React SSR render and this accounts for 18% of the most time-consuming work our EC2 has to do:
<img width="443" alt="Screen Shot 2021-02-25 at 4 41 05 PM" src="https://user-images.githubusercontent.com/26739/109223288-647b6c80-7788-11eb-911c-d44582735c5e.png">
That number will shrink by itself once Yari takes over the search and it's just a `/api/v1/search` XHR request.

I speculate that we can leverage the CDN to:

- **Make users get faster search results**
- **Reduce the stress on our Kubernetes cluster and be able to use a lot less EC2 power and save money and electricity**

Many searches are a single word (i.e. 0 spaces)
```
0 spaces: 91.3%
1 spaces: 8.5%
2 spaces: 0.2%
```

Most searches are short single words. A complete list of the [top 1,000 most common searches are here](https://gist.github.com/peterbe/7cc2c8fa1cc72e976b9ac0caa0852055)

If you analyze the length of the search terms you get:
```
0-10       85.1%
11-20      14.6%
21+        0.3%
```

Since all traffic always goes through the CDN anyway, we can set a `Cache-Control` on these. Say, 24h. I don't think it's expected that a search for `splice` should get documents that have recently changed within the last day. 

We could potentially make the setting of the `Cache-Control` only on short and simple searches without any advanced filtering (e.g `?locale=ja&locale=en-us&slug_prefix=Web/CSS`). That way our CDN won't have to store too many things that aren't worth caching.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `kuma/api/v1/search/__init__.py`
Content:
```
1 from django import http
2 from django.conf import settings
3 from elasticsearch import exceptions
4 from elasticsearch_dsl import Q, query, Search
5 from redo import retrying
6 
7 from kuma.api.v1.decorators import allow_CORS_GET
8 
9 from .forms import SearchForm
10 
11 
12 class JsonResponse(http.JsonResponse):
13     """The only reason this exists is so that other Django views can call
14     views that return instances of this and then get to the data before it
15     gets JSON serialized.
16     This is something that rest_framework's JsonResponse supports.
17     Ultimately, the only view that cares is the (old) Kuma search view page
18     that calls the view function here in this file. Now it can do something like:
19 
20         response = kuma.api.v1.search.search(request)
21         found = response.data
22 
23     """
24 
25     def __init__(self, data, *args, **kwargs):
26         self.data = data
27         super().__init__(data, *args, **kwargs)
28 
29 
30 def legacy(request, locale=None):
31     raise NotImplementedError("work harder")
32 
33 
34 @allow_CORS_GET
35 def search(request, locale=None):
36     initial = {"size": 10, "page": 1, "archive": SearchForm.ARCHIVE_CHOICES[0]}
37     if locale:
38         initial["locale"] = locale
39     form = SearchForm(request.GET, initial=initial)
40     if not form.is_valid():
41         return JsonResponse({"errors": form.errors.get_json_data()}, status=400)
42 
43     locales = form.cleaned_data["locale"] or [settings.LANGUAGE_CODE]
44     assert isinstance(locales, list)
45 
46     params = {
47         "locales": [x.lower() for x in locales],
48         "archive": form.cleaned_data["archive"],
49         "query": form.cleaned_data["q"],
50         "size": form.cleaned_data["size"],
51         "page": form.cleaned_data["page"],
52         "sort": form.cleaned_data["sort"],
53         # The `slug` is always stored, as a Keyword index, in lowercase.
54         "slug_prefixes": [x.lower() for x in form.cleaned_data["slug_prefix"]],
55     }
56 
57     # By default, assume that we will try to make suggestions.
58     make_suggestions = True
59     if len(params["query"]) > 100 or max(len(x) for x in params["query"].split()) > 30:
60         # For example, if it's a really long query, or a specific word is just too
61         # long, you can get those tricky
62         # TransportError(500, 'search_phase_execution_exception', 'Term too complex:
63         # errors which are hard to prevent against.
64         make_suggestions = False
65 
66     results = _find(
67         params,
68         make_suggestions=make_suggestions,
69     )
70     return JsonResponse(results)
71 
72 
73 def _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):
74     search_query = Search(
75         index=settings.SEARCH_INDEX_NAME,
76     )
77     if make_suggestions:
78         # XXX research if it it's better to use phrase suggesters and if
79         # that works
80         # https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html#phrase-suggester
81         search_query = search_query.suggest(
82             "title_suggestions", params["query"], term={"field": "title"}
83         )
84         search_query = search_query.suggest(
85             "body_suggestions", params["query"], term={"field": "body"}
86         )
87 
88     sub_queries = []
89     sub_queries.append(Q("match", title={"query": params["query"], "boost": 2.0}))
90     sub_queries.append(Q("match", body={"query": params["query"], "boost": 1.0}))
91     if " " in params["query"]:
92         sub_queries.append(
93             Q("match_phrase", title={"query": params["query"], "boost": 10.0})
94         )
95         sub_queries.append(
96             Q("match_phrase", body={"query": params["query"], "boost": 5.0})
97         )
98 
99     sub_query = query.Bool(should=sub_queries)
100 
101     if params["locales"]:
102         search_query = search_query.filter("terms", locale=params["locales"])
103     if params["archive"] == "exclude":
104         search_query = search_query.filter("term", archived=False)
105     elif params["archive"] == "only":
106         search_query = search_query.filter("term", archived=True)
107 
108     if params["slug_prefixes"]:
109         sub_queries = [Q("prefix", slug=x) for x in params["slug_prefixes"]]
110         search_query = search_query.query(query.Bool(should=sub_queries))
111 
112     search_query = search_query.highlight_options(
113         pre_tags=["<mark>"],
114         post_tags=["</mark>"],
115         number_of_fragments=3,
116         fragment_size=120,
117         encoder="html",
118     )
119     search_query = search_query.highlight("title", "body")
120 
121     if params["sort"] == "relevance":
122         search_query = search_query.sort("_score", "-popularity")
123         search_query = search_query.query(sub_query)
124     elif params["sort"] == "popularity":
125         search_query = search_query.sort("-popularity", "_score")
126         search_query = search_query.query(sub_query)
127     else:
128         popularity_factor = 10.0
129         boost_mode = "sum"
130         score_mode = "max"
131         search_query = search_query.query(
132             "function_score",
133             query=sub_query,
134             functions=[
135                 query.SF(
136                     "field_value_factor",
137                     field="popularity",
138                     factor=popularity_factor,
139                     missing=0.0,
140                 )
141             ],
142             boost_mode=boost_mode,
143             score_mode=score_mode,
144         )
145 
146     search_query = search_query.source(excludes=["body"])
147 
148     search_query = search_query[
149         params["size"] * (params["page"] - 1) : params["size"] * params["page"]
150     ]
151 
152     retry_options = {
153         "retry_exceptions": (
154             # This is the standard operational exception.
155             exceptions.ConnectionError,
156             # This can happen if the search happened right as the index had
157             # just been deleted due to a fresh re-indexing happening in Yari.
158             exceptions.NotFoundError,
159             # This can happen when the index simply isn't ready yet.
160             exceptions.TransportError,
161         ),
162         # The default in redo is 60 seconds. Let's tone that down.
163         "sleeptime": settings.ES_RETRY_SLEEPTIME,
164         "attempts": settings.ES_RETRY_ATTEMPTS,
165         "jitter": settings.ES_RETRY_JITTER,
166     }
167     with retrying(search_query.execute, **retry_options) as retrying_function:
168         response = retrying_function()
169 
170     if total_only:
171         return response.hits.total
172 
173     metadata = {
174         "took_ms": response.took,
175         "total": {
176             # The `response.hits.total` is a `elasticsearch_dsl.utils.AttrDict`
177             # instance. Pluck only the exact data needed.
178             "value": response.hits.total.value,
179             "relation": response.hits.total.relation,
180         },
181         "size": params["size"],
182         "page": params["page"],
183     }
184     documents = []
185     for hit in response:
186         try:
187             body_highlight = list(hit.meta.highlight.body)
188         except AttributeError:
189             body_highlight = []
190         try:
191             title_highlight = list(hit.meta.highlight.title)
192         except AttributeError:
193             title_highlight = []
194 
195         d = {
196             "mdn_url": hit.meta.id,
197             "score": hit.meta.score,
198             "title": hit.title,
199             "locale": hit.locale,
200             "slug": hit.slug,
201             "popularity": hit.popularity,
202             "archived": hit.archived,
203             "summary": hit.summary,
204             "highlight": {
205                 "body": body_highlight,
206                 "title": title_highlight,
207             },
208         }
209         documents.append(d)
210 
211     try:
212         suggest = getattr(response, "suggest")
213     except AttributeError:
214         suggest = None
215 
216     suggestions = []
217     if suggest:
218         suggestion_strings = _unpack_suggestions(
219             params["query"],
220             response.suggest,
221             ("body_suggestions", "title_suggestions"),
222         )
223 
224         for score, string in suggestion_strings:
225             if score > min_suggestion_score or 1:
226                 # Sure, this is different way to spell, but what will it yield
227                 # if you actually search it?
228                 total = _find(dict(params, query=string), total_only=True)
229                 if total["value"] > 0:
230                     suggestions.append(
231                         {
232                             "text": string,
233                             "total": {
234                                 # This 'total' is an `AttrDict` instance.
235                                 "value": total.value,
236                                 "relation": total.relation,
237                             },
238                         }
239                     )
240                     # Since they're sorted by score, it's usually never useful
241                     # to suggestion more than exactly 1 good suggestion.
242                     break
243 
244     return {
245         "documents": documents,
246         "metadata": metadata,
247         "suggestions": suggestions,
248     }
249 
250 
251 def _unpack_suggestions(query, suggest, keys):
252     alternatives = []
253     for key in keys:
254         for suggestion in getattr(suggest, key, []):
255             for option in suggestion.options:
256                 alternatives.append(
257                     (
258                         option.score,
259                         query[0 : suggestion.offset]
260                         + option.text
261                         + query[suggestion.offset + suggestion.length :],
262                     )
263                 )
264     alternatives.sort(reverse=True)  # highest score first
265     return alternatives
266
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py
--- a/kuma/api/v1/search/__init__.py
+++ b/kuma/api/v1/search/__init__.py
@@ -1,5 +1,6 @@
 from django import http
 from django.conf import settings
+from django.utils.cache import patch_cache_control
 from elasticsearch import exceptions
 from elasticsearch_dsl import Q, query, Search
 from redo import retrying
@@ -8,6 +9,12 @@
 
 from .forms import SearchForm
 
+# This is the number of seconds to be put into the Cache-Control max-age header
+# if the search is successful.
+# We can increase the number as we feel more and more comfortable with how
+# the `/api/v1/search` works.
+SEARCH_CACHE_CONTROL_MAX_AGE = 60 * 60 * 12
+
 
 class JsonResponse(http.JsonResponse):
     """The only reason this exists is so that other Django views can call
@@ -67,7 +74,25 @@
         params,
         make_suggestions=make_suggestions,
     )
-    return JsonResponse(results)
+    response = JsonResponse(results)
+
+    # The reason for caching is that most of the time, the searches people make
+    # are short and often stand a high chance of being reused by other users
+    # in the CDN.
+    # The worst that can happen is that we fill up the CDN with cached responses
+    # that end up being stored there and never reused by another user.
+    # We could consider only bothering with this based on looking at the parameters.
+    # For example, if someone made a search with "complex parameters" we could skip
+    # cache-control because it'll just be a waste to store it (CDN and client).
+    # The reason for not using a "shared" cache-control, i.e. `s-max-age` is
+    # because the cache-control seconds we intend to set are appropriate for both
+    # the client and the CDN. If the value set is 3600 seconds, that means that
+    # clients might potentially re-use their own browser cache if they trigger
+    # a repeated search. And it's an appropriate number for the CDN too.
+    # For more info about how our search patterns behave,
+    # see https://github.com/mdn/kuma/issues/7799
+    patch_cache_control(response, public=True, max_age=SEARCH_CACHE_CONTROL_MAX_AGE)
+    return response
 
 
 def _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):

{"golden_diff": "diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py\n--- a/kuma/api/v1/search/__init__.py\n+++ b/kuma/api/v1/search/__init__.py\n@@ -1,5 +1,6 @@\n from django import http\n from django.conf import settings\n+from django.utils.cache import patch_cache_control\n from elasticsearch import exceptions\n from elasticsearch_dsl import Q, query, Search\n from redo import retrying\n@@ -8,6 +9,12 @@\n \n from .forms import SearchForm\n \n+# This is the number of seconds to be put into the Cache-Control max-age header\n+# if the search is successful.\n+# We can increase the number as we feel more and more comfortable with how\n+# the `/api/v1/search` works.\n+SEARCH_CACHE_CONTROL_MAX_AGE = 60 * 60 * 12\n+\n \n class JsonResponse(http.JsonResponse):\n     \"\"\"The only reason this exists is so that other Django views can call\n@@ -67,7 +74,25 @@\n         params,\n         make_suggestions=make_suggestions,\n     )\n-    return JsonResponse(results)\n+    response = JsonResponse(results)\n+\n+    # The reason for caching is that most of the time, the searches people make\n+    # are short and often stand a high chance of being reused by other users\n+    # in the CDN.\n+    # The worst that can happen is that we fill up the CDN with cached responses\n+    # that end up being stored there and never reused by another user.\n+    # We could consider only bothering with this based on looking at the parameters.\n+    # For example, if someone made a search with \"complex parameters\" we could skip\n+    # cache-control because it'll just be a waste to store it (CDN and client).\n+    # The reason for not using a \"shared\" cache-control, i.e. `s-max-age` is\n+    # because the cache-control seconds we intend to set are appropriate for both\n+    # the client and the CDN. If the value set is 3600 seconds, that means that\n+    # clients might potentially re-use their own browser cache if they trigger\n+    # a repeated search. And it's an appropriate number for the CDN too.\n+    # For more info about how our search patterns behave,\n+    # see https://github.com/mdn/kuma/issues/7799\n+    patch_cache_control(response, public=True, max_age=SEARCH_CACHE_CONTROL_MAX_AGE)\n+    return response\n \n \n def _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):\n", "issue": "Let's CDN cache most site-search queries\nI downloaded the 1,000 most frequent URLs when people do a site-search in `en-US`. \r\nIn en-US alone we have 7,586 searches per day. And that's based only on the top 1,000 search terms. Remember, this only counts people who have GA tracking not blocked. \r\n\r\nAnd I estimate that roughly 59% of all our `/$locale/docs/` traffic is `en-US`. That means that **we get 13.5k site-searches per day**. And remember, this is just for the top 1,000 search terms. And since site-searches today is never cached by the CDN, that's how many times Kuma has to process 13.5k Elasticsearch queries per day (and equally all the Django code that wraps Elasticsearch into a request/(JSON)response). Perhaps roughly 20k site-searches per when you include all search terms and the people who don't let GA track.\r\nAlso, each request that goes into Django also triggers the rate limiter which means it has to use Redis too. \r\n\r\nAs of today, site-search triggers a full Django+React SSR render and this accounts for 18% of the most time-consuming work our EC2 has to do:\r\n<img width=\"443\" alt=\"Screen Shot 2021-02-25 at 4 41 05 PM\" src=\"https://user-images.githubusercontent.com/26739/109223288-647b6c80-7788-11eb-911c-d44582735c5e.png\">\r\nThat number will shrink by itself once Yari takes over the search and it's just a `/api/v1/search` XHR request.\r\n\r\nI speculate that we can leverage the CDN to:\r\n\r\n- **Make users get faster search results**\r\n- **Reduce the stress on our Kubernetes cluster and be able to use a lot less EC2 power and save money and electricity**\r\n\r\nMany searches are a single word (i.e. 0 spaces)\r\n```\r\n0 spaces: 91.3%\r\n1 spaces: 8.5%\r\n2 spaces: 0.2%\r\n```\r\n\r\nMost searches are short single words. A complete list of the [top 1,000 most common searches are here](https://gist.github.com/peterbe/7cc2c8fa1cc72e976b9ac0caa0852055)\r\n\r\nIf you analyze the length of the search terms you get:\r\n```\r\n0-10       85.1%\r\n11-20      14.6%\r\n21+        0.3%\r\n```\r\n\r\nSince all traffic always goes through the CDN anyway, we can set a `Cache-Control` on these. Say, 24h. I don't think it's expected that a search for `splice` should get documents that have recently changed within the last day. \r\n\r\nWe could potentially make the setting of the `Cache-Control` only on short and simple searches without any advanced filtering (e.g `?locale=ja&locale=en-us&slug_prefix=Web/CSS`). That way our CDN won't have to store too many things that aren't worth caching.\n", "before_files": [{"content": "from django import http\nfrom django.conf import settings\nfrom elasticsearch import exceptions\nfrom elasticsearch_dsl import Q, query, Search\nfrom redo import retrying\n\nfrom kuma.api.v1.decorators import allow_CORS_GET\n\nfrom .forms import SearchForm\n\n\nclass JsonResponse(http.JsonResponse):\n    \"\"\"The only reason this exists is so that other Django views can call\n    views that return instances of this and then get to the data before it\n    gets JSON serialized.\n    This is something that rest_framework's JsonResponse supports.\n    Ultimately, the only view that cares is the (old) Kuma search view page\n    that calls the view function here in this file. Now it can do something like:\n\n        response = kuma.api.v1.search.search(request)\n        found = response.data\n\n    \"\"\"\n\n    def __init__(self, data, *args, **kwargs):\n        self.data = data\n        super().__init__(data, *args, **kwargs)\n\n\ndef legacy(request, locale=None):\n    raise NotImplementedError(\"work harder\")\n\n\n@allow_CORS_GET\ndef search(request, locale=None):\n    initial = {\"size\": 10, \"page\": 1, \"archive\": SearchForm.ARCHIVE_CHOICES[0]}\n    if locale:\n        initial[\"locale\"] = locale\n    form = SearchForm(request.GET, initial=initial)\n    if not form.is_valid():\n        return JsonResponse({\"errors\": form.errors.get_json_data()}, status=400)\n\n    locales = form.cleaned_data[\"locale\"] or [settings.LANGUAGE_CODE]\n    assert isinstance(locales, list)\n\n    params = {\n        \"locales\": [x.lower() for x in locales],\n        \"archive\": form.cleaned_data[\"archive\"],\n        \"query\": form.cleaned_data[\"q\"],\n        \"size\": form.cleaned_data[\"size\"],\n        \"page\": form.cleaned_data[\"page\"],\n        \"sort\": form.cleaned_data[\"sort\"],\n        # The `slug` is always stored, as a Keyword index, in lowercase.\n        \"slug_prefixes\": [x.lower() for x in form.cleaned_data[\"slug_prefix\"]],\n    }\n\n    # By default, assume that we will try to make suggestions.\n    make_suggestions = True\n    if len(params[\"query\"]) > 100 or max(len(x) for x in params[\"query\"].split()) > 30:\n        # For example, if it's a really long query, or a specific word is just too\n        # long, you can get those tricky\n        # TransportError(500, 'search_phase_execution_exception', 'Term too complex:\n        # errors which are hard to prevent against.\n        make_suggestions = False\n\n    results = _find(\n        params,\n        make_suggestions=make_suggestions,\n    )\n    return JsonResponse(results)\n\n\ndef _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):\n    search_query = Search(\n        index=settings.SEARCH_INDEX_NAME,\n    )\n    if make_suggestions:\n        # XXX research if it it's better to use phrase suggesters and if\n        # that works\n        # https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html#phrase-suggester\n        search_query = search_query.suggest(\n            \"title_suggestions\", params[\"query\"], term={\"field\": \"title\"}\n        )\n        search_query = search_query.suggest(\n            \"body_suggestions\", params[\"query\"], term={\"field\": \"body\"}\n        )\n\n    sub_queries = []\n    sub_queries.append(Q(\"match\", title={\"query\": params[\"query\"], \"boost\": 2.0}))\n    sub_queries.append(Q(\"match\", body={\"query\": params[\"query\"], \"boost\": 1.0}))\n    if \" \" in params[\"query\"]:\n        sub_queries.append(\n            Q(\"match_phrase\", title={\"query\": params[\"query\"], \"boost\": 10.0})\n        )\n        sub_queries.append(\n            Q(\"match_phrase\", body={\"query\": params[\"query\"], \"boost\": 5.0})\n        )\n\n    sub_query = query.Bool(should=sub_queries)\n\n    if params[\"locales\"]:\n        search_query = search_query.filter(\"terms\", locale=params[\"locales\"])\n    if params[\"archive\"] == \"exclude\":\n        search_query = search_query.filter(\"term\", archived=False)\n    elif params[\"archive\"] == \"only\":\n        search_query = search_query.filter(\"term\", archived=True)\n\n    if params[\"slug_prefixes\"]:\n        sub_queries = [Q(\"prefix\", slug=x) for x in params[\"slug_prefixes\"]]\n        search_query = search_query.query(query.Bool(should=sub_queries))\n\n    search_query = search_query.highlight_options(\n        pre_tags=[\"<mark>\"],\n        post_tags=[\"</mark>\"],\n        number_of_fragments=3,\n        fragment_size=120,\n        encoder=\"html\",\n    )\n    search_query = search_query.highlight(\"title\", \"body\")\n\n    if params[\"sort\"] == \"relevance\":\n        search_query = search_query.sort(\"_score\", \"-popularity\")\n        search_query = search_query.query(sub_query)\n    elif params[\"sort\"] == \"popularity\":\n        search_query = search_query.sort(\"-popularity\", \"_score\")\n        search_query = search_query.query(sub_query)\n    else:\n        popularity_factor = 10.0\n        boost_mode = \"sum\"\n        score_mode = \"max\"\n        search_query = search_query.query(\n            \"function_score\",\n            query=sub_query,\n            functions=[\n                query.SF(\n                    \"field_value_factor\",\n                    field=\"popularity\",\n                    factor=popularity_factor,\n                    missing=0.0,\n                )\n            ],\n            boost_mode=boost_mode,\n            score_mode=score_mode,\n        )\n\n    search_query = search_query.source(excludes=[\"body\"])\n\n    search_query = search_query[\n        params[\"size\"] * (params[\"page\"] - 1) : params[\"size\"] * params[\"page\"]\n    ]\n\n    retry_options = {\n        \"retry_exceptions\": (\n            # This is the standard operational exception.\n            exceptions.ConnectionError,\n            # This can happen if the search happened right as the index had\n            # just been deleted due to a fresh re-indexing happening in Yari.\n            exceptions.NotFoundError,\n            # This can happen when the index simply isn't ready yet.\n            exceptions.TransportError,\n        ),\n        # The default in redo is 60 seconds. Let's tone that down.\n        \"sleeptime\": settings.ES_RETRY_SLEEPTIME,\n        \"attempts\": settings.ES_RETRY_ATTEMPTS,\n        \"jitter\": settings.ES_RETRY_JITTER,\n    }\n    with retrying(search_query.execute, **retry_options) as retrying_function:\n        response = retrying_function()\n\n    if total_only:\n        return response.hits.total\n\n    metadata = {\n        \"took_ms\": response.took,\n        \"total\": {\n            # The `response.hits.total` is a `elasticsearch_dsl.utils.AttrDict`\n            # instance. Pluck only the exact data needed.\n            \"value\": response.hits.total.value,\n            \"relation\": response.hits.total.relation,\n        },\n        \"size\": params[\"size\"],\n        \"page\": params[\"page\"],\n    }\n    documents = []\n    for hit in response:\n        try:\n            body_highlight = list(hit.meta.highlight.body)\n        except AttributeError:\n            body_highlight = []\n        try:\n            title_highlight = list(hit.meta.highlight.title)\n        except AttributeError:\n            title_highlight = []\n\n        d = {\n            \"mdn_url\": hit.meta.id,\n            \"score\": hit.meta.score,\n            \"title\": hit.title,\n            \"locale\": hit.locale,\n            \"slug\": hit.slug,\n            \"popularity\": hit.popularity,\n            \"archived\": hit.archived,\n            \"summary\": hit.summary,\n            \"highlight\": {\n                \"body\": body_highlight,\n                \"title\": title_highlight,\n            },\n        }\n        documents.append(d)\n\n    try:\n        suggest = getattr(response, \"suggest\")\n    except AttributeError:\n        suggest = None\n\n    suggestions = []\n    if suggest:\n        suggestion_strings = _unpack_suggestions(\n            params[\"query\"],\n            response.suggest,\n            (\"body_suggestions\", \"title_suggestions\"),\n        )\n\n        for score, string in suggestion_strings:\n            if score > min_suggestion_score or 1:\n                # Sure, this is different way to spell, but what will it yield\n                # if you actually search it?\n                total = _find(dict(params, query=string), total_only=True)\n                if total[\"value\"] > 0:\n                    suggestions.append(\n                        {\n                            \"text\": string,\n                            \"total\": {\n                                # This 'total' is an `AttrDict` instance.\n                                \"value\": total.value,\n                                \"relation\": total.relation,\n                            },\n                        }\n                    )\n                    # Since they're sorted by score, it's usually never useful\n                    # to suggestion more than exactly 1 good suggestion.\n                    break\n\n    return {\n        \"documents\": documents,\n        \"metadata\": metadata,\n        \"suggestions\": suggestions,\n    }\n\n\ndef _unpack_suggestions(query, suggest, keys):\n    alternatives = []\n    for key in keys:\n        for suggestion in getattr(suggest, key, []):\n            for option in suggestion.options:\n                alternatives.append(\n                    (\n                        option.score,\n                        query[0 : suggestion.offset]\n                        + option.text\n                        + query[suggestion.offset + suggestion.length :],\n                    )\n                )\n    alternatives.sort(reverse=True)  # highest score first\n    return alternatives\n", "path": "kuma/api/v1/search/__init__.py"}], "after_files": [{"content": "from django import http\nfrom django.conf import settings\nfrom django.utils.cache import patch_cache_control\nfrom elasticsearch import exceptions\nfrom elasticsearch_dsl import Q, query, Search\nfrom redo import retrying\n\nfrom kuma.api.v1.decorators import allow_CORS_GET\n\nfrom .forms import SearchForm\n\n# This is the number of seconds to be put into the Cache-Control max-age header\n# if the search is successful.\n# We can increase the number as we feel more and more comfortable with how\n# the `/api/v1/search` works.\nSEARCH_CACHE_CONTROL_MAX_AGE = 60 * 60 * 12\n\n\nclass JsonResponse(http.JsonResponse):\n    \"\"\"The only reason this exists is so that other Django views can call\n    views that return instances of this and then get to the data before it\n    gets JSON serialized.\n    This is something that rest_framework's JsonResponse supports.\n    Ultimately, the only view that cares is the (old) Kuma search view page\n    that calls the view function here in this file. Now it can do something like:\n\n        response = kuma.api.v1.search.search(request)\n        found = response.data\n\n    \"\"\"\n\n    def __init__(self, data, *args, **kwargs):\n        self.data = data\n        super().__init__(data, *args, **kwargs)\n\n\ndef legacy(request, locale=None):\n    raise NotImplementedError(\"work harder\")\n\n\n@allow_CORS_GET\ndef search(request, locale=None):\n    initial = {\"size\": 10, \"page\": 1, \"archive\": SearchForm.ARCHIVE_CHOICES[0]}\n    if locale:\n        initial[\"locale\"] = locale\n    form = SearchForm(request.GET, initial=initial)\n    if not form.is_valid():\n        return JsonResponse({\"errors\": form.errors.get_json_data()}, status=400)\n\n    locales = form.cleaned_data[\"locale\"] or [settings.LANGUAGE_CODE]\n    assert isinstance(locales, list)\n\n    params = {\n        \"locales\": [x.lower() for x in locales],\n        \"archive\": form.cleaned_data[\"archive\"],\n        \"query\": form.cleaned_data[\"q\"],\n        \"size\": form.cleaned_data[\"size\"],\n        \"page\": form.cleaned_data[\"page\"],\n        \"sort\": form.cleaned_data[\"sort\"],\n        # The `slug` is always stored, as a Keyword index, in lowercase.\n        \"slug_prefixes\": [x.lower() for x in form.cleaned_data[\"slug_prefix\"]],\n    }\n\n    # By default, assume that we will try to make suggestions.\n    make_suggestions = True\n    if len(params[\"query\"]) > 100 or max(len(x) for x in params[\"query\"].split()) > 30:\n        # For example, if it's a really long query, or a specific word is just too\n        # long, you can get those tricky\n        # TransportError(500, 'search_phase_execution_exception', 'Term too complex:\n        # errors which are hard to prevent against.\n        make_suggestions = False\n\n    results = _find(\n        params,\n        make_suggestions=make_suggestions,\n    )\n    response = JsonResponse(results)\n\n    # The reason for caching is that most of the time, the searches people make\n    # are short and often stand a high chance of being reused by other users\n    # in the CDN.\n    # The worst that can happen is that we fill up the CDN with cached responses\n    # that end up being stored there and never reused by another user.\n    # We could consider only bothering with this based on looking at the parameters.\n    # For example, if someone made a search with \"complex parameters\" we could skip\n    # cache-control because it'll just be a waste to store it (CDN and client).\n    # The reason for not using a \"shared\" cache-control, i.e. `s-max-age` is\n    # because the cache-control seconds we intend to set are appropriate for both\n    # the client and the CDN. If the value set is 3600 seconds, that means that\n    # clients might potentially re-use their own browser cache if they trigger\n    # a repeated search. And it's an appropriate number for the CDN too.\n    # For more info about how our search patterns behave,\n    # see https://github.com/mdn/kuma/issues/7799\n    patch_cache_control(response, public=True, max_age=SEARCH_CACHE_CONTROL_MAX_AGE)\n    return response\n\n\ndef _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):\n    search_query = Search(\n        index=settings.SEARCH_INDEX_NAME,\n    )\n    if make_suggestions:\n        # XXX research if it it's better to use phrase suggesters and if\n        # that works\n        # https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html#phrase-suggester\n        search_query = search_query.suggest(\n            \"title_suggestions\", params[\"query\"], term={\"field\": \"title\"}\n        )\n        search_query = search_query.suggest(\n            \"body_suggestions\", params[\"query\"], term={\"field\": \"body\"}\n        )\n\n    sub_queries = []\n    sub_queries.append(Q(\"match\", title={\"query\": params[\"query\"], \"boost\": 2.0}))\n    sub_queries.append(Q(\"match\", body={\"query\": params[\"query\"], \"boost\": 1.0}))\n    if \" \" in params[\"query\"]:\n        sub_queries.append(\n            Q(\"match_phrase\", title={\"query\": params[\"query\"], \"boost\": 10.0})\n        )\n        sub_queries.append(\n            Q(\"match_phrase\", body={\"query\": params[\"query\"], \"boost\": 5.0})\n        )\n\n    sub_query = query.Bool(should=sub_queries)\n\n    if params[\"locales\"]:\n        search_query = search_query.filter(\"terms\", locale=params[\"locales\"])\n    if params[\"archive\"] == \"exclude\":\n        search_query = search_query.filter(\"term\", archived=False)\n    elif params[\"archive\"] == \"only\":\n        search_query = search_query.filter(\"term\", archived=True)\n\n    if params[\"slug_prefixes\"]:\n        sub_queries = [Q(\"prefix\", slug=x) for x in params[\"slug_prefixes\"]]\n        search_query = search_query.query(query.Bool(should=sub_queries))\n\n    search_query = search_query.highlight_options(\n        pre_tags=[\"<mark>\"],\n        post_tags=[\"</mark>\"],\n        number_of_fragments=3,\n        fragment_size=120,\n        encoder=\"html\",\n    )\n    search_query = search_query.highlight(\"title\", \"body\")\n\n    if params[\"sort\"] == \"relevance\":\n        search_query = search_query.sort(\"_score\", \"-popularity\")\n        search_query = search_query.query(sub_query)\n    elif params[\"sort\"] == \"popularity\":\n        search_query = search_query.sort(\"-popularity\", \"_score\")\n        search_query = search_query.query(sub_query)\n    else:\n        popularity_factor = 10.0\n        boost_mode = \"sum\"\n        score_mode = \"max\"\n        search_query = search_query.query(\n            \"function_score\",\n            query=sub_query,\n            functions=[\n                query.SF(\n                    \"field_value_factor\",\n                    field=\"popularity\",\n                    factor=popularity_factor,\n                    missing=0.0,\n                )\n            ],\n            boost_mode=boost_mode,\n            score_mode=score_mode,\n        )\n\n    search_query = search_query.source(excludes=[\"body\"])\n\n    search_query = search_query[\n        params[\"size\"] * (params[\"page\"] - 1) : params[\"size\"] * params[\"page\"]\n    ]\n\n    retry_options = {\n        \"retry_exceptions\": (\n            # This is the standard operational exception.\n            exceptions.ConnectionError,\n            # This can happen if the search happened right as the index had\n            # just been deleted due to a fresh re-indexing happening in Yari.\n            exceptions.NotFoundError,\n            # This can happen when the index simply isn't ready yet.\n            exceptions.TransportError,\n        ),\n        # The default in redo is 60 seconds. Let's tone that down.\n        \"sleeptime\": settings.ES_RETRY_SLEEPTIME,\n        \"attempts\": settings.ES_RETRY_ATTEMPTS,\n        \"jitter\": settings.ES_RETRY_JITTER,\n    }\n    with retrying(search_query.execute, **retry_options) as retrying_function:\n        response = retrying_function()\n\n    if total_only:\n        return response.hits.total\n\n    metadata = {\n        \"took_ms\": response.took,\n        \"total\": {\n            # The `response.hits.total` is a `elasticsearch_dsl.utils.AttrDict`\n            # instance. Pluck only the exact data needed.\n            \"value\": response.hits.total.value,\n            \"relation\": response.hits.total.relation,\n        },\n        \"size\": params[\"size\"],\n        \"page\": params[\"page\"],\n    }\n    documents = []\n    for hit in response:\n        try:\n            body_highlight = list(hit.meta.highlight.body)\n        except AttributeError:\n            body_highlight = []\n        try:\n            title_highlight = list(hit.meta.highlight.title)\n        except AttributeError:\n            title_highlight = []\n\n        d = {\n            \"mdn_url\": hit.meta.id,\n            \"score\": hit.meta.score,\n            \"title\": hit.title,\n            \"locale\": hit.locale,\n            \"slug\": hit.slug,\n            \"popularity\": hit.popularity,\n            \"archived\": hit.archived,\n            \"summary\": hit.summary,\n            \"highlight\": {\n                \"body\": body_highlight,\n                \"title\": title_highlight,\n            },\n        }\n        documents.append(d)\n\n    try:\n        suggest = getattr(response, \"suggest\")\n    except AttributeError:\n        suggest = None\n\n    suggestions = []\n    if suggest:\n        suggestion_strings = _unpack_suggestions(\n            params[\"query\"],\n            response.suggest,\n            (\"body_suggestions\", \"title_suggestions\"),\n        )\n\n        for score, string in suggestion_strings:\n            if score > min_suggestion_score or 1:\n                # Sure, this is different way to spell, but what will it yield\n                # if you actually search it?\n                total = _find(dict(params, query=string), total_only=True)\n                if total[\"value\"] > 0:\n                    suggestions.append(\n                        {\n                            \"text\": string,\n                            \"total\": {\n                                # This 'total' is an `AttrDict` instance.\n                                \"value\": total.value,\n                                \"relation\": total.relation,\n                            },\n                        }\n                    )\n                    # Since they're sorted by score, it's usually never useful\n                    # to suggestion more than exactly 1 good suggestion.\n                    break\n\n    return {\n        \"documents\": documents,\n        \"metadata\": metadata,\n        \"suggestions\": suggestions,\n    }\n\n\ndef _unpack_suggestions(query, suggest, keys):\n    alternatives = []\n    for key in keys:\n        for suggestion in getattr(suggest, key, []):\n            for option in suggestion.options:\n                alternatives.append(\n                    (\n                        option.score,\n                        query[0 : suggestion.offset]\n                        + option.text\n                        + query[suggestion.offset + suggestion.length :],\n                    )\n                )\n    alternatives.sort(reverse=True)  # highest score first\n    return alternatives\n", "path": "kuma/api/v1/search/__init__.py"}]}