Query with a few subqueries and appropriate indicies taking, on average, 613ms to execute
Your query plan shows two table scans on leads, which is not surprising given the search predicates with OR: FROM leads WHERE to_id = lead_id OR from_id = lead_id.

It also shows a spilled sort, suggesting that the sort buffer is too small.

Consider rewriting your query with joins instead of subselects, something like:

SELECT 
  COALESCE(leads_to.lead_id, leads_from.lead_id) as lead_id, 
  COALESCE(CONCAT(leads_to.first_name, " ", leads_to.last_name), CONCAT(leads_from.first_name, " ", leads_from.last_name)) as lead_name 
FROM 
  text_messages t 
LEFT JOIN leads leads_to ON to_id = leads_to.lead_id
LEFT JOIN leads leads_from ON to_id = leads_from.lead_id
WHERE sent_to_number = "xxx" OR sent_from_number = "xxx" 
  AND organization = 1 GROUP BY lead_id ORDER BY sent_at DESC


To address the sort spill try increasing sort_buffer_size.