How to run analysis on a 16M row table faster I have two tables students with 6k records and transactions with 16M records. 

I have a query similar to this:

select section, count(s.students) from students as s
where upper(name) in 
    (select upper(name) from transactions as t 
        where (length(transaction_id) &lt; 43 or 
            lower(transaction_name) = 'NA' or 
            lower(transaction_type) = 'food')
            and upper(name) not in ('JOHN DOE')) 
group by section 
order by section asc


The query plan for this is similar to below:

 GroupAggregate  (cost=2828274.40..2828291.32 rows=6 width=12) (actual time=101295.078..101301.568 rows=6 loops=1)
   Group Key: section
   -&gt;  Sort  (cost=2828274.40..2828280.02 rows=2247 width=25) (actual time=101294.727..101300.274 rows=2414 loops=1)
         Sort Key: section
         Sort Method: quicksort  Memory: 285kB
         -&gt;  Hash Join  (cost=2827697.15..2828149.32 rows=2247 width=25) (actual time=101278.618..101292.861 rows=2414 loops=1)
               Hash Cond: ((s.name)::text = (t.name)::text)
               -&gt;  Seq Scan on students s  (cost=0.00..412.16 rows=5716 width=25) (actual time=0.007..2.887 rows=5720 loops=1)
               -&gt;  Hash  (cost=2827669.07..2827669.07 rows=2247 width=21) (actual time=101278.578..101278.578 rows=2445 loops=1)
                     Buckets: 4096  Batches: 1  Memory Usage: 159kB
                     -&gt;  HashAggregate  (cost=2827646.60..2827669.07 rows=2247 width=21) (actual time=101274.023..101276.849 rows=2445 loops=1)
                           Group Key: (t.name)::text
                           -&gt;  Gather  (cost=1000.00..2804729.14 rows=9166982 width=21) (actual time=97.170..100735.029 rows=449343 loops=1)
                                 Workers Planned: 2
                                 Workers Launched: 2
                                 -&gt;  Parallel Seq Scan on transactions t  (cost=0.00..1887030.94 rows=3819576 width=21) (actual time=130.428..100660.838 rows=149781 loops=3)
                                       Filter: (((name)::text &lt;&gt; 'JOHN DOE'::text) AND ((length((transaction_id)::text) &lt; 23) OR (lower((transaction_type)::text) = 'food'::text)) OR OR (lower((transaction_name)::text) = 'NA'::text))))
                                       Rows Removed by Filter: 4897554
 Planning time: 10.027 ms
 Execution time: 101301.982 ms
All your time is spent in a sequential scan that discards almost all the rows it scans. Unfortunately you have an OR in there, so the best you can do is to create three indexes and hope for a "bitmap or" of the three index scans:

CREATE INDEX transactions_type_idx ON transactions (lower(transaction_type::text));
CREATE INDEX transactions_name_idx ON transactions (lower(transaction_name::text));
CREATE INDEX transactions_id_length_idx ON transactions (length(transaction_id::text));