Highest cost section in Actual Execution Plan doesn't show any logical reads? In the following query plan - https://www.brentozar.com/pastetheplan/?id=H1dwOCHed (it has 3 sections, the last one is the problem), there's a node at the very bottom that consumes 37% of the resources. This is the actual query plan, not the estimated query plan.
However, the table it references does not show up at all, when looking at logical reads via Set Statistics IO. This includes the 2 sections for ActivityFeedTeamCache and ActivityFeedCache.
In the past, I've been able to use Logical Reads as a pretty good proxy for performance. Reduce the logical reads, and you've done the right thing for performance.
However, since in this case, there are no logical reads for the chunks that are the worst (ActivityFeedTeamCache, ActivityFeedCache, Locales), what is a good proxy for testing performance? Just that the % cost in the Actual Execution Plan goes down?
The % cost is still just an estimation, even in an Actual Execution Plan. You need to look at the row source level statistics here to tell you how much time each step took.
Judging by the XML, 274ms was taken up to the point of the hash join at node 7, 72ms of that was just the cluster scan of Pk_TeamMembers (node 8) and 35ms was up to the the Left Semi Join (node 9). This means that a whopping 167ms was taken just to execute the Hash Join step (likely assigning memory for the 263,600 rows from Pk_TeamMembers).
It looks like you've already tried to target this join before:
LEFT JOIN TeamMembers tm on AF.CompletionUserID = tm.UserID AND AF.TeamID = tm.TeamID AND tm.TeamID = t.TeamID -- Overloaded to help SQL find indexes

You probably want to do an (Outer) nested loop to this index instead. The query planner has decided to use a hash join because it thinks that you will get 59000 rows from the other side of this join, when in fact you get 3.
You have choices

Hint so that a nested loop join is used
Figure out why the planner thinks it will have so many rows and give it some help (statistics, simplify the filtering)

The hint would look like
LEFT LOOP JOIN TeamMembers tm on AF.CompletionUserID = tm.UserID AND AF.TeamID = tm.TeamID AND tm.TeamID = t.TeamID

The second choice will be particularly hard because of the complexity of your driving filters. One way to do this would be to insert the rows that match your main driving filters, and then join to this temp table in your main query. Here, it looks like you need to join 3 of the 5 tables together first in order to utilize your @userId@ filters fairly:
FROM ActivityFeed AF
    JOIN ActivityType A ON AF.ActivityType = A.ActivityType
    LEFT JOIN Teams t on AF.TeamID = t.TeamID
WHERE 
    AF.SearchIndexUpdateDate IS NULL -- Only get items that aren't in the search index
    AND
    AF.Active=1 
    AND 
    A.Active=1
    AND
    (A.AdminOnly = 0 OR AF.TeamID IN (SELECT TeamID FROM @tmpFindAdminFlagsForTeamsMembershipsForUser x WHERE x.IsAdmin=1))
    AND
    ( 
      (
        A.TeamOnly=1 
        AND AF.TeamID IN (SELECT TeamID FROM @tmpFindAdminFlagsForTeamsMembershipsForUser x WHERE A.HideWhenOwnerless = 0 OR x.IsOwnerless = 0)
        AND (A.ShowSelf=1 OR AF.CompletionUserID &lt;&gt; @UserID)
        AND ISNULL(t.IsDeleted,0) = 0 AND t.active=1
        AND (A.IncludeOnsite = 1 OR EXISTS(SELECT 1 FROM @tmpFindAdminFlagsForTeamsMembershipsForUser x WHERE x.TeamID = AF.TeamID AND x.IsOnsiteOwner = 0))
       )
       OR 
      (
        A.UserOnly=1 
        AND AF.UserID=@UserID
      )
    )
    AND
    -- Don't included moderated activity (owner moderation actions are in Group activity view API_GetActivityFeedByTeam)
    AF.Moderated = 0 
    AND
   -- don't show completions or plan completions to other team members if opted out (tm.ShowUserActivityInFeed)
    (ISNULL(A.ShowOnlyWhenUserActivityActive,0) = 0 OR (A.ShowOnlyWhenUserActivityActive = 1 AND tm.ShowUserActivityinFeed = 1))
    AND
    (AF.TeamID IS NULL OR AF.TeamID IN (SELECT TeamID FROM @tmpFindAdminFlagsForTeamsMembershipsForUser as x))