 CONTINUATION PHASE ARMS A and C: Patients receive pembrolizumab IV over 30 minutes on day 1. Patients also receive idelalisib orally PO twice daily BID on days 1-21 OR ibrutinib PO once daily QD on days 1-21. Treatment repeats every 21 days for up to 12 or 24 months in the absence of disease progression or unacceptable toxicity. Patients receiving benefit may continue to receive treatment for an additional 12 months at the discretion of the investigator. 