A central part of CASCI (and other active-space or multi-reference configuration interaction calculations) includes performing an FCI expansion within an active space defined by a number of active electrons N el with all possible excitations within a number of active orbitals N orbs , (N el , N orbs ), where just as in FCI, the number of determinants scales factorially with the number of active electrons and orbitals. A number of algorithmic advances have pushed the limits of the sizes of these calculations. A major advance in direct CI algorithms in the 1990s due to Olsen and coworkers led to active spaces on the order of (10,10) with roughly 1 billion determinants. The current state of the art in massively-parallel direct active space CI algorithms can treat active space sizes of (22,22) with roughly 1 trillion determinants. There have also been several exciting developments that can push even beyond this (22,22) active space size through various means of reducing the number of determinants that are included in the CAS wavefunction, including the Selected CI approach using a basis of tensor product states, adaptive sampling CI, the so-called iCISCF approach, and active learning approaches to CI. Additionally, a family of active-space approaches where the 2-particle reduced density matrix (rather than the many-electron wavefunction) is variationally optimized can also enable active spaces beyond this (22,22) limit, including CASSCF-like calculations with active spaces as large as (64,64). In this work, we adopt a serial implementation of the direct CI approach of Olsen and co-workers to ab initio QED using the Pauli-Fierz Hamiltonian within the dipole and Born-Oppenheimer approximation. We present two formulations of QED-CASCI: PN-QED-CASCI, in which the photonic space is represented in the basis of photon number (PN) states, and CS-QED-CASCI, in which the photonic space is represented in the coherent state (CS) basis. Our serial implementation can routinely handle (12,12) active spaces for the electronic subsystem with at least 100 photonic states.