1. A system for offloading processing from a host to one or more storage processing units each comprising a non-volatile memory (NVM), the system comprising: a host having a processing task; a plurality of storage processing units (SPUs); a host interface configured to enable communications between the host and each of the plurality of SPUs; and an interconnection network coupled to at least two of the plurality of SPUs; wherein the host is configured to: command at least one of the plurality of SPUs to perform the processing task; and command the interconnection network to couple two or more of the plurality of SPUs.