1. A hardware accelerator comprising: one or more tiles, wherein each tile includes: a plurality of processing elements (PEs) to perform matrix operations involving, as a first operand, one or more very- or hyper-sparse matrices that are stored by a memory; and a data management unit (DMU) to provide the plurality of PEs access to the memory, the memory to be coupled with the hardware accelerator via an interface that is optimized to provide low-latency, parallel, random accesses to data; wherein the plurality of PEs, via the DMU, perform the matrix operations by, issuing a first set of random access read requests for values of the one or more matrices after identifying locations of the values by issuing random access read requests for pointer values; issuing a second set of random access read requests for values of a first set of one or more vectors serving as a second operand; and issuing a third set of random access write requests for values of a second set of one or more vectors serving as a result.