I have a machine learning problem where a single training example can be represented by an m by n matrix, where $m$ is fixed but n can vary. Essentially, a training example consists of n instances, each with $m$ features. The task is to perform binary classification. I want to find a ML model to deal with this data with the following properties:

1. The architecture should be able to deal with the fact that n is not fixed.
2. The predictions of the model should be invariant under permutations of the n columns.
3. The architecture should be able to make use of all or almost all of the information encoded by the matrix, up to permutations of the columns.
4. (Less important) Uses as few parameters as possible while satisfying the first three conditions

Does anyone have any suggestions for suitable models, particularly NN architectures?

The main idea I've had so far was to compress the matrix into a vector of length $m$ by finding the average value of each feature across the $n$ instances and feed that into a dense NN layer but that fails to satisfy property 3 since we will lose all information about which features appeared in the same instance.