Machine learning approach to auction game
There probably is no solution that will be optimal for all possible situations.  So, you are left with heuristics: trying to find a technique that will often work well in the situations that arise for you in practice, but with no guarantees that it will work for everything.  Machine learning might be one possible such approach.

You might start by trying to learn the function $f$.  You said that the agent is free to invoke $f$ on any set of items.  Therefore, a possible starting point is to try to learn a model that predicts $f$: e.g., given a representation of the set $I$, outputs an estimate for $f(I)$.  You can easily collect a training set by querying $f$ on many sets and observing its output.

Reinforcement learning would be a plausible approach to the broader problem.  There are many approaches to reinforcement learning, but in one typical approach, we need to compute a function $V(a)$ that computes the anticipated value of being in state $a$ (i.e., if the agent is in state $a$ and then follows its policy/strategy, what do we expect the final value of $f(I)$ will be?).  Normally machine learning techniques are used to build a model that estimates the value of $V(a)$.  This then requires forming a representation of the state $a$, i.e., a feature vector that summarizes or extract relevant information about the current state.  The current state includes: (a) the set of items bought by this agent so far, (b) the budget and number of slots available to this agent, (c) the number and list of items remaining to be bought, and (d) information about the other agents, including what they've bought so far and the number of slots and budget remaining to them.  So, you can think about how to build a feature vector that represents them, then apply standard methods for reinforcement learning to learn the function $V(a)$.

I suggest reading a tutorial on reinforcement learning to learn more about how it works and see some examples in a simpler setting.