Using a factory pattern many times
First of all, a client-server relationship is kind of a 1:1 relation. That means that the client should probably have a connection to a server (or, if abstracted, a server) to talk with. Currently you are using a server parameter, but I would argue that this should be a field. This will also free one parameter of every method. Furthermore, currently it seems allowed to call just any server from the client class. First register and then call a totally different server.

Furthermore, if such a generic &amp; relatively static parameter is present then it should probably be the first parameter. Currently the location of the server parameter seems to change per function, and that makes reading the functions harder.

The dispatchFollowEvent seems to be used to "follow a user", but that's only clear to me after reading the code. Similarly we have an upload method that seemingly uploads an image. Why not just followUser or uploadImage? It's a client, we expect it to "dispatch" and that it creates an "event" at the server.

I'm not sure why you would use a factory in above code. The makePacket code is really just an intermediate to the specific packets that you need anyway. So now you go: "packet specific method -> packet enum -> packet specific method". Sure you can do that, but why should you?

A more grievous thing is that your factory method seems to create partial packets without a payload. That means that packets are initially in an invalid state. The whole idea is that you manufacture concrete products inside a factory.

Another minor issue is the send part of the sendToServer method. A send method generally doesn't receive any data back. If you have a callback handler, why not call it call instead? You can leave the server part out of it maybe, that a client calls a server seems kind of logical.

Unfortunately I have to leave it at this, not just because it is late, but also because I don't think I've seen enough of your code / architecture to refactor the given code. It's not terrible code or anything, but it seems rather over-engineered and especially the naming suffers from it.