How to trace a process sending data over network? I found that some mobile data is consumed by unknown process. There's a huge difference between my ISP report and glasswire report, around 40mb for each 100mb used. 

I know my phone is infected and someone spying on me but couldn't find how to get rid of that.

I wiped boot/recovery/data/cache and dalvik partitions before installing a custom ROM (dotOS) from a trusted computer. I managed to root the phone and install a firewall to check what's happening but got nothing worth it.

My assumption is that some process is still running somewhere, I have no idea how can I, at least, see what's going on. I do not have any background with Android development but I understand most of the concepts around it.

It's not a system application so please do not mention that.
There are multiple ways to sniff outgoing network traffic. Since Android uses Linux kernel, we can make use of in-kernel reliable firewall: netfilter which is controlled using userspace utility iptables. It's the easiest available option for rooted phones.
Through a root commandline:

# iptables -I OUTOUT -j LOG --log-prefix "SNIFFER " --log-uid --log-level 7


This will log all outgoing traffic to kernel log that you can capture and save to text file:

# dmesg -w | grep SNIFFER


This will provide you uid of the all the processes sending out traffic, which can be used to trace the processes.  

AFWall+ app also makes use of iptables owner module to block traffic from certain user/system apps (UIDs) including root.

For further details on capturing packets, see View network traffic requested by app.