Data sets of academic conference budgets
Based on my experience as a conference organizer, there are only two sets of organizations likely to have this information:


Professional societies, such as IEEE and ACM, which require their sponsored conferences to make budget reports as a condition of sponsorship.
Funding agencies, such as the US National Science Foundation (NSF), which have explicit programs for sponsoring meetings, and which therefore require reporting on how those funds are expended.


No other organization is in a position to compel systematic reporting (not even universities, unless they are managing the conference accounts), and thus conference organizers will not in general report it anywhere else.

Neither funding agencies nor professional societies are likely to give this financial information out willingly, since it is financial---though you never know what you might get if you ask the right people nicely enough.  Government agencies like the NSF, however, have public reporting requirements and may be susceptible to data gathering via requests for records release, under FOIA in the US or its equivalents in certain other countries.