A contractor at Blue Cross and Blue Shield in Jacksonville, Florida sent the names and social security numbers of employees, vendors, and contractors to his home computer. A programming consultant at an insurance company in Wisconsin stole and sold social security numbers. And another contractor at the Bank of America leaked personal customer information to an identity-theft ring.

Contractors, freelancers and temp workers are treated like regular employees. They swipe into offices with a security badges or use key-fobs to walk into buildings. But they’re often placed under much less scrutiny. So much less scrutiny, in fact, that many companies don’t even know how many contract workers they have hired. And that presents a security risk.

Because contract workers are often hired by project managers, or local branches of businesses, they can bypass a centralized HR process. Their presence may be accounted for as a line item on a budget rather than as part of a tally of headcounts.

Meanwhile, many temporary workers, such as IT workers, are given administrative logins to company systems. Others may not have unique logins to systems, which means it’s difficult to track what programs they’ve accessed or if they’ve leaked sensitive data. And in the eyes of a customer, anyone working on location for a company works for the company.

As the contingent workforce grows, this has become a bigger problem. “Headcount is fuzzy,” says Lisa Disselkamp, who specializes in workforce management as a managing director at Deloitte. “This never used to be the case. In a traditional company, it was easy to determine–it was how many people on payroll. Now with the Uber workforce, the Uberization of the workforce, we’re hiring and contracting, but nobody knows what the actual headcount is or what it should be.” She says that some of her clients are concerned about people who they don’t know about having access to their facilities, their equipment, their data.

No sector seems to have yet solved the problem. The Pentagon? The US federal government? Neither has cracked the code on counting its own workers. In fact, in a poll by freelance management software Work Market last year, almost all of the chief financial officers polled said that their companies hired contract workers. Only about two-thirds of them said that they knew exactly how many.