Measuring Value in Data Science?
I read your question as:


  tl;dr. How do you know that a data scientist adds value to a company?


The job of a data scientist is to pick out the best data set, scrub it, fit a statistical model. And most importantly, a data scientist needs to know the problem statement and why is the analysis being done.

For example, consider a DS team in an e-commerce company. Their primary (let's assume) is to build a highly effective recommender system, which needs to adapt itself to a lot of variables, like the user behaviour, the session-based behaviour, user segments, etc.

So, if the recommender system is still not being able to get the company a considerable CTR or upsells/cross-sells, then maybe the team is not adding proper value to the company.

So, a data scientist needs to have a keen knowledge of the business value which an experiment is expected to add to the company. The decisions made from the experiments should ultimately add value to a company (monetary or otherwise). If not, then just like any other employee, it means that the person have failed in the job.