Publishing vs. putting work online under a free license
There are two separate questions tangled together here.  The first is whether scientists will cite something that has not gone through a traditional peer review process, and the second is whether a copyleft / open source repository project is a good way to manage such a non-traditional publication.

To the first, the answer is most certainly yes.  Many non-peer-reviewed publications are not just cited, but highly cited, particularly technical reports and standards.  These do not undergo peer review, but if they are useful to people, they will cite them---sometimes thousands of times.

For the second, well, consider the fate of most free and open projects, whether on GitHub or elsewhere.  Almost every project of this type dies quietly in the dark, because nobody notices or cares about it.  Some, of course, succeed, and a few have massive impact.  The question is: why should anybody care about your project?  If they do, then copylefting in a git repository is no better or worse a way to manage the document than any other way that communities manage non-peer-reviewed documents (though make sure you do it with a CC license, not a code license).  If they don't then the technological or ideological elegance of your approach will make no difference.

The bottom line is: you are focusing on the technology, rather than the community.  You need to understand how your work will interact with your community, and then the technology is a secondary issue that will follow from that interaction.