What is the grammatical number of an inline citation?
I personally am comfortable with both ways, for exactly the reasons that you give: it's entirely reasonable to refer to either the authors or the paper.  The important thing is to be consistent with your usage throughout a manuscript, else your readers may wonder why you feel it appropriate to refer to some authors directly but not others.