Do I need to tell my rec letter provider that I have language disorder?
No, nobody needs to know. Your letter writers need to attest to your academic abilities, whereas your medical history is not something they need to know nor do they have any business telling anyone else about it.

As a matter of fact, all that ought to matter are your academic abilities. The fact that you're applying for graduate school implies to me that you must have been reasonably good in school/college -- in other words, that your disability has no implications for your academic performance. That's all the admissions committee wants to know: that you'll excel academically, and they will be able to see that from your transcripts and what your letter writers describe in their letters.