The 22-year-old, who had a loan spell at Rotherham earlier this season, has agreed a deal with the Addicks until the summer of 2019.
Forster-Caskey scored nine goals in 82 appearances for the Seagulls after making his debut aged 16 in 2010.
"Jake is a real talent who has shown his quality in the Championship," boss Karl Robinson told the club website.
Former England Under-21 international Forster-Caskey worked under 36-year-old Robinson last season during two loan stints at former club MK Dons.
He is Charlton's second signing of the January transfer window, and his arrival at The Valley came after winger Ademola Lookman left the Addicks to join Premier League side Everton in an £11m deal.
Meanwhile, Brighton under-23 midfielder Jack Harper has moved to Spanish side Malaga for an undisclosed fee.
Find all the latest football transfers on our dedicated page.
League One side Charlton Athletic have signed midfielder Jake Forster-Caskey from Brighton for an undisclosed fee.