Fetch wikipedia articles query optimization in mysql wikipedia Database
OFFSET must locate, but skip over, the rows.  If you can "remember where you left off", performance may be significantly better.

But there is a worse problem with your query.  Since there is no ORDER BY, the optimizer is free to provide the rows in any order it likes, thereby leading to missing or duplicated rows.

Even if you add an ORDER BY, OFFSET can give miss or duplicate rows if the table is being added to or deleted from.

Read my blog on this problem, then start over on your query.