Why equation Y^2=X^3 +AX+B can't work with finte field of charateristic 2?
In $GF(2^n)$, $y+y = 0$ for all $y$. These elements are not integers; using something like “2” to denote a field element is merely a convention, and, as you demonstrate, a confusing one. In this case, $2y$ means $y+y$, not the product in the field of $y$ with the element conventionally represented by “2”.