Why COUNT(*) on linked server is inaccurate?
Some of the things that could describe a different count on the same table (remote or not) are:


query a different table without knowing it due to default schema
query at a different time, where data changes have happened in the meantime
query with different isolation semantics (e.g. using NOLOCK or snapshot during data changes)
query a table with row-level security
query a view with filtering of any kind
query a synonym that actually points to a different object


Other explanations are usually that they are not the same table. Does server B have a loopback linked server to server B? Or did you execute a slightly different query on server B? Is it possible it was in a different database, or that the linked server on server A might be pointing at something else?