Which bitcoin script forms should be detected when tracking wallet balance? I am implementing a library for tracking wallet balance of given bitcoin addresses.

How should the transaction output script be parsed to find if the output belongs to the given address?

Is it dirty if I just match pay-to-pubkey-hash and pay-to-script-hash transaction by simply matching

OP_DUP OP_HASH160 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OP_EQUALVERIFY OP_CHECKSIG


and 

OP_HASH160 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OP_EQUAL


or should I implement more sophisticated script parser which can detect "non-standard" scripts as well?
These are the scriptPubKey's that are standard, arranged in order of popularity:


P2PKH (Pay to public key hash)

OP_DUP OP_HASH160 &lt;20 bytes of public key hash&gt; OP_EQUALVERIFY OP_CHECKSIG

P2SH (Pay to script hash)

OP_HASH160 &lt;20 bytes of script hash&gt; OP_EQUAL

P2PK (Pay to public key)

This is now mostly unused, except by miners, but there are still unspent outputs in the form:

&lt;33 or 65 bytes of public key&gt; OP_CHECKSIG

Segregated witness outputs

There are two ways of signaling that a scriptPubKey should be interpreted as a segregated witness program:


Native witness program

&lt;version byte&gt; &lt;2 to 40 bytes of witness data&gt;


The version byte must be equal to or less than 16.
P2SH wrapped witness program

OP_HASH160 &lt;20 bytes of script hash&gt; OP_EQUAL


The scriptSig must have exactly one item. That item is interpreted as a witness program, as above.


&nbsp;

Once you extract the witness program, you must match it to one of the witness program types:


P2WPKH

0 &lt;20 bytes of public key hash&gt;


This is hashed identically to P2PKH.
P2WSH

0 &lt;32 bytes of witness script hash&gt;


This is pretty similar to P2SH, except that the script hash is SHA256, not HASH160. 
Other version 0 scripts, not 20 or 32 bytes in length, are nobody-can-spend, so ignore them.


&nbsp;

Note that any witness program format can be combined with any witness program. For example, you can have P2SH wrapping P2WSH, like this:

scriptPubKey:

OP_HASH160 &lt;20 byte hash of scriptSig element&gt; OP_EQUAL


scriptSig:

&lt;34 byte serialized witness program&gt;


unserialized witness program:

0 &lt;32 byte hash of first witness item&gt;


Then, the first witness item is unpacked and interpreted as the actual program.

More information here.
m-of-n bare multisig

Also pretty rare.

&lt;m&gt; [n &lt;public key&gt;s] &lt;n&gt; OP_CHECKMULTISIG

Data-carrying output

Only one of these is allowed per transaction. These outputs can never be spent, so you probably don't need to detect them.

OP_RETURN &lt;less than 80 bytes of data&gt;




  Should I implement more sophisticated script parser which can detect "non-standard" scripts as well?


No. It's a waste of development time, when nobody uses transactions like that. (If they did, those transaction types would be added to the list of standard transactions. :))

It's also going to be very difficult to implement correctly. If you don't do it correctly, then you'll think you have coins when you really don't. For example, imagine someone sends you a transaction with a scriptPubKey like this:

OP_DUP OP_HASH160 &lt;20 bytes of public key hash&gt; OP_2DROP OP_CHECKSIG


Note that this is like a P2PKH output, except that OP_EQUALVERIFY has been replaced by OP_2DROP. That means that the script checks that the signature is correct, but doesn't check that it's signed by the right key.

That means that anybody can spend it, not just you. Correctly detecting edge cases like these make writing a program that can understand nonstandard outputs more trouble than it's worth.