Is it possible to modify an AV-8B Harrier II to a supersonic aircraft by applying some modification?
There was actually a plan to convert the Harrier into a supersonic aircraft. 

The original plan that lead to the development of the Harrier actually had a supersonic variant, the Hawker Siddeley P.1154. The P.1154 was supposed to be a supersonic aircraft using the Bristol Siddeley BS100 engine, which used Plenum Chamber Burning(PCB).

In the BS100 engine, cold bypass and hot core turbine airflows are split between two sets of nozzles, front and rear, like the Rolls Royce Pegasus engine which powered the Harrier. The PCB called for additional fuel and afterburning applied to the front cold air nozzles only, increasing thrust. The PCB increased the thrust from 116 kN to 160 kN.



Source: aeromodelbasic.blogspot.com

However, the P.1154 never saw the light of the day due to various reasons and the subsonic Hawker P.1127 was developed into the AV-8 Harrier. 

Later, Hawker and McDonnel Douglas joined to develop the AV-16 concept, basically a plan to double the range and payload of the Harrier. One version of the AV-16 called for supersonic flight too.

Some modifications that were considered in the study for supersonic aircraft were,


A more powerful engine (called Pegasus 15) which used PCB
A bigger fuselage for the bigger engine
A larger 'supercritical' wing
A lengthier, more slender fuselage


This version was called HS.1185, as shown below.



Source: harrier.org.uk

However, in March 1975, Hawker pulled out due to budget cuts and other reasons. 

Later, Rolls Royce (ground) tested Plenum Chamber Burning in the Pegasus engine and made some progress in the 1980s. This was supposed to be used in yet another (supersonic) variant of Harrier. One of the versions was the HS. 1205, which had redsigned F-16 type inlets.



Source: harrier.org.uk

However, this also came to nothing and the project was shelved. For a full list, see Harrier site

To answer your question, it would be possible to make the Harrier go supersonic using more powerful engines and modifications to airframe (listed above).