Stripboard layout for a 16x16 dot matrix display
To answer your first question, either glue/wire or both should work fine. As will anything you can come up with that's suitably electrically/mechanically sound.

To answer the second question:
Yes it's achievable with stripboard - folk have built entire discrete logic based processors and other similarly complex circuits using stripboard.
That said, anything past a rough prototype is better done with a proper professionally made PCB. If it's a personal (i.e. one off non commercial) project then you could maybe etch your own.
There are plenty of cheap prototyping services available nowadays that offer far better results that you can get at home.
We just tried out the Seeed Studio service (10 boards for $10, sounded a bit too good to be true) and the results were excellent. The "catch" is the $10 is only for the smallest (5cm x 5cm IIRC) boards but even for the larger ones it's cheaper than any place I have seen so far (e.g. 10cm x 10cm is only an extra $15)    

In the end which route you choose it depends very much on the project's end requirements, but the stripboard will certainly work okay for something like this.