OFB or CTR mode without an IV I'm reading the SSL specs here.

The interesting thing seems to be that RC4 is not re-keyed with a new IV for each message. The stream cipher state is simply carried over to the next record. Why is this secure? Shouldn't messages be encrypted with different IVs? Honestly, this is one part I don't fully understand about IVs: each message needs a different IV, but what counts as a message? A full stream of records? One record?

The CBC modes do have different IVs for each block.

My question is, how about a stream ciphery mode of operation, like OFB or CTR? Do I need new IVs for each smallish record, or can I do what SSL does with RC4 and just carry over the cipher state? Is that in any way insecure? WEP seems to do the new-IV-per-packet thing, and that seems to have caused its downfall. Why?
The answer from owlstead and its comments covers WEP part quite nicely.

This answer concentrates on CTR and OFB.

Strictly speaking, CBC, CFB, CTR, and OFB modes always use IV or counter. I'm assuming that the question was more like is it possible to use CTR or OFB mode securely without transmitting IV. I.e. for instance, start at all zeroes IV/counter and increment the counter for each message. This appears to be possible and secure. (As long as you're careful enough to never reuse same counter+key combination.)

Details for selecting IV for CTR and OFB modes

The answer for IV(/nonce in CTR and OFB modes is found in document NIST SP 800-38A: Recommendation for Block Cipher Modes of Operation, which describes these modes. This document tells in Appendixes B and C various recommendations by NIST for choosing IV/counter for CBC, CFB, CTR and OFB. Long story short,
for CTR and OFB modes, it is sufficient that IV/counter used is always unique for the same key, as long as it is also unique for each invocation of the encryption operation.

In other words, counter is ok for CTR and OFB. In OFB mode, it is not safe to use cipher text from a previous operation as IV, a practice that has been often used with e.g. CBC mode. The language of NIST specification may be misleading. There is evaluation of cipher modes of operation from Rogaway, which you may find easier to read and understand, and containing a lot of additional detail.