You can ride in almost anything, though on longer rides jeans and other garments with bulky seams are not ideal.  Wearing a number of layers lets you control your temperature easier than just wearing one or two layers.

A wide range of cycle specific clothing is available from outer coats to underwear.  Breathable waterproofs can help you keep comfortable and cut out wind chill.

Gloves add to comfort by keeping the wind chill at bay, and should you fall off they offer great protection for your hands.  If you’re going a long way in the rain waterproof overshoes and trousers can also help.
By having the right clothing for the season you can comfortably cycle in all weathers.