I've never had a problem with it....
it depends on how long the ride is, and where your bike is stored, and also how you get to the trail head. Cold will do two things to an air fork or shock. It will shrink the seals and reduce the air pressure in the chambers (just like it reduces the air pressure in car tires the colder it gets) and it tends to thicken the oils and lubes that are present in both. What the tighter seals usually do is make for a stickier feeling suspension, and can result in minor leakage of lube or oil. The thicker sluggish oil tends to make things slower and a bit stiffer feeling as well. The recuction in air pressure is oviously going to cause too much sag and you'r blow through travel and bottom out more.
What I do for winter riding is this. The bike is stored INDOORS. When transporting to a trail it NEVER goes on an outside rack, always inside the vehicle. What I find is that keeping everything at room temperature before the ride negates most of the negative effects of cold. By the time the cold would normally start affecting lubes and seals the riding that you are doing is keeping them warm enough to prevent any problems. As forks and shocks work they do generate some of their own heat. It seems to be enough to keep things working right. The only thing that I find that I have to do is add about 5psi to the tires to compensate for the cold. And if it's only gonna be a 20 to 60 minute ride you don't even have to do that usually.
Anyway, 32 degree weather shouldn't have that much of affect on your suspension unless the bike is stored in an unheated shed or garage. Then you can look forward to making some adjustments to compensate for the cold. Air pressure adjustments, lighter oils and lubes etc. And the fork or shock will feel stiffer until you get things warmed up a bit through riding. But for the most part you won't noticeably shorten the life of seals etc.
Good Dirt