The air conditioners that have both heat and cool modes only heat at 60F and above. They do not switch from cooling to heating automatically.
One solution that we've found is to buy a little heater (buy the blower type, not radiant type) Plug it into a pass-through thermostat like the DuroStat Portable Waterproof Prewired Thermostat from FarmTek.com (Item #CR2045; it's about $79). Another product that some of our customers have used (particularly for micro-brewing) is the AGPtek STC-1000 (Amazon link). Make sure you choose a thermostat that can be set below 40 degrees F!
Whichever thermostat you choose, we recommend setting the CoolBot 5F - 7F degrees warmer than the heater so they don't overlap and fight with each other.