For what it's worth, it's more heat than sun. Direct sun is great, of course, but if you can rig up a mini greenhouse, even a small sheet of plastic held around the plant by bamboo stakes, your peppers will thank you.
Here in HMB, the temperature never really varies all that much. I'd bet 90% of our days see a high and a low within 15° of 57°, with may of those days within 1° of 57° for both the high and low. That's not pepper weather! To improve conditions, I had to grow them under cover.* Even on a crappy, foggy day (we have a few of those…) temps in the greenhouse get well over 90° and thick, but the peppers really respond to it. My shishito and chocolate yum-yum peppers (yeah, that's their actual name) were all grown together in a 25-gal pot (2 plants of each), and I probably harvested 4lbs of shishitos and 2lbs of the chocolates. Habaneros are even tighter and denser-growing than those. I'll bet if you can find space for a 5-gal can you could make an acceptable harvest.
Even now, in November, I've got a couple of pepper plants going. One is a spicy thai pepper that's nearly ready for its first harvest (and destined for a chili oil bottle), and the other is something that I've completely forgotten, but is also nearly ready. Not enough to bottle up for sauce, but plenty for a stir-fry or whatnot.
*Last winter I built a greenhouse out of 9 old sliding glass patio doors I had collected over the past couple of years. It's about 150 sf with potting areas, a sink, and garden tool storage. I've got a small area for direct planting, including a satsuma mandarin that's still a few years away from producing anything, but most of what I grew this year was in 5-, 15- or 25- gallon surgery pots.
Yeah, to set fruit mid 80s during the day and mid 60s at night is best. The light, in my estimation, is important to get enough vegetative growth to support the fruit production.
And to bolster what you said, I grow all my peppers in five-gallon containers and probably will for the foreseeable future. I'm quite happy with my yields.