This beast needs to be split up in many tiny little manageable pieces. Preferably with a chainsaw.
allow for multithreading, perhaps use libevent. Not just yet, though; follow the road map.