Investigate sse/avx for fast frame scaling, probably makes sense to scale YUV before converting to RGB?
Consider putting it out as a separate library (libscale?), perhaps even incorporate into 9front itself.