Pushing the Limits of Accelerator Efficiency While Retaining Programmability