Do I need to back off on the extrusion speed or is there a

Do I need to back off on the extrusion speed or is there a better way to even out the starts?

You could try reducing the prime amount after a retraction. Or enabling coasting. Or both. I had to do both.