Damping of electron center-of-mass oscillation in ultracold plasmas