We have found numerically that damping phases appear in the time evolution of the perturbation energy of high-n ballooning modes in the presence of toroidal flow shear, where n is a toroidal mode number. The damping dominates exponential growth which occurs in the bad curvature region, resulting in stabilization of ballooning modes. D-shaping of plasma cross-section, reduction of aspect ratio, and arrangement of an X-point at inner side of the torus enhance the stabilization effect of the toroidal flow shear through this mechanism.