A general expression for the growth rate of the toroidal Alfvén eigenmode (TAE) instability is derived by employing a Hamiltonian action-angle approach. It takes into account the finite orbit width effects, the finite Larmor radius (FLR) effects of energetic particles and the effect of mode localization, as well as the possible mode excitation by both passing and trapped energetic ions. In particular, the stabilizing effect of the large particle orbit widths on the TAE modes excited by fast passing and trapped ions is investigated. In the limiting cases when and , the instability drive is reduced by factors of and , respectively, as compared with the prediction of the narrow orbit theory, when , where is the particle orbit width and is the inner (outer) width of the mode structure. For the TFTR parameters, the stabilization effect is much stronger in the case of TAE excitation by fusion-produced alpha particles than that in connection with neutral beam injection (NBI)-generated energetic ions.