In the linear collisionless limit, a zonal potential perturbation in a toroidal plasma relaxes, in general, to a non-zero residual value. Expressions for the residual value in tokamak and stellarator geometries, and for arbitrary wavelengths, are derived. These expressions involve averages over the lowest order particle trajectories, that typically cannot be evaluated analytically. In this work, an efficient numerical method for the evaluation of such expressions is reported. It is shown that this method is faster than direct gyrokinetic simulations performed with the Gene and EUTERPE codes. Calculations of the residual value in stellarators are provided for much shorter wavelengths than previously available in the literature. Electrons must be treated kinetically in stellarators because, unlike in tokamaks, kinetic electrons modify the residual value even at long wavelengths. This effect, that had already been predicted theoretically, is confirmed by gyrokinetic simulations.