Saddle coils (SCs) are proposed as coils with a stabilizing effect of a vertical plasma position. This effect comes from an average magnetic field along a magnetic field line. Magnetic field line tracing was performed to investigate the structure and the quantitative values of the averaged magnetic field generated by SCs and toroidal magnetic field coils (TFCs). Experiments to stabilize the vertical position of the plasma were also carried out in a small tokamak device, PHiX. It was experimentally demonstrated that the averaged magnetic field generated with SCs and TFCs could stabilize the vertically unstable plasmas. In addition, three-dimensional equilibrium calculation using VMEC suggested that the plasmas were vertically elongated on the toroidal average when the vertical positions were stabilized by SCs. From the above results, it was shown that the non-axisymmetric magnetic fields generated by SCs could realize the plasmas with stable vertical position and an elongated cross-section.