In the first campaign Wendelstein 7-X was operated with five poloidal graphite limiters installed stellarator symmetrically. In an ideal situation the power losses would be equally distributed between the limiters. The limiter shape was designed to smoothly distribute the heat flux over two strike lines. Vertically the strike lines are not uniform because of different connection lengths. In this paper it is demonstrated both numerically and experimentally that the heat flux distribution can be significantly changed by non-resonant perturbation field of the order of . Numerical studies are performed with field line tracing. In experiments perturbation fields are excited with five error field trim coils. The limiters are diagnosed with infrared cameras, neutral gas pressure gauges, thermocouples and spectroscopic diagnostics. Experimental results are qualitatively consistent with the simulations. With a suitable choice of the phase and amplitude of the perturbation a more symmetric plasma-limiter interaction can be potentially achieved. These results are also of interest for the later W7-X divertor operation.