Theory of magnetic turbulence and shock formation induced by a collisionless plasma instability