This paper addresses the control problem of inspecting underwater pipeline on the seabed, with coordinated multiple autonomous underwater vehicles in a formation. Based on the leader-follower strategy, the dedicated nonlinear path following controller is rigorously built on Lyapunov-based design, driving a fleet of vehicles onto assigned parallel paths elevated and offset from the underwater pipeline, while keeping a triangle formation to capture complete 3D images for inspection. Due to the spatial-temporal decoupling characteristics of individual path following controller, the velocities of the followers can be adapted in the coordinated control level, only relying on the information of generalized along-path length from the leader, in order to build the desired formation. Thus, the communication variable broadcast from the leader is kept to a minimum, which is feasible under the severely constraints of acoustic communication bandwidth. Simulation results illustrate the efficiency of coordinated formation controller proposed for underwater pipeline inspection.