Controlled waterways are special areas in the upper Yangtze River with narrow channels and sharp corners.Ships need to be sequenced and scheduled by the intelligent traffic signaling system to pass through the controlled waterways and are allowed to pass in one direction at a time for the safety reason.If the ships do not obey traffic command signal during the voyage ,the disastrous nautical accidents such as collision,stranding of the ships will be more likely to occur.This paper proposes a new approach to detect illegal navigation based on AIS,the video detection system and the traffic regulations.Firstly,the ships are identified by eliminating the background noises.After that,several cameras are used to track the ships trajectories in different scenarios.Finally,the illegal ships are estimated according to the traffic regulations of the ships on the Yangtze River.The experiment result confirms the effectiveness and efficiency of the proposed method.