The reflection and transmission properties are the key evaluating indicators of the breakwater. We carry out the physical experiments to study the influence of wave nonlinearity on the reflection and transmission of a sloping mound breakwater under the irregular waves，analyze the incident and reflected waves in front of the breakwater using a time-domain separation method based on the wavelet transform，obtain the reflection and transmission coefficients of the breakwater，compare the experimental results with the transmission coefficient of the existing sloping breakwater，and discuss the effect of wave nonlinearity and water level on the reflection and transmission. The results show that the reflection coefficient of breakwater increases with the enhancement of wave nonlinearity and water level. The transmission coefficient decreases with the enhancement of wave nonlinearity but increases with rising of water level. Almost all the existing wave transmission coefficient prediction formulae are underestimated the actual wave transmission，especially for a larger water level. This breakwater has a good performance for reflecting low-frequency waves.