The shear strength of unsaturated soils, a research hotspot in geotechnical engineering, has great guiding significance for geotechnical engineering design. Although kinds of calculation models for the shear strength of unsaturated soil have been put forward by predecessors, there is still need for new models to extensively consider the nonlinear variation of shear strength, particularly for the nonlinear effect of the net normal stress on the shear strength of unsaturated soil. Here, the shear strength of unsaturated soils is explored to study the nonlinear effects of net normal stress with the introduction of a general nonlinear Mohr-Coulomb (M-C) strength criterion, and the relationship between the matric suction (or suction stress) and degree of saturation (DOS) constructed by the soil-water characteristics curve (SWCC) of van Genuchten is also applied for unsaturated soil. Then, two calculation models (i.e., an envelope shell model and an effective stress model) are established for the shear strength of unsaturated soils under the nonlinear strength theory. In these two models, the curve of the shear strength of unsaturated soils versus the net normal stress exhibits a tendency to gently. Moreover, the proposed formulas have flexibility and convenience with five parameters (for the effective stress model) or six parameters (for the envelope shell model), which are from the M-C strength parameters of the saturated soil and fitting parameters of SWCC of van Genuchten. Thereafter, by comparison with the classical theory of the shear strength of unsaturated soils from some actual cases, the rationality and accuracy of the present models were verified.