By the simulated rainfall experiments, we identified the effects of different vertical structures on runoff and sediment production processes in the community of Artemisia capillaries. The results showed that the reduction effects of runoff and sediment yield of the multi-layer vertical structure (Artemisia capillaries + Setaria viridis (L.) Beauv + biological crusts) and double-layer vertical structure (Artemisia capillaries + biological crusts) were significantly greater than those of the single-layer vertical structure (Artemisia capillaris) under the same condition of vegetation coverage. Compared with the single-layer vertical structure, the other two structures could reduce soil erosion significantly. In additional, runoff was reduced by 22.6% and 40.5% over the double-layer vertical structure and multi-layer vertical structure, respectively, and the sediment yield was reduced by 73.5% and 86.6%, respectively. The cumulative sediment yield could be expressed as a power function of the cumulative production flow (R2＞0.98), showing that the cumulative sediment yield increased with the augment of the runoff production, and the degree of the enhancement decreased when the community structure became more complex. The results also showed that plant roots could improve soil structure by increasing rainfall infiltration and reducing overland runoff, which was the key factor influencing the runoff and sediment yield processes. All results indicated that the ability of community to prevent soil erosion could be improved with complicated vertical structure and components. In conclusion, in order to provide the theory basis for grassland vegetation restoration and structural configuration optimization for the Loess Plateau, we have to study further the vertical structure of grassland community and reveal the mechanisms of the soil and water conservation.