Coarse granular materials such as railway ballast and rockfill are often subjected to three-dimensional (3D) stress conditions including the influence of intermediate principal stress. Modeling the deformation and breakage of these materials under the presence of intermediate principal stress is important for assessing their long-term performance. This paper presents a mathematical model to describe the mechanical behavior of granular materials incorporating the intermediate principal stress and capture particle breakage. The model formulation encompasses interparticle contact planes using a multilaminate mathematical framework based on generalized plasticity and associated critical state concepts. The model that has been calibrated based on recent experimental data on latite basalt, captures the stress-strain and volumetric strain behavior for a range of confining pressures under triaxial compression. This paper also describes the influence of intermediate principal stress on the strength and deformation response of selected granular materials following 3D stress paths. It is evident from the results that the current modeling technique successfully captured the effects of particle breakage, intermediate principal stress, and confining pressure on the shear behavior of various granular assemblies. The results also highlight the influence of intermediate principal stress in reducing the peak deviatoric strength of the material. The model predictions were validated using four independent sets of past experimental data on crushed basalt, limestone, sandstone, and granite aggregates.