The ground-penetrating radar (GPR) has been widely used in many applications. However, the processing and interpretation of the acquired signals remain challenging tasks since an experienced user is required to manage the entire operation. In this paper, we present an automatic classification system to assess railway-ballast conditions. It is based on the extraction of magnitude spectra at salient frequencies and their classification using support vector machines. The system is evaluated on real-world railway GPR data. The experimental results show that the proposed method efficiently represents the GPR signal using a small number of coefficients and achieves a high classification rate when distinguishing GPR signals reflected by ballasts of different conditions.