Samples with one through three passes with 100% overlap were created using friction stir processing (FSP) in order to locally modify the microstructure and mechanical properties of a cold-rolled Al-5083 alloy. A constant traverse speed and two different rotational speeds were used for processing. The results indicated that single-pass FSP caused dynamic recrystallization in the stir zone, leading to equiaxed grains with high angle grain boundaries. The grain size increased with increasing the rotational speed. The product of ultimate tensile strength and total elongation of the stir zone was significantly enhanced compared with that of the base metal. During the annealing heat treatment, abnormal grain growth (AGG) occurred throughout the stir zone when a low rotational speed was adopted. The stir zone produced with a high rotational speed was found to be more resistant to AGG. The application of multiple process passes did not change the grain size and mechanical properties of the stir zone significantly. The expanding of AGG in the stir zone was inhibited as the number of passes was increased.