Genomic selection (GS) is a statistical and breeding methodology designed to improve genetic gain. It has proven to be successful in animal breeding; however, key points of difference have not been fully considered in the transfer of GS from animal to plant breeding. In plant breeding, individuals (varieties) are typically evaluated across a number of locations in multiple years (environments) in formally designed comparative experiments, called multi-environment trials (METs). The design structure of individual trials can be complex and needs to be modelled appropriately. Another key feature of MET data sets is the presence of variety by environment interaction (VEI), that is the differential response of varieties to a change in environment. In this paper, a single-step factor analytic linear mixed model is developed for plant breeding MET data sets that incorporates molecular marker data, appropriately accommodates non-genetic sources of variation within trials and models VEI. A recently developed set of selection tools, which are natural derivatives of factor analytic models, are used to facilitate GS for a motivating data set from an Australian plant breeding company. The power and versatility of these tools is demonstrated for the variety by environment and marker by environment effects.