Physical activity (PA) tends to decline during late childhood and adolescence. In Australia, this decline has been shown to particularly occur non-organized PA (e.g., active play, informal sport). This study aimed to explore predictors of non-organized PA at 13y; and decline between 11y and 13y. Data were sourced from the Longitudinal Study of Australian Children. Non-organized PA was measured using time-use diaries. Analyses included Cragg hurdle and binary logistic regressions. The likelihood of non-organized PA at 13y was predicted by PA enjoyment (OR = 1.36, p = 0.007), number of siblings (OR = 1.11, p < 0.001), sex (females) (OR = 0.66, p < 0.001) and home computer use at 11y (OR = 0.98, p = 0.002) (n = 3,193). Females also engaged in less non-organized PA at 13y (β = −77.77, p = 0.048) (n = 3,193). Significant predictors of decline in non-organized PA were socioeconomic position (OR = 0.92, p = 0.047) and sugary drink consumption at 11y (OR = 1.06, p = 0.033) (n = 3,222). Strategies to promote non-organized PA may include targeting females and those of lower socioeconomic position, and focusing on types of PA that may replace the perceived benefits of computer use.