Camera trapping to study wildlife allows for data collection, without the need to capture animals. Traditionally, camera traps have been used to target larger terrestrial mammal species, though recently novel methods and adjustments in procedures have meant camera traps can be used to study small mammals. The selfie trap (a camera trapping method) may present robust sampling and ecological study of small mammals. This study aimed to evaluate the selfie trap method in terms of its ability to detect species and estimate population density. To address this aim, standard small mammal live trapping was undertaken, immediately followed by camera trapping using the selfie trap. Both methods were set to target the arboreal sugar glider (Petaurus breviceps) and semi-arboreal brown antechinus (Antechinus stuartii). The more ground-dwelling bush rat (Rattus fuscipes) was also live trapped and recorded on camera. Across four survey areas, the probability of detection for each of the three species was higher for selfie traps than for live trapping. Spatially explicit capture-recapture models showed that selfie traps were superior at estimating density for brown antechinus and sugar gliders, when compared to simulated live trapping data. Hit rates (number of videos per various time intervals) were correlated with abundance. When correlating various hit rate intervals with abundance, the use of 10-min hit rate was best for predicting sugar glider abundance (R2 = 0.94). The abundance of brown antechinus was estimated from selfie traps using a 24-h hit rate as a predictor (R2 = 0.85). For sugar gliders, the selfie trap can replace live trapping as individuals can be identified through their unique facial stripes and natural ear scars, and thus used in capture-recapture analysis. This method may be useful for monitoring the abundance of other small mammal species that can also be individually recognized from photographs.