To date, a number of photon up-conversion (UC) systems with high quantum efficiencies have been demonstrated, including at non-concentrated solar illumination intensities. These are, however, yet to be successfully combined with photovoltaic devices in order to provide substantial increases in photocurrent, as compared with devices without UC. In the present work, we perform simple calculations, based on the optical properties of an UC + solar cell system, specifically a dye-sensitized solar cell and sensitized-triplet-triplet annihilation based UC system, in line with our previous experimentally realized proof-of-concept devices. This model is then used to more specifically identify the limitations and losses, thereby providing a pathway toward improvements, based on either real or hypothetical materials. This model highlights that only a fraction of the benefit may be realized without substantial improvements being made in terms of various transmission losses, coupled with the fact that UC emission typically scales quadratically in this flux regime. The model presented can help identify where research efforts should be focused as well as where there may be little utility, with these lessons also being valuable in the future as even more efficient UC systems are developed. Furthermore, a spreadsheet is provided in the supplementary material to assist other researchers in their efforts.