This study aimed to evaluate the absorbed doses received by the organs at risk (OARs) following Iridium-192 (192Ir) high dose rate (HDR) brachytherapy of the left breast. The MIRD5 adult female anthropomorphic phantom, readily available in the Geant4 Monte Carlo package was used. However, the left breast was modified from 195 to 145 cm3, to represent a breast following lumpectomy. Left breast was chosen due to its higher cancer occurrence than the right breast. The HDR sources were constructed with an outer cylindrical dimension of 4.5 mm (length) × 0.9 mm (diameter). Various influencing parameters were studied, i.e. catheter arrangement (single versus dual plane), source inter-dwell distances (5 versus 10 mm), and different radionuclides, i.e. Cobalt-60 (60Co) and 192Ir, by delivering a total treatment dose of 32 Gy to the left breast. Absorbed doses to the OARs (e.g. left lung, heart, right breast, spleen, etc.) were then evaluated. A maximum left lung dose of 1.5 Gy was recorded, while doses to the other OARs were all below 1 Gy. The treatment using dual plane catheter arrangement contributed to a slightly higher dose to the OARs, despite equal dose to the breast. There was no dose difference between different inter-dwell distances used in this study. 60Co resulted in a slightly higher left lung dose than that of 192Ir, while the results were the opposite for the other OARs. HDR brachytherapy allows high dose to be delivered to the breast within a short period of time, with minimal absorbed doses to the OARs.