In this study, complex variable theory was employed to study the stress distribution for rock mass containing a hole with complex shapes. The method of mapping function calculation using optimization technique was investigated and improved. The shape-related mapping function was defined for parameter analysis. A new objective function for mapping function determination was developed to calculate related parameters without the consideration of restrictive conditions between boundary points in z-plane and ��-plane, which significantly simplified the calculation process. In addition, the mapping function for planes containing a rotated hole was deducted to simplify analytical stress solution process for holes under complex stress conditions. Finally, for six kinds of holes with typical shapes in practical rock engineering, the mapping functions were determined, and the external analytical stress distribution were further presented. The analytical solutions combined with numerical and experimental results show that holes with arched boundaries suffer less stress concentration compared with those with corners. Hole shape mainly affects the stability of rock containing a hole via its influence on stress concentration extent around the hole.