Solid electrolyte has good mechanical strength and can effectively inhibit the growth of lithium dendrite, so it has a good application prospect in lithium batteries, especially solid polymer electrolyte (SPE), such as PEO. Due to the stability of lithium metal and the excellent dissociation ability of lithium salts, PEO and its derivatives are the mainstream SPE materials. PEO performs best at high temperatures. However, the flammability and fluidity of PEO at high temperatures may pose safety concerns. In this study, sodium alginate (SA) was introduced into PEO to improve the mechanical properties of SPE and the safety of the battery through the nanofiber membrane structure. The PEO/SA composite solid electrolyte has excellent flame retardance (LOI = 28.6), good thermal stability (>120 °C) and wide electrochemical window (>4.6 V). This will help further research on lithium batteries with high energy density and voltage.