Perovskite oxide interfaces have been used recently as platforms for demonstrating rich physical properties that are not possessed by their bulk constituents. In this work, we will not only demonstrate rich physical properties at polar perovskite oxide interfaces, but also the tuning of such properties by switching the polarization in PbTiO3/tetragonal BiFeO3 (BFO) two-dimensional ferroelectric heterostructures (HSs). Our first-principles calculations reveal that both magnetism and conductivity can be induced by the discontinuity of the ferroelectric (FE) polarization and the valence states at the interface, and they can also be robustly manipulated by reversing polarization. These properties include the coupling between an interfacial two-dimensional electron gas and magnetism, a transition from half-metal to band insulator or Mott insulator, and the conversion from G-type antiferromagnetic order to local ferromagnetic (FM) ordering, and finally, to the local C-type antiferromagnetic order (C-AFM). Furthermore, by setting different terminations, we can artificially control the local FM ordering that occurs arbitrarily in any one of the polarization directions, and we can even control whether the local FM ordering appears at the interface or at the surface. This novel control can be applied in the fabrication of memory devices for information storage where bits can be written by an electric field (to switch ferroelectric polarization) while reading magnetically. Such a device will drastically reduce energy consumption.