Oligomers of the amyloid-β (Aβ) peptide have been implicated in the neurotoxicity associated with Alzheimer’s disease. We have used single-molecule techniques to examine quantitatively the cellular effects of adding well characterized Aβ oligomers to primary hippocampal cells and hence determine the initial pathway of damage. We found that even picomolar concentrations of Aβ (1–40) and Aβ (1–42) oligomers can, within minutes of addition, increase the levels of intracellular calcium in astrocytes but not in neurons, and this effect is saturated at a concentration of about 10 nM of oligomers. Both Aβ (1–40) and Aβ (1–42) oligomers have comparable effects. The rise in intracellular calcium is followed by an increase in the rate of ROS production by NADPH oxidase in both neurons and astrocytes. The increase in ROS production then triggers caspase-3 activation resulting in the inhibition of long-term potentiation. Our quantitative approach also reveals that only a small fraction of the oligomers are damaging and that an individual rare oligomer binding to an astrocyte can initiate the aforementioned cascade of responses, making it unlikely to be due to any specific interaction. Preincubating the Aβ oligomers with an extracellular chaperone, clusterin, sequesters the oligomers in long-lived complexes and inhibits all of the physiological damage, even at a ratio of 100:1, total Aβ to clusterin. To explain how Aβ oligomers are so damaging but that it takes decades to develop Alzheimer’s disease, we suggest a model for disease progression where small amounts of neuronal damage from individual unsequestered oligomers can accumulate over time leading to widespread tissue-level dysfunction.