This work concerns the receiver design for light emitting diode (LED) communications, where the LED nonlinearity can severely degrade the system performance. The LED nonlinearity makes high speed LED communications more challenging when it is time-varying (e.g., due to temperature drifting) and/or combined with time-varying channels (due to the relative movement between the transmitter and the receiver). In this work, we use adaptive neural network techniques to address this issue. We first propose a new adaptive extreme learning machine (AELM) with a variable forgetting factor for adaptive learning in dynamic scenarios. Then, an AELM based (turbo) receiver is designed to handle the time-varying LED nonlinearity and memory effects jointly. It is demonstrated that the proposed AELM based receiver can efficiently mitigate the dynamic nonlinearity and memory effects, and outperform the state-of-the-art adaptive techniques significantly.