This paper proposes the application of an improved extended active observer (IEAOB) based teleoperation controller for both master and slave manipulators. The proposed approach successfully deals with all the major problems in teleoperation systems associated with force/position tracking, external force estimation, communication delay compensation, and suppression of disturbances such as robot inertial parameter variation, unmodeled dynamics, measurement noise, and frictions in joints. As an enhancement of the extended active observer (EAOB) reported earlier, the proposed IEAOB relies on an accurate model of system rigid-body dynamics and measured motion to estimate external torque and friction torque, which are both subsequently used in the design of master and slave controllers. Stability of the entire teleoperation system is verified through stability analysis. In order to demonstrate effectiveness of the approach, the IEAOB-based teleoperation controller is applied to a pair of 3-DOF Phantom haptic devices connected via a communication channel with time-varying delays. The performance of the proposed algorithm is compared with the Nicosia observer-based teleoperation controller. The experimental results show that the proposed approach is superior to the Nicosia observer-based approach for a time variant non-linear teleoperation system in tracking force and position and suppressing disturbance in the presence of time varying delays. Meanwhile, another contact motion experiment with a force sensor is conducted on Phantom Omni haptic device; the result demonstrates the force estimation feasibility of IEAOB in practical applications.