Authentication refers to the process of confirming the identity of the authenticating entity. The use of passwords for user authentication has become ubiquitous in our everyday lives. Despite its wide-spread usage, password-based authentication has numerous deficiencies. For instance, password theft is becoming a common occurrence due to a variety of security problems associated with passwords.As such, many organizations are moving towards adopting alternative solutions like one-time passwords, which are only valid for a single session. Nevertheless, various one-time password schemes also suffer from a number of drawbacks in terms of their method of generation or delivery. In this chapter, we present a challenge-response visual one-time password authentication scheme that is to be used in conjunction with the camera on a mobile device. The main feature of the proposed scheme is to allow the server to send a challenge over a public channel for a user to obtain a session key while safeguarding the user’s long-term secret key. We present the authentication protocol, its security analysis, the various design considerations and the advantages provided by our system.