Signer anonymity is an important security requirement for many digital signature schemes due to the need of user privacy in many applications. In this paper, we study signer anonymity for proxy signature which allows a signer to delegate his/her signing right to another user (or proxy). Since the proxy signer is the actual singer in a proxy signature scheme, we are interested in protecting the proxy signer's identity in this paper. However, one potential problem in an anonymous proxy signature scheme is that the proxy signer may abuse the delegated signing right due to the anonymity property of a proxy signature. In this paper, we propose a novel anonymous proxy signature scheme with restricted traceability, which allows the original singer to trace dishonest proxy signers. However, if the proxy signer is honest, then his/her identity is well protected against any user (including the original signer). Our scheme will be useful in many applications such as anonymous authentication protocols and anonymous voting schemes.