Vanishing point estimation is a crucial task in vision-based road detection. This paper presents a new texture-based voting scheme, which enhances both accuracy and speed of vanishing point estimation. In the proposed method, color tensors analysis is adopted to calculate local orientations and color edges. The search space is reduced by optimizing the set of vanishing point candidates and voters. A new strategy based on Bayesian classifier is proposed to select a suitable voting function. The proposed method is evaluated on a benchmark dataset of 4000 images of pedestrian lanes with annotated vanishing points. The experimental results show that it offers an improved accuracy and significantly faster processing time compared with other state-of-the-art methods.