Road maintenance requires local city governments to dedicate a substantial amount of funds in finding and repairing damaged traffic marks and pavements. In developed cities, the total road length is so large that the cost becomes unreasonably high. In this paper, we propose a method of sensing damaged traffic marks from images captured by a camera mounted to a car, for the purpose of reducing road maintenance cost. In particular, we utilized convolutional neural networks (CNN), as well as linear support vector machines (SVM) and Random Forest, in developing a system of damage detection. The experiments used thousands of images captured in the wild and showed that the method can detect damages using CNN with 93% accuracy, at maximum, and at reasonable speed (55 images per second).