Social media services allow users to share and annotate their resources freely with keywords or tags that have valuable information to support organizing or searching uploaded images or videos. Tag recommendation is used to encourage users to annotate their resources. Recommending tags of images to users not only depends on user preference but also strongly relies on the contents of images. In this paper, we propose a method for image tag recommendation using both image visual features and user past tagging behaviours by combining convolutional neural networks (CNN), which are widely used and have achieved high performance in image classification and recognition, and factorization machines (FM), since factorization models are the state-of-the-art approach for tag recommendation. Empirically, we demonstrate that learnable features extracted by CNNs can improve up to 7 percent the performance of FMs in image tag recommendation.