Nowadays, since multimedia information has been extensively growing from a variety of sources, such photos from social networks, unstructured text from different websites, or raw video feed from digital sensors, multimedia recommender system has been emerging as a tool to help users choose which multimedia objects might be interesting for them. However, given the complexity of multimedia, it is still challenging to provide effective recommendations, and research so far could only address limited aspects. Therefore, in this paper we propose a set of research challenges, which can be used to implicate the future research directions for multimedia recommender systems. For each research challenge, we have also provided the insights to explain which aspects are worth further investigation.