Welcome to RS_c, the central platform for the RecSys community. We provide curated lists of recommender-systems datasets, algorithms, books, conferences and many resources more. Maybe most importantly, we publish the latest recommender-system news. If you want your news to be reported on RS_c, read here.
“[T]he Recommender Systems research community is facing a crisis where a significant number of papers present results that contribute little to collective knowledge […] often because the research lacks the […] evaluation to be properly judged and, hence, to provide meaningful contributions.”
The machine learning community faces similar problems, but in contrast to recommender systems, the machine learning community has some guidelines that potentially alleviate the problem. Particularly noteworthy, in my opinion, are the Tips for Publishing Research Code by ‘Papers with Code’. These tips are now also the official guidelines at the NeurIPS conference, one of the major machine learning conferences (formerly NIPS). Even though NeurIPS is not requiring authors to follow the guidelines, they strongly recommend it.
Frankly, while my group usually publishes its source code on GitHub, we haven’t been following any particular guidelines, and haven’t been too careful about it. Nevertheless, the Tips for Publishing Research Code by ‘papers with code’ are an important step towards more reproducibility, and I believe the recommender-system community should pay more attention to it (including myself).
What do you think? Are such guidelines on publishing code important? Should they be mandatory e.g. for ACM RecSys submissions? Let us know your thoughts in the comments and vote in the poll.