Artifact evaluation, theory and practice

Previously on the SIGOPS blog, we discussed the current status of artifact evaluation (AE) and how to improve it. At HotOS’23, we took another step in this direction: attendees discussed artifact evaluation in a panel led by Shriram Krishnamurthi (Brown), Margo Seltzer (UBC), and Neeraja Yadwadkar (UT Austin), and organized by Roberta De Viti (MPI-SWS), … Read more

Systems Trivia: Behind The Scenes

Editor’s note: Since HotOS ’21, Systems Trivia has become systems researchers’ favorite activity – it’s always incredibly fun and educational. Systems Trivia was created by Roberta De Viti and Vaastav Anand who have then been successfully running it at HotOS ’21, SOSP ’21, and HotOS ’23. In this article, We invite Roberta and Vaastav to … Read more

Artifact evaluation, present and future

Science should be transparent and reproducible. In computer science, this means code and data should be publicly available and anyone should be able to validate a paper’s claims given reasonable hardware. Dedicated scientists have made their code and data public for decades, but until recently, this was done in a decentralized fashion. Groups had their … Read more