Editors’ note: AI has been actively pushing the frontier of applied formal methods for computing systems. In this article, the Specula team wrote about their experience of evaluating LLMs on modeling system code, the basic capability for agentic model checking, using TLA+, a specification language for concurrent and distributed systems. The article is the 7th … Read more
Reproducibility is a cornerstone of scientific progress, yet in systems research, it remains a persistent challenge. Over the past five years, the Artifact Evaluation (AE) process at the European Conference on Computer Systems (EuroSys) has aimed to address this challenge head-on. In this blog post, we reflect on what we have learned from organizing AE … Read more
In early 2023, we received an invitation from the SOSP PC chairs to serve as Artifact Evaluation Co-chairs for SOSP 2023. This was an incredibly exciting opportunity for all of us. As junior members in the systems community, being able to take on a leadership role at our flagship conference is an honor and a … Read more
Previously on the SIGOPS blog, we discussed the current status of artifact evaluation (AE) and how to improve it. At HotOS’23, we took another step in this direction: attendees discussed artifact evaluation in a panel led by Shriram Krishnamurthi (Brown), Margo Seltzer (UBC), and Neeraja Yadwadkar (UT Austin), and organized by Roberta De Viti (MPI-SWS), … Read more