Software and Computational Systems Lab

Publications about Dataset

1. Dirk Beyer, Lars Grunske, Matthias Kettl, Marian Lingsch-Rosenfeld, and Moeketsi Raselimo. P3: A Dataset of Partial Program Patches. In Proc. MSR, 2024. ACM. doi:10.1145/3643991.3644889 Keyword(s): Partial Fix, Dataset, Mining Funding: DFG-IDEFIX Publisher's Version PDF Supplement
   Artifact(s)

   1. doi:10.5281/zenodo.10319627
   Abstract

   Identifying and fixing bugs in programs remains a challenge and is one of the most time-consuming tasks in software development. But even after a bug is identified, and a fix has been proposed by a developer or tool, it is not uncommon that the fix is incomplete and does not cover all possible inputs that trigger the bug. This can happen quite often and leads to re-opened issues and inefficiencies. In this paper, we introduce P3, a curated dataset composed of in- complete fixes. Each entry in the set contains a series of commits fixing the same underlying issue, where multiple of the intermediate commits are incomplete fixes. These are sourced from real-world open-source C projects. The selection process involves both auto- mated and manual stages. Initially, we employ heuristics to identify potential partial fixes from repositories, subsequently we validate them through meticulous manual inspection. This process ensures the accuracy and reliability of our curated dataset. We envision that the dataset will support researchers while investigating par- tial fixes in more detail, allowing them to develop new techniques to detect and fix them.
   BibTeX Entry

   @inproceedings{MSR24, author = {Dirk Beyer and Lars Grunske and Matthias Kettl and Marian Lingsch-Rosenfeld and Moeketsi Raselimo}, title = {P3: A Dataset of Partial Program Patches}, booktitle = {Proc.\ MSR}, pages = {}, year = {2024}, publisher = {ACM}, doi = {10.1145/3643991.3644889}, sha256 = {56954209ffff83bf7c77824a46a4bd2fa4ff52173bea5a1c3fe2b5504f19e6ef}, url = {https://gitlab.com/sosy-lab/research/data/partial-fix-dataset}, pdf = {}, abstract = {Identifying and fixing bugs in programs remains a challenge and is one of the most time-consuming tasks in software development. But even after a bug is identified, and a fix has been proposed by a developer or tool, it is not uncommon that the fix is incomplete and does not cover all possible inputs that trigger the bug. This can happen quite often and leads to re-opened issues and inefficiencies. In this paper, we introduce P3, a curated dataset composed of in- complete fixes. Each entry in the set contains a series of commits fixing the same underlying issue, where multiple of the intermediate commits are incomplete fixes. These are sourced from real-world open-source C projects. The selection process involves both auto- mated and manual stages. Initially, we employ heuristics to identify potential partial fixes from repositories, subsequently we validate them through meticulous manual inspection. This process ensures the accuracy and reliability of our curated dataset. We envision that the dataset will support researchers while investigating par- tial fixes in more detail, allowing them to develop new techniques to detect and fix them.}, keyword = {Partial Fix, Dataset, Mining}, annote = {}, artifact = {10.5281/zenodo.10319627}, funding = {DFG-IDEFIX}, }

2. Po-Chun Chien, Nian-Ze Lee, and Zhengyang Lu. Performance Dataset for Hardware Model Checking on Btor2 Benchmarks. arXiv/CoRR, May 2025. Keyword(s): Btor2, Benchmarking, Benchmarks, Dataset PDF Supplement
   Abstract

   This technical report presents a performance evaluation of several hardware model-checking tools on a collection of benchmark tasks in the Btor2 format. The resulting dataset is intended to support machine-learning research for hardware model checking, particularly in areas such as algorithm selection, performance prediction, and automated tool configuration. It has been used, for example, in the development and evaluation of Btor2-Select, a machine-learning-based algorithm selection framework for hardware model checking. To construct the dataset, we benchmarked a diverse set of model-checking tools and algorithmic configurations. Each verification engine was evaluated on a common set of Btor2 tasks and the performance measurements, including CPU time, wall time, and memory usage, were collected. All data, including scripts and files required to reproduce the experiments, are publicly available at: https://gitlab.com/sosy-lab/research/data/perf-eval-hwmc.
   BibTeX Entry

   @techreport{HWMCPerfDataset-TR, author = {Po-Chun Chien and Nian-Ze Lee and Zhengyang Lu}, title = {Performance Dataset for Hardware Model Checking on Btor2 Benchmarks}, number = {}, year = {2025}, doi = {}, url = {https://gitlab.com/sosy-lab/research/data/perf-eval-hwmc}, pdf = {https://www.sosy-lab.org/research/pub/2025-TR.Performance_Dataset_for_Hardware_Model_Checking_on_Btor2_Benchmarks.pdf}, abstract = {This technical report presents a performance evaluation of several hardware model-checking tools on a collection of benchmark tasks in the Btor2 format. The resulting dataset is intended to support machine-learning research for hardware model checking, particularly in areas such as algorithm selection, performance prediction, and automated tool configuration. It has been used, for example, in the development and evaluation of Btor2-Select, a machine-learning-based algorithm selection framework for hardware model checking. To construct the dataset, we benchmarked a diverse set of model-checking tools and algorithmic configurations. Each verification engine was evaluated on a common set of Btor2 tasks and the performance measurements, including CPU time, wall time, and memory usage, were collected. All data, including scripts and files required to reproduce the experiments, are publicly available at: <a href="https://gitlab.com/sosy-lab/research/data/perf-eval-hwmc">https://gitlab.com/sosy-lab/research/data/perf-eval-hwmc</a>.}, keyword = {Btor2, Benchmarking, Benchmarks, Dataset}, artifact = {}, institution = {arXiv/CoRR}, month = {May}, }