.png){kind=icon}
返回
Promoting open science in test-driven software experiments
.png){kind=icon}
OA
AI
DOI：10.1016/j.jss.2024.111971.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
A core principle of open science is the clear, concise and accessible publication of empirical data, including rawobservational data as well as processed results. However, in empirical software engineering there are no established standards (de jure or de facto) for representing and openingobservations collected in testdriven software experiments - that is, experiments involving the execution of software subjects in controlled scenarios. Execution data is therefore usually represented in ad hoc ways, often making it abstruse and difficult to access without significant manual effort. In this paper we present new data structures designed to address this problem by clearly defining, correlating and representing the stimuli and responses used to execute software subjects in test-driven experiments. To demonstrate their utility, we show how they can be used to promote the repetition, replication and reproduction of experimental evaluations of AI -based code completion tools. We also show how the proposed data structures facilitate the incremental expansion of execution data sets, and thus promote their repurposing for new experiments addressing new research questions.
Keyword:
Software
Engineering
Empirical
Experimentation
Observation
Behavior
Reproducibility
Replication
Data structures
Open science
Large language models
Machine learning
Generative artificial intelligence
Benchmark
Language-to-code
HumanEval
Automation
Measurement
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.1
论文数:
5.5K
被引数:
8.4K
机构
引用论文
Qoala-T: A supervised-learning tool for quality control of FreeSurfer segmented MRI data.png)
NeuroImageOAAI
NeuroImageIF0
OAAI分享收藏
查看更多内容