Effort and Quality of Recovering Requirements-to-Code Traces: Two Exploratory Experiments

Authors: Alexander Egyed, Florian Graf and Paul Grünbacher

Trace links between requirements and code are essential for many software development and maintenance activities. Despite significant advances in traceability research, creating links remains a human-intensive activity and surprisingly little is known about how humans perform basic tracing tasks. We investigate fundamental research questions regarding the effort and quality of recovering traces between requirements and code. Our paper presents two exploratory experiments conducted with 100 subjects (half with industrial experience, the other half without) who recovered trace links for two open source software systems in a controlled environment and cast over 125.000 votes. In the first experiment, subjects recovered trace links between the two systems’ requirements and implementation classes. In the second experiment trace links were established between requirements and implementation methods. In order to assess the validity of the trace links cast by subjects, key developers of the two software systems participated in our research and provided benchmarks. Our study yields surprising observations: trace capture is surprisingly fast and can be done within minutes even for larger classes; the quality of the captured trace links, while good, does not improve with higher trace effort; it is not harder though slightly more expensive to recover the trace links for larger, more complex classes; and, trace capture should be performed by multiple engineers because "hard-to-do" traces differed per subject and were not uniform to certain parts of code. These findings open interesting possibilities for future research.

Download PDF here!