Publications

2026

1. CHASE

   Folklore in Software Engineering: A Definition and Conceptual Foundations

   Eduard Paul Enoiu, Jean Malm, and Gregory Gay

   In Proceedings of the 19th ACM/IEEE International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2026), Rio de Janiero, Brazil, 2026

   Abs PDF

   We explore the concept of folklore within software engineering, drawing from folklore studies to define and characterize narratives, myths, rituals, humor, and informal knowledge that circulate within software development communities. Using a literature review and thematic analysis, we curated exemplar folklore items (e.g., beliefs about where defects occur, the 10x developer legend, and technical debt). We analyzed their narrative form, symbolic meaning, occupational relevance, and links to knowledge areas in software engineering. To ground these concepts in practice, we conducted semi-structured interviews with 12 industrial practitioners in Sweden to explore how such narratives are recognized or transmitted within their daily work and how they affect it. Synthesizing these results, we propose a working definition of software engineering folklore as informally transmitted, traditional, and emergent narratives and heuristics enacted within occupational folk groups that shape identity, values, and collective knowledge. We argue that making the concept of software engineering folklore explicit provides a foundation for subsequent ethnography and folklore studies and for reflective practice that can preserve context-effective heuristics while challenging unhelpful folklore.

2. AST

   From Logs to Lessons: An Exploration of LLM-based Log Summarization for Debugging Automotive Software

   Anton Ekström, Anton Rhedin Stam, Francisco Gomes Oliveira Neto, and 2 more authors

   In Proceedings of the 7th ACM/IEEE International Conference on Automation of Software Test (AST 2026), Rio de Janiero, Brazil, 2026

   Abs PDF

   Identifying where faults occur is an essential part of debugging, yet examining extensive system logs can be slow and mentally demanding, especially in complex software environments. One emerging strategy to enhance log analysis is to employ large language models (LLMs) to distill log information into more manageable summaries that can guide human reasoning during diagnosis. We report on a case study carried out in an automotive setting, where engineers investigated actual failures with and without support from an LLM-based summarization tool. During fault localization sessions where participants analyzed real failure logs, we collected cognitive load measurements, observed their reasoning processes, and gathered feedback on both the LLM-based summarization and the workflow through post-session interviews. Our results indicate that although the use of summaries raised certain cognitive demands, particularly related to mental effort and time pressure, participants experienced less frustration overall and considered the support helpful in focusing their attention. They also expressed a clear interest in being able to shape and refine summaries as their understanding evolved. These findings offer insights into how LLM-generated summaries influence practitioners’ diagnostic work and point toward the need for more adaptive, interactive, and workflow-aware support.

2025

1. STVR

   Exploring the Interaction of Code Coverage and Non-Coverage Objectives in Search-Based Test Generation

   Afonso Fontes, Gregory Gay, and Robert Feldt

   To appear, Software Testing, Verification, and Validation, 2025

   Abs PDF

   Context: Search-based test generation typically targets structural coverage of source code. Past research suggests that targeting coverage alone is insufficient to yield tests that achieve common testing goals (e.g., discovering situations where a class-under-test throws exceptions) or detect faults. A suggested alternative is to perform multi-objective optimization targeting both coverage and additional objectives directly related to the goals of interest. However, it is not fully clear how coverage and goal-based objectives interact during the generation process and what effects this interaction will have on the generated test suites. Objectives: We assess five hypotheses about multi-objective test generation and the relationships between coverage-based and goal-based objectives, focusing on the effects on coverage, goal attainment, fault detection, test suite size, test case length, and the impact of the search budget. Methods: We generate test suites using the EvoSuite framework targeting Branch Coverage, three testing goals—Exception Count, Output Coverage, and Execution Time—and combinations of coverage and goal-based objectives. Results: Targeting multiple objectives does not reduce code coverage, yields no or minor reductions in goal attainment, and detects more faults compared to single-target configurations. It produces larger test suites, but test case length is not increased. The benefits of multi-objective optimization are often more limited than hypothesized in past research, but improved fault detection is still sufficient to recommend multi-objective optimization over targeting coverage or testing goals alone. Conclusion: Our study offers insights and guidance into how coverage and goal-based objectives interact during multi-objective test generation.

2. JSS

   An intelligent test management system for optimizing decision making during software testing

   Albin Lönnfält, Viktor Tu, Gregory Gay, and 2 more authors

   Journal of Systems and Software, 2025

   Abs DOI PDF

   To ensure the proper testing of any software product, it is imperative to cover various functional and non-functional requirements at different testing levels (e.g., unit or integration testing). Ensuring appropriate testing requires making a series of decisions—e.g., assigning features to distinct Continuous Integration (CI) configurations or determining which test specifications to automate. Such decisions are generally made manually and require in-depth domain knowledge. This study introduces, implements, and evaluates ITMOS (Intelligent Test Management Optimization System), an intelligent test management system designed to optimize decision-making during the software testing process. ITMOS efficiently processes new requirements presented in natural language, segregating each requirement into appropriate CI configurations based on predefined quality criteria. Additionally, ITMOS has the capability to suggest a set of test specifications for test automation. The feasibility and potential applicability of the proposed solution were empirically evaluated in an industrial telecommunications project at Ericsson. In this context, ITMOS achieved accurate results for decision-making tasks, exceeding the requirements set by domain experts.

3. Pre-Print

   Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes

   Jingxiong Liu, Ludvig Lemner, Linnea Wahlgren, and 3 more authors

   2025

   Abs HTML PDF

   Much of the cost and effort required during the software testing process is invested in performing test maintenance - the addition, removal, or modification of test cases to keep the test suite in sync with the system-under-test or to otherwise improve its quality. Tool support could reduce the cost - and improve the quality - of test maintenance by automating aspects of the process or by providing guidance and support to developers. In this study, we explore the capabilities and applications of large language models (LLMs) - complex machine learning models adapted to textual analysis - to support test maintenance. We conducted a case study at Ericsson AB where we explore the triggers that indicate the need for test maintenance, the actions that LLMs can take, and the considerations that must be made when deploying LLMs in an industrial setting. We also propose and demonstrate a multi-agent architecture that can predict which tests require maintenance following a change to the source code. Collectively, these contributions advance our theoretical and practical understanding of how LLMs can be deployed to benefit industrial test maintenance processes.

2024

1. ASE

   Integrating Mutation Testing Into Developer Workflow: An Industrial Case Study

   Stefan Alexander Heijningen, Theo Wiik, Francisco Gomes de Oliveira Neto, and 3 more authors

   In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, Sacramento, CA, USA, 2024

   Abs DOI PDF

   Mutation testing is a potentially effective method to assess test suite adequacy. Researchers have made mutation testing more computationally efficient, and new frameworks are regularly emerging. However, there is still limited adoption of mutation testing in industry. We hypothesize that such adoption is hindered by a lack of guidance on how to effectively and efficiently utilize mutation testing in a development workflow. To that end, we have conducted an industrial case study exploring the technical challenges of implementing mutation testing in continuous integration, what information from mutation testing is of use to developers, and how that information should be presented (in textual and visual form). Our results reveal five technical challenges of integrating mutation testing and nine key findings regarding how the results of mutation testing are used and presented. We also offer a dashboard to visualize mutation testing results, as well as 16 recommendations for making effective use of mutation testing in practice1.