Recent & Upcoming Events

This event is hosted by the TU Delft Programming Languages Group as part of their seminar. Abstract: This talk presents a tool for genetic program repair utilizing the rich features provided by Haskell and the available frameworks. It covers basics of program repair, genetic search and typed holes. We then look at our novel approach of type-based program repair using properties and present some of the (mixed) results we achieved. In general, the patches are cool but often too fine-grained, but we hope that they can help especially for classroom settings.

Abstract: Code completion is an essential feature of IDEs, yet current autocompleters are restricted to either grammar-based or NLP-based single token completions. Both approaches have significant drawbacks: grammar-based autocompletion is very restricted in dynamically-typed language scenarios, whereas NLP-based autocompletion struggles to understand the semantics of the programming language, giving suggestions that ignore a developer’s context. In this work, we present CodeFill, a language model for autocompletion that combines structure and naming information.

Abstract: all Graphs are a rich data source and built the foundation for advanced static analysis that can, for example, detect security vulnerability or dead code. Call Graph generation is usually considered to be a full program analysis: not just the program, but also all its dependencies are processed together. Unfortunately, this is very expensive and makes it hard to run these static analyses in build systems that are typically limited in resources.

Abstract Code mixes natural language in identifier names, comments, and stylistic choices (ordering and typesetting) with a formal language that defines a computation. The snippets in each language form a communication channel. Developers read both channels; a CPU processes only the formal channel. These two channels interact and constrain each other. The theory of dual channel constraints elucidates these interactions and points to their exploitation. One prominent application is probabilistic type inference.

Ask Maurício for the Zoom link.

Abstract: TBD

Ask Maurício for the Zoom link.

Ask Maurício for the Zoom link.

Ask Maurício for the Zoom link.

For my Go-No-Go I want to share with you what I worked on last year; I show the idea behind my first paper Lampion, my current work with Haskell Program Repair and our concept for a new java bug data-set. You will find the Teams-Invite on Mattermost

Ask Maurício for the Zoom link.

Abstract: The success of Deep Neural Networks with image classification prompted researchers to explore the applications of Deep Learning in Medical Imaging and Medical Image Analysis (MIA). Deep Neural Networks have sufficiently demonstrated their capabilities of performing MIA tasks tirelessly and with fewer errors as opposed to their human counterpart. However the challenge of training neural networks using sensitive medical data, without violating the privacy of patients remains an active field of research.

In the last session we looked at how Machine Learning is being used in Software Engineering (specifically, to predict flakiness of tests). In this session, we explore the opposite perspective ie. how is Software Engineering different for ML systems compared to traditional software? The paper for discussion is “Software Engineering Challenges of Deep Learning”.

Abstract: In this talk I will go over how we can use Emacs (the mighty text editor) and org-mode (markdown’s big brother) to craft a research workflow. We will look at how we can leverage the power of Emacs and org-mode to capture, store, search and retrieve research data, all in plain text! The talk will also touch upon how org-mode can be used as an environment for literate programming and reproducible research.

A demonstration showing the reliability of serving users recommendations with tradeoff for large music collections, by leveraging diverse Recommender Systems and Evolutionary Algorithms. To sign up for the event you can either fill in this form (https://forms.gle/icuLgPD8gn6Ej19i9) or email s.p.bobde@student.tudelft.nl for the link.

Abstract: Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver’s gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time—where each individual key-point may be critical in the targeted application—and images can vary a great deal according to many factors.

While SQL engines are now capable of detecting a large number of syntactic mistakes, most often semantic errors are not detected, which can lead to serious performance issues or even security vulnerabilities being introduced in the system. This thesis proposes a set of 25 validated heuristics together with a new rule-based static analysis tool for detecting the most common types of semantic bugs in SQL queries, based on evidence from previous research.

Abstract: In this talk, I will first explain some of my previous studies as a Ph.D. student. In these studies, we introduced multiple techniques to improve automated test generation using various available information sources, such as the source code and manually-written tests. I will also talk about some of the lessons that I learned in my journey as a Ph.D. student. Finally, I briefly talk about the new topics that I am currently working on as a PostDoc.

Dear all, Join us on Zoom for the next SERG Reading Club 📖📚! In this session, we read and review the paper “FlakeFlagger: Predicting Flakiness Without Rerunning Tests”, which is presented at the ICSE'21. Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

As we all know, software testing is a critical part of the software engineering process. However, testing comes with its own set of challenges and problems. To this end, we have developed TestKnight, a plugin for IntelliJ IDEA that helps developers test their code. Some of the features provided by TestKnight are: Quick test method duplication and adjustment Testing checklist generation based on structural and black-box testing Assertion suggestion Visualization of coverage changes TestKnight was developed as part of the CSE2000 Software Project course.

Abstract: In this talk, we will introduce SynTest - A framework for generating synthetic tests for programs based on JavaScript. One of the main use cases we have been working on lately, is smart contract testing for Solidity. It is very important for smart contracts to be thouroughly tested since after these contracts are deployed, it is impossible to change or update them. Hence any bugs or security exploits in the code will be in the contract for the duration of its lifetime.

The most common reason for Continuous Integration (CI) build failures is failing tests. When a build fails, a developer often has to scroll through hundreds to thousands of log lines to find which test is failing and why. Finding the issue is a tedious process that relies on a developer’s experience and increases the cost of software testing. Providing CI build test results with additional context in the developer’s local development environment could help solve failing tests more quickly.

Dear all, Join us on Zoom for the next SERG Reading Club 📖📚! We’re reading “Towards a Model of Testers’ Cognitive Processes: Software Testing as a Problem Solving Approach” by Eduard Enoiu, Gerald Tukseferi and Robert Feldt. → https://ieeexplore-ieee-org.tudelft.idm.oclc.org/stamp/stamp.jsp?arnumber=9282711 ← Caro will give a short introduction to the paper at the beginning, so feel free to join the discussion even if you haven’t read the paper! 😊 See you on Friday 👋🏼

Abstract: Researchers and practitioners have designed and implemented various automated test case generators to support effective software testing. Such generators exist for various languages (e.g., Java, C#, or Python) and for various platforms (e.g., desktop, web, or mobile applications). Such generators exhibit varying effectiveness and efficiency, depending on the testing goals they aim to satisfy (e.g., unit-testing of libraries vs. system-testing of entire applications) and the underlying techniques they implement. In this context, practitioners need to be able to compare different generators to identify the most suited one for their requirements, while researchers seek to identify future research directions.

Felienne Hermans will talk about cognitive refactoring. Felienne is an Associate Professor at Leiden University, and the author of The Programmer’s Brain. Hyrum Wright will talk about large-scale refactoring. Hyrum is a staff software engineer at Google and one of the authors of the Software Engineering at Google, also known as the Flamingo book.

Dear all, Join us on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session has to still be decided on 🙂 Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Diomidis Spinellis will talk about CScout, the C refactoring browser: Despite its maturity and popularity, the C programming language still lacks tool support for reliably performing even simple refactoring, browsing, or analysis operations. This is primarily due to identifier scope complications introduced by the C preprocessor. The CScout refactoring browser analyses complete program families by tagging the original identifiers with their precise location and classifying them into equivalence classes orthogonal to the C language’s namespace and scope extents.

Dear all, Join us on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session has to still be decided on 🙂 Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Type4Py: Deep Similarity Learning-Based Type Inference for Python (Amir Mir) Dynamic languages, such as Python and Javascript, trade static typing for developer flexibility. While this allegedly enables greater productivity, lack of static typing can cause runtime exceptions, type inconsistencies, and is a major factor for weak IDE support. To alleviate these issues, PEP 484 introduced optional type annotations for Python. As retrofitting types to existing codebases is error-prone and laborious, learning-based approaches have been proposed to enable automatic type annotations based on existing, partially annotated codebases.

Dear all, Join us on May the 7th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reading the chapter “Code Review” from “Software Engineering at Google”. Link: https://learning-oreilly-com.tudelft.idm.oclc.org/library/view/software-engineering-at/9781492082781/ch18.html#build_systems_and_build_philosophy Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Dear all, Join us on April the 30th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reading: Utilizing a Journal Club to Build Research Competencies in a Cross-Disciplinary Environment Some quotes to give an impression: “The goal of this process-focused field note is to detail the steps taken to design and implement the pilot of a cross-disciplinary research-trainee-led journal club within a hospital-based research institute.”

Abstract: Quantum Software Development is the process of conceiving, specifying, designing, programming, documenting, and testing executable quantum programs that are meant to run on practical quantum hardware. Even though Quantum Software Development research has gained traction over the years, it is still mainly focused on problem analysis, language design, and implementation. Software testing, which is the process of executing a program or application with the intent of finding failures, and verifying that the software product is fit for use is yet to receive substantial attention in quantum software development.

Dear all, Join us on April the 23th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reading the chapter “Build Systems and Build Philosophy” from “Software Engineering at Google”. Link: https://learning-oreilly-com.tudelft.idm.oclc.org/library/view/software-engineering-at/9781492082781/ch18.html#build_systems_and_build_philosophy Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Summary Software testing plays a crucial role in software development to improve the software’s consistency and performance. Since software testing activities demand considerable effort in the development process, many automated techniques have been introduced to aid de- velopers and testers in various testing phases, thereby reducing the costs related to these tasks. One category of these automated approaches seeks to generate software tests auto- matically using different strategies. One of the successful strategies, which is applied to industrial cases, uses metaheuristic search-based approaches for test generation automa- tion.

Dear all, Join us on April the 16th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reading “Socio-Technical Grounded Theory for Software Engineering” by Rashina Hoda. You can find the pre-print here: https://arxiv.org/pdf/2103.14235.pdf Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

An Exploratory Study of Log Placement Recommendation in an Enterprise System (Jeanderson Barros Cândido) Logging is a development practice that plays an important role in the operations and monitoring of complex systems. Developers place log statements in the source code and use log data to understand how the system behaves in production. Unfortunately, anticipating where to log during development is challenging. Previous studies show the feasibility of leveraging machine learning to recommend log placement despite the data imbalance since logging is a fraction of the overall code base.

Topic 1: Developer-Friendly Test Amplification: The Interplay between Automatic Generation and Human Exploration (Carolin Brandt) Caro gives some insights on her first PhD project & we can learn from it: Automatically generating test cases for software is an active research topic for many years. However, the powerful tools we create are up until now rarely used in practice. Starting from issues reported by developers in previous studies, we investigate what aspects are important to design test generation approaches that developers appreciate.

Summary Specialized tools, such as IDEs, issue trackers, and code review tools, are an indispensable part of the modern software engineering process. These tools are constantly evolving. Besides enabling tools to support a wider range of technologies and frameworks, we are learning to provide additional features in completely new ways. One prominent stream of innovation in software engineering tools is dedicated to utilizing historical data to enable data-driven features, such as defect prediction engines and recommender systems, which leverage records of prior activity to assist with decision making.

Topic 1: What if(True)? Testing your ML models with metamorphic transformations on Code (Leonhard Applis) What does a deep neural network, such as CodeBERT, really learn? The metrics are State of the Art - but we have no clue why. With our last project Annibale and me looked into ways to alter the test-set to gain more information on what is happening in that beautiful model of yours. We cover everything your heart desires: Machine Learning, Code, Testing and Statistics.

Dear all, Join us on March the 19th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reading the chapter “Code Search” from the “Software Engineering at Google” book. Link: https://learning-oreilly-com.tudelft.idm.oclc.org/library/view/software-engineering-at/9781492082781/ch17.html#code_search Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Architecting for Business as Unusual

Summary Automated testing has become an essential process for improving the quality of software systems. Automated tests can help ensure that production code is robust under many usage conditions and that code meets performance and security needs. Nevertheless, writing effective tests is challenging and, unfortunately, often neglected. In the first part of this dissertation, we summarize and explain why test and production code are not treated with the same care, and we set our goal: we want to discover new techniques and tools to support developers when writing and reviewing code.

Dear all, Join us on March the 12h, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session is from Vivek! 🙂 Head on to the reading club channel in Mattermost to find the pdf. Wanna join & you are not part of our Mattermost? Send an email to c.

Architecting for Operations: Boring is Awesome. Steffan Norberhuis is independent Cloud & DevOps Consultant, and a TU Delft alumnus. He has worked for such companies as ABN AMRO, Independer, and Portbase (Rotterdam).

Dear all, Join us on March the 5th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’ll read one chapter from Software Engineering at Google! “Measuring Engineering Productivity” (https://learning-oreilly-com.tudelft.idm.oclc.org/library/view/software-engineering-at/9781492082781/ch07.html) Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Architecture in Motion: How Adyen Achieved 100x Bert Wolters is Executive Vice President Technology at Adyen, where he started working in 2010. He obtained his MSc degree in Computer Science from Delft University in 2012. Bert will take us through the growth history of Adyen, explaining how Adyen managed to double its volume year upon year.

Unix has evolved for almost five decades, shaping modern operating systems, key software technologies, and development practices. Studying the evolution of this remarkable system from an architectural perspective can provide insights on how to manage the growth of large, complex, and long-lived software systems. Along main Unix releases leading to the FreeBSD lineage we examine core architectural design decisions, the number of features, and code complexity, based on the analysis of source code, reference documentation, and related publications.

Dear all, Join us on February the 26th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session is a work in progress by Mauricio 🙂 We’ll evaluate it against the “Qualitative Surveys (Interview Studies)” Guidelines + the General Standard. Wanna join & you are not part of our Mattermost?

Dear all, Join us on February the 19th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session is from Carolin and about developer-friendly test amplification. 😄 It’s unpublished, so please ask Carolin for the pdf. The standards to evaluate against are the general standards and “Qualitative Surveys (Interview Studies)”.

Dear all, Join us on February the 12th, 14:00 - 15:00 on Zoom for the next SERG Reading Club 📖📚! We’re reviewing one of our internal papers or an exemplar paper against the ACM SIGSOFT Empirical Standards. The paper for this session is from Amir Mir: “Type4Py: Deep Similarity Learning-Based Type Inference for Python” you can find it on arXiv See you there 👋🏼😄 Wanna join & you are not part of our Mattermost?

Abstract: The ACM SIGSOFT Paper and Peer Review Quality Initiative, which I will present in this talk, seeks to increase paper quality, review quality, acceptance rates and consensus around research practices by generating and evolving empirical standards. An Empirical Standard is a brief public document that communicates expectations for a given kind of study. Empirical standards contain lists of specific practices or attributes related to a particular method, e.g., “uses random assignment” (experiment) or “presents clear chain of evidence from interviewee quotations to proposed concepts” (qualitative survey).

Abstract: Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other.

Abstract: Continuous Integration (CI) is a software development practice that enables developers to build reliable software faster. Given its proven benefits, most organizations have started adopting CI and fostered the application of several principles, such as commit often, that reduce conflicts in the team and ensure that the build is continuously executable. However, when facing a deadline or a difficult task, developers can deviate from such best practices and follow anti-patterns, which are ineffective solutions to recurrent problems.

Software development is more and more reliant on external code. This external code is developed, bundled into packages and shared using package repositories like crates.io or npm. Reusing shared code bundles greatly improves development speed but without proper care and knowledge of included external code it can cause issues as well. Over-reliance on simple trivial packages that can be easily implemented locally can cause severe consequences like not being able to build if the package is no longer available.

ABSTRACT: Cloud computing services play an important role in today’s modern society. They enable daily operation and advances in key application domains, from banking to e-commerce, from science to gaming, from governance to education. Combining technology developed since the 1960s (e.g., modes of resource sharing) with new paradigms that could only have emerged in the 2010s (e.g., FaaS), they promise to enable unprecedented efficiency and seamless access to services for many.

TbD

Agile originated as a method to be used by a small number of co-located teams. When the amount of teams and the distance between the teams becomes bigger, issues will start to arise. With the rise of the computer and internet, programming-related work environments have become very large and globally distributed, implying a friction when using agile in these environments. This study aims to explore what challenges occur when using agile in very-large globally distributed environments and explore for practices that can be used to deal with them.

Large scale production distributed systems are difficult to design and test. Correctness must be ensured when processes run asynchronously, at arbitrary rates relative to each other, and in the presence of failures, e.g., process crashes or message losses. These conditions create a huge space of executions that is difficult to explore in a principled way. Current testing techniques focus on systematic or randomized exploration of all executions of an implementation while treating the implemented algorithms as black boxes.

Software maintenance is an important and time consuming task during the software development cycle. Readability of test code is a crucial element for performing programming tasks, such as testing, bug fixing and maintaining code. Hence poorly written tests are difficult to maintain and lose their value to developers. In order to overcome this problem, we need to understand how programmers read test code. Therefore in this paper we conducted an empirical study to analyse the various reading patterns in novices and professionals using a sophisticated eye tracking device.

Dear all, Join us on November the 6th, 13:00 - 14:00 on Zoom for the next SERG Reading Club 📖📚!: “Sampling in Software Engineering Research: A Critical Review and Guidelines” by Sebastian Baltes and Paul Ralph https://empirical-software.engineering/assets/pdf/tosem20-sampling.pdf Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

In this presentation, I’ll discuss the three MSc thesis defenses we have had at Adyen: The thesis of Henk Grent, who worked on automatically inferring constraints in large-scale APIs The thesis of Hendrig Sellik, who worked on a deep learning model that identifies off-by-one mistakes The thesis of Casper Schröder, who worked on a search-based algorithm to improve the modularization of Adyen’s large codebase. I’ll also discuss what our next works will be about.

Dear all, For this Friday we want to dive deeper into industry-academia collaborations. What is the state of collaborations? Do the partners benefit from the joint projects? how could we increase the process of collaborations between academia and industry? Join us on October the 30th, 13:00 - 14:00 on Zoom for the next SERG Reading Club 📖📚!: “Characterizing industry-academia collaborations in software engineering: evidence from 101 projects” by Garousi et al.

Abstract: In 2014, a Microsoft study investigated the sort of questions that data science applied to software engineering should answer. This resulted in 145 questions that developers considered relevant for data scientists to answer, thus providing a research agenda to the community. Fast forward to five years, no further studies investigated whether the questions from the software engineers at Microsoft hold for other software companies, including software-intensive companies with different primary focus (to which we refer as software-defined enterprises).

Dear all, In this week’s reading club we want to take a closer look at the qualitative method of semi-structured interviews. The paper combines experiences from 12 software engineering studies and discusses particularly challenging areas: effort estimation, interview skills, good interaction and appropriate tools and artifacts. If you wanna know more: Join us on Friday, October the 23th, 13:00 - 14:00 on Zoom for the next SERG Reading Club 📖📚!:

Abstract: Test smells attempt to capture design issues in test code that reduce their maintainability. Previous work found such smells to be highly common in automatically generated test-cases, but based this result on specific static detection rules; although these are based on the original definition of “test smells”, a recent empirical study showed that developers perceive these as overly strict and non-representative of the maintainability and quality of test suites. This leads us to investigate how effective such test smell detection tools are on automatically generated test suites.

Monitoring software behaviour is being done in various ways. Log messages are being output by almost any kind of running software system. Therefore, learning how software behaves from doing analysis over log data can lead to new insights about the system. However, the number of log messages in a computer system grow fast, and analysing the log data by hand is a time-consuming job. The objective of this study is to propose and implement a scalable architecture for doing real-time log analysis.

Dear all, Join us on Friday, October the 16th, 13:00 - 14:00 on Zoom for the next SERG Reading Club 📖📚!: “Diversity in software engineering research” by Nagappan, M., Zimmermann, T., & Bird, C. The title could be a bit missleading, so let me include the abstract 😉: One of the goals of software engineering research is to achieve generality: Are the phenomena found in a few projects reflective of others?

Dear all, Join us on October the 9th, 13:00 - 14:00 on Zoom for the next SERG Reading Club 📖📚!: “The Nature of Theory in Information Systems” by Shirley Gregor (https://bit.ly/2GbWgdA) Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Mistakes in binary conditions are a source of error in many software systems. They happen when developers use < or > instead of <= or >=. These boundary mistakes are hard to find for developers and pose a manual labor-intensive work. While researches have been proposing solutions to identify errors in boundary conditions, the problem remains a challenge. Therefore it is necessary to research and develop new techniques to cope with such errors.

Dear all, In next weeks' reading club we want to put all that theoretical theory discussion into practice by looking at a paper that actually proposes a theory in software engineering 📑📚 Join us on September the 18th, 13:00 - 14:00 on Zoom to discuss an internal paper proposing a theory 🙂

Web APIs can have constraints on parameters, such that not all parameters are either always required or always optional. Sometimes the presence or value of one parameter could cause another parameter to be required. Additionally, parameters could have restrictions on what kinds of values are valid. We refer to these as inter-parameter and single-parameter constraints respectively. Having a clear overview of the constraints can help API consumers to integrate without the need for additional support and with fewer integration faults.

Evolutionary-based crash reproduction techniques aid developers in their debugging practices by generating a test case that reproduces a crash given its stack trace. In these techniques, the search process is typically guided by a single search objective called Crash Distance. Previous studies have shown that current approaches could only reproduce a limited number of crashes due to a lack of diversity in the population during the search. In this talk, I will present our recent work on the redesign of the search-objectives for crash reproduction by (i) applying Multi-Objectivization using Helper-Objectives (MO-HO) on crash reproduction.

Quality of code structure often degrades while a codebase grows. Counteracting this process or remedying its effects by improving the structure takes lots of effort. Software Modularization is a research topic that has tried to reduce this effort, by developing approaches that can find change suggestions that improve the code structure from the perspective of certain metrics. This topic has been researched for around 25 years and many approaches have been introduced and applied.

Online controlled experimentation (OCE), also called A/B testing, is an often used tool in industry to determine if deploying changes into production is the right decision to make. Running experiments has shown an immense impact to the revenue of companies in industry, however this type of experimentation comes with a lot of pitfalls, some which can invalidate the entire experiment. As more organizations start relying on online experimentation, relying only on experts in OCE’s becomes infeasible and creating guidance on the platform where the experiments take becomes a mandatory item.

Dear all, We’re starting up the reading club again :) Next Wednesday we’ll be talking about empirical software engineering and building theories 🧑‍🏫🧑‍💻 Join us on September the 11th, 13:00 - 14:00 on Zoom to discuss: “Buildng Theories in Software Engineering” by Easterbrook et al. (Chapter 12 in this pdf: https://www.cin.ufpe.br/~fmcf2/Doutorado/2008-Guide%20to%20Advanced%20Empirical%20Software%20Engineering.pdf#page=317) Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Many approaches for testing configurable software systems start from the same assumption: it is impossible to test all configurations. This motivated the definition of variability-aware abstractions and sampling techniques to cope with large configuration spaces. Yet, there is no theoretical barrier that prevents the exhaustive testing of all configurations by enumerating them if the effort required to do so remains acceptable. In this talk, I will introduce the different elements currently used to abstract and reason about software variability, and the most common sampling techniques applied to select representative configurations for testing.

Summary Code review is a widely used technique to support software quality. It is a manual activity, often subject to repetitive and tedious tasks that increase the mental load of reviewers and compromise their effectiveness. The developer-centered nature of code review can represent a bottleneck that does not scale in large systems with the consequence of com- promising firms’ profits. This challenge has led to an entire line of research on code review improvement.

Code duplication is a form of technical debt frequently observed in software systems. Its existence negatively affects the maintainability of a system in numerous ways. In order to tackle the issues that come with it, various automated clone detection techniques have been proposed throughout the years. However, the vast majority of them operate using the entire codebase as input, resulting in redundant calculations and undesirable delays when this process is repeated for every new revision of a project.

As organizations start to adopt machine learning in critical business scenarios, the development processes change and the reliability of the applications becomes more important. To investigate these changes and improve the reliability of those applications, we conducted two studies in this thesis. The first study aims to understand the evolution of the processes by which machine learning applications are developed and how state-of-the-art lifecycle models fit the current needs of the fintech industry.

Systematic literature reviews in software engineering as well as other disciplines, serve as the foundation for sound scientific research. The aim for these literature reviews is to aggregate all existing knowledge on a research problem and produce informed guidelines for practitioners. This enables practitioners to apply appropriate software engineering solutions in a specific contexts. However, one major problem exists regarding systematic literature reviews, the overall execution duration may take up as much as 24 months.

The recommendation of tasks for newcomers within a software project through good first issues is being done within the domain of software development, such as on Github platform. These issues aim to help newcomers identify tasks that are suitable for them and their level of expertise within the project. This thesis report investigates the effectiveness regarding developer onboarding and task completion of good first issues by data mining a set of 105 repositories and manually analyzing at most 30 good first issues and 30 initial commits per sampled project.

Personality plays a significant role in our lives; it does not only influence what we think, feel, and do, but also affects what we say about what we think, feel, and do. In software engineering (SE), it might help in improving team composition through a combination of personalities within a team, and it could help explain work preferences and work satisfaction. Earlier studies in the field of software engineering have focused on extracting personality from developers with the use of questionnaires and automatic tools such as psycholinguistic tests.

Join the next SERG Reading Club 📖📚! Next week Thursday, June the 25th, 14:00 - 15:00 we come together on Zoom to discuss: “Selecting Empirical Methods for Software Engineering Research” by Easterbrook et al. (Chapter 11 in this pdf: https://www.cin.ufpe.br/~fmcf2/Doutorado/2008-Guide%20to%20Advanced%20Empirical%20Software%20Engineering.pdf#page=343) Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

The main goal of this thesis is to investigate, improve and extend the applicability of mutation testing. To seek the potential directions of how to improve and extend the applicability of mutation testing, we have started with a systematic literature review on the current state of how mutation testing is applied. The results from the systematic literature review have further guided us towards three directions of research: speeding up mutation testing; deepening our understanding of mutation testing; exploring new application domains of mutation testing.

Debugging application crashes is an expensive and time-taking process, relying on the developer’s expertise, and requiring knowledge about the system. Over the years, the research community has developed several automated approaches to ease debugging. Among those approaches, search-based crash reproduction, which tries to generate a test case capable of reproducing a given crash to make it observable to the developers, solely based on the stack trace included in the crash report.

Join the next SERG Reading Club 📖📚! On Thursday, June the 18th, 14:00 - 15:00 we come together on Zoom to discuss: “Building Theories from Multiple Evidence Sources” by Forrest Shull and Raimund L. Feldmann. (pdf: https://www.cin.ufpe.br/~fmcf2/Doutorado/2008-Guide%20to%20Advanced%20Empirical%20Software%20Engineering.pdf#page=343) Wanna join & you are not part of our Mattermost? Send an email to c.e.brandt /at/ tudelft.nl to get the Zoom link 😊

Continuous Delivery for Machine Learning (CD4ML) deals with the challenges of applying Continuous Delivery principles to ML systems to make the end-to-end process of developing and deploying them more repeatable and reliable. These systems are generally more complex than traditional software applications, and ML models are non-deterministic and hard to explain. In this talk we will discuss the challenges of testing and quality in ML systems, and share some practices for applying different types of tests to help overcome those issues.

Hello Everyone, I will be presenting my ongoing research this Thursday (11th June, 15:30 hours) as part of my Go/No-Go meeting. I will talk about key challenges in integration of software and security from developer’s perspective. I will also cover the work done during the first year of my phd and future research plan. Please feel free to join to provide your valuable feedback. The audio/video conferencing details will be shared later.

This presentation will be about my ongoing research at TU Delft. We have worked on using ML techniques for predicitng topics for Github repositories. Github added topic tagging functionality in 2017, but so far only about 2M public repositories have at least one tag. Topic tags define the main features, functionality, and the goal behind the creation of a software repository. In this research, we propose an automatic approach for predicting topics for software repositories using state-of-the-art machine learning techniques.

This talk discusses the results of my three months internship in the SERG team, done as part of my master thesis at the University of Namur. During this internship, I worked with Xavier Devroey and Pouria Derakhshanfar on JCrashPack, a benchmark for Java crash reproduction. More specifically, I explored the following question: Can we measure the difficulty for search-based crash reproduction to reproduce a crash using software quality metrics? Abstract: Search-Based Crash Reproduction (SBCR) aims to help developers in their debugging tasks by generating a test case that reproduces a specific crash based on its stack trace and the source code.

Join the next SERG Reading Club📚📖! On Friday, May the 29th, 14:00 - 15:00 we come together on Zoom to discuss the generic problem of distinguishing causality and correlation based on this paper: “Failure is a four-letter word: a parody in empirical research." Zeller, Andreas, Thomas Zimmermann, and Christian Bird. Proceedings of the 7th International Conference on Predictive Models in Software Engineering. 2011. plus these slides: http://darribas.org/gds15/slides/lecture_10.pdf Wanna join & you are not part of our Mattermost?

Mistakes in what a program’s source code actually does are inevitable. One of the potential causes are “atoms of confusion''; the smallest possible patterns of misinterpretable source code. These patterns and the misunderstandings and errors they can cause are explored in past studies in the C programming language. They are found to occur in many large projects and style guides. For this work, the existing tested set of atoms of confusion is translated into the Java programming language.

On May 14th we’re discussing the second half of this long and dense paper 😃 (starting from 3️⃣ below) Everybody can join to listen & discuss, even if you haven’t read the paper! 😊 Ever wondered how to properly survey literature? Want to avoid these ‘you should have done this differently’ comments on the next one you submit? Or are you an expert in literature surveys and want to share your awesome knowledge with us?

The Delta-Maintainability Model (DMM) offers a methodology for assessing the way an individual commit affects the maintainability of a system. In a nutshell, the DMM measures the proportion of low-risk change in a commit. The resulting value ranges from 0.0 (all changes are risky) to 1.0 (all changes are low risk). It rewards making methods better, and penalizes making things worse. Recently, we implemented a simple version of the DMM as part of PyDriller, the open source git repository mining tool.

Ever wondered how to properly survey literature? Want to avoid these ‘you should have done this differently’ comments on the next one you submit? Or are you an expert in literature surveys and want to share your awesome knowledge with us? Join the next SERG Reading Club📚📖! On Thursday, May the 7th, 14:00 - 15:00 we come together on Zoom to discuss: “Guidelines for performing Systematic Literature Reviews in Software Engineering” by Barbara Kitchenham (2007 Version).

In this talk, I will be discussing the challenges of logging an enterprise system in the context of Adyen, our industry partner, and talk about my ongoing work. Log data plays a key role in many important analysis tasks that critical for business such as auditing, application performance monitoring (APM), and root cause analysis (RCA). However, developing and maintaining logging code is hard: developers need to choose an appropriate log level, message format, variables that must be logged, and where to place log statements.

Dynamic programming languages, such as Python and JavaScript, enable fast prototyping. This leads to unexpected runtime behavior, suboptimal IDE support, and type errors. To mitigate these issues, dynamic languages added support for type annotations, which allows developers to specify types for the variables. However, the process of manually annotating codebases is cumbersome and error-prone. Aside from the traditional static analyzers, Machine Learning (ML) models can be employed to aid the type annotation process.

Search-based test data generation approaches have come a long way over the past few years, but these approaches still have some limitations when it comes to exercising specific behavior for triggering particular kinds of faults (e.g., crashes or specific types of integration between classes/modules). In this thesis, we are investigating new fitness functions and evolutionary-based algorithms and techniques to tackle these limitations. We have defined multiple novel approaches for crash reproduction and class integration testing.

In this talk, I will present a technique related to the test of Software Product Lines. This problem is hard due to (i) the high number of variants that are usually considered in SPLs, and (ii) the new dimension added to the tests for the assessment of performances (i.e., providing inputs that may impact the performances of the variants). Thus, it is necessary to try to reduce the number of inputs to assess the performances of the different variants, which, in the end, may have an impact on the choice made by the end-users for choosing one specific variant.

This presentation will be about the FASTEN project and ongoing research that builds the basis of FASTEN. Slides are available here. The main goal of the FASTEN project is to make package management more intelligent. Incidents like leftpad (A significant portion of the Internet has been broken by removing a package from a package manager) proved that the current way of managing packages has some flaws. FASTEN wants to mitigate such flaws by enriching package managers with method-level information.

You will hear the presentation I gave for my master thesis defense in Munich :) Slides are available here. Continuous integration produces detailed logs about the status and results of the various tools involved in the build. These build logs are a valuable data source for developers and researchers to inspect test results, to check the duration of build steps and to understand the cause of a build failure. However, build logs are very verbose, at best semi-structured and their structure differs highly between projects.

Robbert Krebbers discusses his ongoing effort to scale up formal verification techniques to challenging programming paradigms like concurrency in real programmling languages like C and Rust by developing mathematical foundations and usable tool implementations.

“Great software doesn’t come from tools, it comes from people.” During software development and maintenance, companies continually deal with issues related to the management of human resources. Such factors as team dynamics, experience, seniority, training, and office culture could influence the success of a company as well as the quality of software produced. Hence, human factors should be carefully considered and monitored since they are an integral part of the software process and could lead to dangerous consequences (e.

A Critical Comparison of Issue- and Risk-based Approaches to Assess Technical Debt In this lunch talk, we are going to present results from a recent study where we compared issue-based and risk-based measurements approaches to detect technical debt. We performed an empirical study on data gathered over 30,000 commits, both with risk- and issue-based technical debt, over 12 Apache Software Foundation projects. Further, we conducted a qualitative study on a subset of two of the projects to understand the benefits for the end-user when using both approaches together.

Replication work about the factors that influence pull request acceptance.

Abstract Distributed and concurrent applications often have subtle bugs that only get exposed under specific schedules. While these schedules may be found by systematic model checking techniques, in practice, model checkers do not scale to large systems. On the other hand, naive random exploration techniques often require a very large number of runs to find the specific interactions needed to expose a bug. In recent years, several random testing algorithms have been proposed that, on the one hand, exploit state-space reduction strategies from model checking and, on the other, provide guarantees on the probability of hitting bugs of certain kinds.

Abstract The utility of a smartphone is limited by its battery capacity and the ability of its hardware and software to efficiently use the device’s battery. To properly characterize the energy consumption of an app and identify energy defects, it is critical that apps are properly tested, i.e., analyzed dynamically to assess the app’s energy properties. However, currently there is a lack of testing tools for evaluating the energy properties of apps.

Abstract There has been a fundamental shift amongst software developers and engineers in the past few years. The software development life cycle (SDLC) for a developer has increased in complexity and scale. Changes that were developed and deployed over a matter of days or weeks are now deployed in a matter of hours. Due to greater availability of compute, storage, better tooling, and the necessity to react, developers are constantly looking to increase their velocity and throughput of developing and deploying changes.

In today’s rapidly changing world, the ability to be agile as well as predictable is essential to modern software development companies. To become more predictable in software deliveries, it is essential to have knowledge of previous similar projects that have been developed by the company, and also to understand the factors that may affect effort estimation. While software effort estimation is an active research are, there are only a few studies that identify factors that affect the predictability of software other than estimation model and process.

Abstract: The Big Code team at Facebook explores the intersection of Machine Learning, Software Engineering, and Programming Languages, in order to create developer tools. This presentation will give an overview of our research, which includes topics such as code search/recommendation, program repair, test selection, and resolving crashes.

We are going to discuss the paper: Ralph, P. (2018). Toward methodological guidelines for process theories and taxonomies in software engineering. IEEE Transactions on Software Engineering. DOI Preprint

abstract

In this talk, I discuss the latest research efforts in logging research spread at different stages in the lifecycle of log data: from the challenges of implementing and maintaining log statements, processing, and analysis. This content is based in a literature review conducted during my first year at TU Delft and it spans papers from different fields (e.g., Machine Learning and Distributed Systems). In addition, I share my experience at Adyen, our industry partner that process payments at a global scale for high volume merchants like Netflix, Spotify, etc, and has been actively investing in research and development with TU Delft.

End-of-the-year dinner of the SERG members.

We are going to discuss the paper: Stol, K. J., & Fitzgerald, B. (2018). The ABC of software engineering research. ACM Transactions on Software Engineering and Methodology (TOSEM), 27(3), 11. Link: https://dl.acm.org/citation.cfm?id=3241743

TL;DR See attached Twitter thread Software developers love package management systems! They provide a simple gateway to import thousands of free third-party libraries into your code. What is not free is the maintenance of third-party libraries: you have to keep up-to-date with news about your libraries (also known as dependencies) such as security fixes and performance bugs. This is challenging as we don’t use one dependency but a small network of them!

We are going to discuss the paper: Stol, K. J., & Fitzgerald, B. (2015). Theory-oriented software engineering. Science of Computer Programming, 101, 79-98. Link: https://www.sciencedirect.com/science/article/pii/S0167642314005425

LIBTwinSVM is an efficient and open-source library for Twin Support Vector Machines (TSVMs). Slides are available here.

Sharanya Konandur, Jón Briem, and Pavel Rapoport (all SERG MSc students) will discuss how SE researchers have been using biometrics in their experiments. More specifically: Sharanya will talk about the different parts of the brain (more specifically, the ones we can measure with our EEG) and what they mean. Pavel will talk about the different experimental designs he has found in the literature. Jón talks about the different tasks researchers give to participants.

Luca presented his ASE 2019 journal-first paper on defect prediction.

Search-based approaches have been used in the literature to automate the process of creating unit test cases. However, related work has shown that generated unit-tests with high code coverage could be ineffective, i.e., they may not detect all faults or kill all injected mutants. In this paper, we proposed an integration-level test case generator named CLING, that exploits the integration code of a pair of classes (caller and callee) that interact with one another through method calls.