Generation of Pull Request Description using Transformers

Pull Requests (PR) is a mechanism by which project owners are notified about the changes made by a developer to merge their proposed changes into a project’s codebase. A PR is made up of several interrelated commits. To create a PR, developers need to provide a title and a description explaining what changes they made and why. These descriptions are important because they help reviewers quickly understand the purpose of the changes without having to get into the details. Developers often forget to write descriptions for their PRs, so it's helpful for them to have a way to generate these descriptions automatically. We conducted a comparative analysis of transformer-based models (T5, FLAN-T5, BART, and PLBART) for generating pull request descriptions. The evaluation utilized ROUGE metrics (ROUGE-1, ROUGE-2, ROUGE-L) to measure recall, precision, and F1-score for each model's effectiveness. Among the models, BARTbase outperformed the other transformer models. We proposed a fine-tuned BART-base transformer model with hyperparameters: a learning rate of 0.00002736 and a weight decay of 0.1. From experimental analysis we inferred that our proposed approach outperformed other state-of-the-art approaches.