Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models
Loading...
Author(s)
Kim, Myeongsoo
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
REST APIs are fundamental to modern web services, necessitating rigorous testing to ensure their reliability. While the OpenAPI Specification provides a structured framework for API documentation, existing black-box testing tools struggle with limited code coverage and fault detection. To address these challenges, this dissertation advances REST API testing by incorporating machine learning and natural language processing (NLP) techniques to enhance automated test generation and execution.
Through an empirical evaluation of ten state-of-the-art black-box testing tools across twenty RESTful web services, this research identified three key limitations: ineffective generation of domain-specific test inputs, overlooked dependencies between API operations and parameters, and inefficient exploration of the vast API search space. Based on these findings, this dissertation introduces several innovations: (1) leveraging NLP techniques to systematically extract testable rules from human-readable API documentation, improving the generation of meaningful test cases; (2) utilizing Large Language Models (LLMs) to refine API specifications, detect nuanced constraints, and generate realistic input values for testing; (3) applying Reinforcement Learning (RL) to dynamically prioritize and optimize test execution based on API response feedback, leading to improved efficiency and effectiveness; (4) integrating fine-tuned Small Language Models (SLMs) to generate realistic input values and resolve parameter dependencies efficiently, significantly improving code coverage and fault detection while ensuring computational efficiency; and (5) developing a multi-agent black-box testing approach that coordinates multiple specialized agents to explore API behaviors more effectively, increasing test coverage and fault detection.
Evaluations demonstrate that these approaches significantly outperform existing testing tools, establishing a new paradigm for more effective automated testing of REST APIs.
Sponsor
Date
2025-04-30
Extent
Resource Type
Text
Resource Subtype
Dissertation