Skip to content
Snippets Groups Projects
Select Git revision
  • main
  • Publication
2 results

test-generation-with-srl

  • Clone with SSH
  • Clone with HTTPS
    • Name Last commit Last update
    evaluation
    libs
    src
    tests
    .classpath
    .gitignore
    .project
    README.md
    annotation-cache
    diss.code-workspace
    pom.xml
    src.code-workspace
    README.md

    Test Case Generation from Natural Language Requirements with Semantic Role Labeling

    This repository contains a tool to generate executable test cases from natural language requirements. It was developed by Alexander Kugler as part of his disseration at the computer science chair "Informatik 11 - Embedded Software" of the RWTH Aachen University. This repository is provided for research purposes and is not actively maintained by the author. The current implementation is a proof of concept and several improvements can be made, as denoted by the ToDos in the code. You are free to use and modify the tool according to your needs.

    Approach

    The test generation pipeline consists of 5 stages and looks as follows. Details are provided in the publication.

    • Stage 1 Preprocessing of requirements & collection of information from other sources.
    • Stage 2 Extraction of syntactic and semantic information from requirements via Natural Language Processing methods, particularly using Semantic Role Labeling.
    • Stage 3 Formalization of extracted information from Stage 2 into logical expressions according to a ruleset. The logical expressions consist of timed conditions and actions. Aggregation of information into a system function that describes the system behavior.
    • Stage 4 Formalization of a system function into First-Order Logic formula. Use of a Satisfiability Modulo Theory solver to find a satisfying solution, which represents one test case.
    • Stage 5 Modification of the Satisfiability Modulo Theory instance according to predefined tactics in order to generate a strengthened test suite.

    Dependencies

    • CogComp-NLP: Used for natural language processing and in particular Semantic Role Labeling. The endpoint is configured in src/cogcompnlp/cogCompNLP.java.
    • Microsoft Z3 SMT Solver: Used for test case generation. The z3 library is contained in libs/.

    Setup & Usage

    1. Build the project: Use Maven to build the project and resolve dependencies:

      mvn clean install

    2. Specify input requirements: Input requirements are specified in src/testgeneration/Main.java.

    3. Generate test cases: Run the project to generate test cases, which will be stored in tests/.

    4. Evaluate test suite performance with MuJava: Use the scripts provided in src/evaluation to set up MuJava and perform a mutation-based strength analysis.

    Publication

    The folder evaluation/ contains data such as test cases with which the evaluation has been performed in the dissertation. The requirements of the three evaluated systems are contained in Main.java. If you use this tool in your research, please cite the following publication.

    Alexander Kugler. Test Case Generation from Natural Language Requirements for Embedded Systems with Semantic Role Labeling. PhD Thesis, Informatik 11 - Embedded Software, RWTH Aachen University, 2024. https://publications.rwth-aachen.de/record/1010169.