Ensuring the quality of embedded systems is a critical task, because bugs discovered by the customer cost a lot - in terms of effort to fix it and in terms of reputation for the company. But from a developer's point of view, it is difficult to judge when there was enough testing done and on which parts of the overall system the tests should be focused. In purely software-based applications, often test coverage metrics provide the answer to that. But for embedded real-time systems, it is much more difficult to get that data. Some parts of these systems can be isolated, unit-tested and provide that data. But to fully check whether an embedded systems works properly, it also must be tested live in operation and there it is much more difficult to get test coverage data, because the system might reboot between the test cases and because different components might take part in the test.
Thesis Type:
Workflow:
In this thesis we want to find out how the effort for embedded system tests can be perfectly targeted. For that, we want to have test coverage data for all our tests - on a unit-test level as well as on a system-test level. The thesis shall contain:
- A theoretical part, which investigates the methodology of coverage-based adaptive testing. The theoretical part shall evaluate strategies how test coverage data can be obtained for different test levels and how such data can be used to target test execution and test development.
- A practical part, which implements mechanisms to collect test coverage data for a system with an industrial PLC. That test coverage data shall extend from a unit-test level to a system-test level, shall be collected during daily test execution, and shall be aggregated. The aggregated data then shall be presented in a way that allows to judge which of the existing tests are the most important ones and that allows to judge where future tests should be targeted.
Tasks:
- Research regarding testing of embedded real-time systems
- Research regarding test coverage and testing strategies
- Implement a test setup with industrial PLCs to collect test coverage data
- Experiment how to collect and aggregate test coverage data for systems that reboot during test execution
- Present test data for daily test execution (e.g. as Jenkins test results)
Recommended Prior Knowledge:
- Experience in embedded systems design
- Experience in C/C++
- Experience in software testing
Start:
Contact: