User Acceptance Test AERIUS Calculator beta 11.1
In March 2015, a User Acceptance Test (GAT) was conducted for AERIUS Calculator beta 11.1. The objective of this GAT was to tests the Calculator’s applicability within the PAS and its user friendliness with regard to the operationalisation on 1 July 2015, when the PAS came into force.
In total, 77 people participated in the GAT at four locations throughout the country. This group consisted mainly of permit applicants (mostly advisers) and representatives from competent authorities.
The applicability of Calculator was tested according to 19 test scenarios for the sectors of agriculture, industry, transport and other. Calculator’s user-friendliness was tested using a Software Usability Measurement Inventory (SUMI test). This is an often-used method with international benchmarking.
The GAT was organised by the PAS desk and the GAT AERIUS Calculator project team consisting of representatives from the provinces/environmental services, Rijkswaterstaat (government service for roads and waterways), RIVM and the AERIUS team. The GAT was commissioned by the Steering committee for Development and Implementation of PAS instruments.
From the test report:
During the evaluating discussion following the four GAT days, AERIUS Calculator was considered a beautiful programme, by general consensus. Many compliments were handed out, particularly on user-friendliness. A few people considered Calculator an enormous improvement on currently used programmes. That not everything was an intuitive experience at the first encounter with AERIUS Calculator also became apparent, but nearly everyone was happy with the instrument: ‘I will be able to work the programme just fine after a few hours of practice.’
The main conclusions in the test report:
- Four scenarios could not be tested (input files with large numbers of sources and large surface areas). Thus, the applicability and user-friendliness of the programme regarding ‘large projects’ was not tested. At the time of the GAT, AERIUS Calculator was not suitable for calculating these types of large projects.
- There is some lack of clarity about the construction of data files using IMAER. IMAER describes the requirements for such data files with respect to structure, but it does not describe, for example, how a shipping route should be entered or how a road construction project should best be uploaded.
- No other barriers were found at the execution of the test scenarios during the GAT days.
- There were eight findings with ‘high seriousness’ and five findings in category 1. These findings make the use of AERIUS under the PAS difficult. Solving these issues is therefore a priority.
- A large number of often occuring but less serious findings were reported that affect user-friendliness.
- The SUMI questionnaire scores indicate an above-average user-friendliness.
The test report indicates that AERIUS Calculator can be made available if the main findings (points 1 and 2) can be solved.