Analyzing the System Test report

The Runtime Trace viewer shows you what happened, but it doesn't make any reference to the success or failure of each WAITTIL. All success or failure values for any system test are recorded in the system test report.

To open the test report:

  1. Close the Project Explorer Window on the right-hand side of the screen to gain additional room for the runtime tracing diagram.

  2. In the Report Window on the left-hand side of the UI, close the INITIALIZATION, SCENARIO main, and TERMINATION nodes. The window should appear as follows:

Look at the Report Window; notice the existence of a node named INSTANCE phone1_0 - this is a reference to Virtual Tester 0 for the phone1 INSTANCE block. For every virtual tester executing the phone1 INSTANCE block, a separate node would exist in this browse tree. Since your test consisted of only one virtual tester, only one node exists in the tree.

By clicking the INITIALIZATION note in the Runtime Trace viewer, you were jumped to the INITIALIZATION section of the system test report. This section of the report could also be accessed by double-clicking the INITIALIZATION node in the Report Window.

  1. Expand the INITIALIZATION node in the Report Window.

Here, in the report, you see all of the CALLs made in the INITIALIZATION block of your system test. If any of these calls failed, that information would be found here.

  1. Expand the SCENARIO main node in the Report Window.

Now you're looking at all of the functions that occur within each SCENARIO block. (Expanding the SCENARIO main block in the Report Window will let you maneuver through the three SCENARIOs.) Again, every action is listed. Successes are color-code pink.

  1. In the Report Window, expand the SCENARIO main node if you haven't done so already, and then double-click the WAITTIL node located within the SCENARIO connect node:

Look at the report. Notice how the WAITTIL section is broken down into a WAITED EVENTS/RECEIVED EVENTS section - specifically, into the expected message (called MATCHING) and the obtained message (called mResponse). The expected message defines what must be in the obtained message; in this case, the obtained message must contain a field named command with a string value of CNX OK. As you can see, the obtained message can contain more data than was tested for; for example, the obtained message contains the additional fields phoneNumber, simCardId and baseStationId.

(The WAITTIL contains the clause WTIME>1000. This means that if it takes more than 10 seconds for the awaited message to arrive, a timeout would occur and the timeout error would be reported. The unit of measurement for this parameter can be modified via a TDP setting.)

  1. To view the test summary, scroll to the top of the report in the Test Report window.

Notice that 4 tests passed and 0 tests failed. This is a reference to the four SCENARIO blocks - the parent SCENARIO block named main and the three child SCENARIO blocks named connect, call_busy, and call_success.

Familiarize yourself with this report, noting that you can left-click all green-colored script functions performed by the virtual tester to view the test script itself.