Implementation: Batch Processing

iGestures provides a tool to test and evaluate different recognition algorithms. The idea behind this tool is to simplify the evaluation of newly developed algorithms and the comparison of different algorithms. Beside the comparison it allows us to test a specific algorithm with different parameters to adjust them in the best possible way for a given gesture set. Figure 9 shows the main classes belonging to the batch process handling.


Figure 9: BatchProcess class diagram

A gesture test set is encapsulated in a TestSet instance which consists of a set of GestureSample instances. Each instance contains the name of the gesture class the sample belongs to or the name is empty, if the gesture does not exist in the gesture set. This naming is necessary to check whether the input was recognised correctly.

Test Configuration

A batch process is configured with an XML file and out of it the different configuration objects are created. To be able to test different parameter adjustments we provide a mechanism to define various parameters.

As usual it is possible to set a parameter to a fixed value. This is realised with the following XML construct: 

							 
<parameter name="MIN_ACCURACCY">
  <value>0.85</value>
</parameter>

Then we can define a sequence of values a parameter can have. This is done as follows: 

						 
<parameter name="RESULTSET_SIZE">
  <sequence>
    <value>1</value>
    <value>8</value>
  </sequence>
</parameter>

The next construct acts like a for-loop. Each value in the given sequence is taken as a parameter. All three arguments are processed as double values. 

							 
<parameter name="MIN_DISTANCE">
  <for start="1" end="3.5" step="0.5"/>
</parameter>

The last construct creates the power-set of the specified length out of a comma delimited list. The two parameters min and max denotes the minimal and maximal length of the created power-set. The power-set of a list has $2^List.length$ elements in general. 

							 
<parameter name="FEATURE_LIST">
    <powerset min="4" max="6">F1,F2,F3,F4,F5,F6</powerset>
</parameter>

Out of this XML configuration file all possible parameter permutations are collected. For each configuration the batch process instantiates the algorithm and processes the test set. The results of the batch process are collected in a TestSetResult data structure. For each configuration the key figures are computed and collected.

Note that this batch process may be very memory and time consuming.