Workspace module

This module contains functions that are used to initialize, load and use a workspace. A workspace is a dedicated directory, separate from the toolkit directory, where experiment-specific results are stored. The directory contains several special subdirectories that contain sequence data, per-tracker raw results, report documents, and cached data.

About workspace

The workspace is automatically populated with several subdirectories:

Evaluation process

By default the entire evaluation is performed sequentially as described by the following pseudo-code:

tracker t
for experiment e:
    for sequence s:
        repeat r times (if tracker is stochastic):
            perform trial for (t, s)

Each trial contains one or more executions of the tracker. The idea is that if the tracker fails during tracking the execution (the failure criterion can be experiment dependent) it is repeated from the point of the failure (plus additional offset frames if specified).

In the case of stochastic trackers, each sequence is evaluated multiple times. If the tracker produces identical trajectories two times in a row, the tracker is considered deterministic and further iterations are omitted. It is therefore important that the stochastic nature of a tracker is appropriately addressed (proper random seed initialization).

Because of the thorough methodology, the entire execution can take quite some time. The function workspace_test provides an option for estimating the processing time for the entire evaluation based on a single run on one sequence.

Module functions