Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## de.jstacs.classifiers.assessment
Class RepeatedHoldOutExperiment

    java.lang.Object
      de.jstacs.classifiers.assessment.ClassifierAssessment
          de.jstacs.classifiers.assessment.RepeatedHoldOutExperiment

    public class RepeatedHoldOutExperiment

extends ClassifierAssessment

This class implements a repeated hold-out experiment for assessing classifiers. The methodology used by a repeated hold-out experiment is as follows. The user supplies a dataset for each class the classifiers are capable to predict. In each step the given datasets are randomly, mutually exclusive partitioned into a test and a train dataset of user specified size. Afterwards the train datasets are used to train the classifiers and the test datasets are used to assess the performance of the classifiers to predict the elements therein using user specified assessment measures. Additional the user defines how often this procedure is repeated.

Author:
  Andre Gohr (bioinf (nospam:.) ag (nospam:@) googlemail (nospam:.) com)

Field Summary

Fields inherited from class de.jstacs.classifiers.assessment.ClassifierAssessment
`myAbstractClassifier, myModel, myTempMeanResultSets, skipLastClassifiersDuringClassifierTraining`

Constructor Summary
`` | `RepeatedHoldOutExperiment(AbstractClassifier... aCs)` Creates a new `RepeatedHoldOutExperiment` from a set of `AbstractClassifier`s.
`` | `RepeatedHoldOutExperiment(AbstractClassifier[] aCs, boolean buildClassifiersByCrossProduct, TrainableStatisticalModel[]... aMs)` This constructor allows to assess a collection of given `AbstractClassifier`s and those constructed using the given `TrainableStatisticalModel`s by a `RepeatedHoldOutExperiment`.
`protected` | `RepeatedHoldOutExperiment(AbstractClassifier[] aCs, TrainableStatisticalModel[][] aMs, boolean buildClassifiersByCrossProduct, boolean checkAlphabetConsistencyAndLength)` Creates a new `RepeatedHoldOutExperiment` from an array of `AbstractClassifier`s and a two-dimensional array of `TrainableStatisticalModel` s, which are combined to additional classifiers.
`` | `RepeatedHoldOutExperiment(boolean buildClassifiersByCrossProduct, TrainableStatisticalModel[]... aMs)` Creates a new `RepeatedHoldOutExperiment` from a set of `TrainableStatisticalModel`s.

Method Summary
`protected void` | `evaluateClassifier(NumericalPerformanceMeasureParameterSet mp, ClassifierAssessmentAssessParameterSet assessPS, DataSet[] s, ProgressUpdater pU)` Evaluates the classifier.

Methods inherited from class de.jstacs.classifiers.assessment.ClassifierAssessment
`assess, assess, assess, getClassifier, getNameOfAssessment, prepareAssessment, test, train`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### RepeatedHoldOutExperiment

    protected RepeatedHoldOutExperiment(AbstractClassifier[] aCs,
                                        TrainableStatisticalModel[][] aMs,
                                        boolean buildClassifiersByCrossProduct,
                                        boolean checkAlphabetConsistencyAndLength)
                                 throws IllegalArgumentException,
                                        WrongAlphabetException,
                                        CloneNotSupportedException,
                                        ClassDimensionException

  Creates a new `RepeatedHoldOutExperiment` from an array of `AbstractClassifier`s and a two-dimensional array of `TrainableStatisticalModel` s, which are combined to additional classifiers. If `buildClassifiersByCrossProduct` is `true`, the cross-product of all `TrainableStatisticalModel`s in `aMs` is built to obtain these classifiers.

  Parameters:
    `aCs` - the predefined classifiers
    `aMs` - the `TrainableStatisticalModel`s that are used to build additional classifiers
    `buildClassifiersByCrossProduct` - Determines how classifiers are constructed using the given models. Suppose a k-class problem. In this case, each classifier is supposed to consist of k models, one responsible for each class.
    Let `S_i` be the set of all models in `aMs[i]`. Let `S` be the set `S_1 x S_2 x ... x S_k` (cross-product).
    `true`: all possible classifiers consisting of a subset (set of k models) of `S` are constructed
    `false`: one classifier consisting of the models `aMs[0][i]`,`aMs[1][i]`,..., `aMs[k][i]` for a fixed `i` is constructed. In this case, all second dimensions of `aMs` have to be equal, say `m`. In total `m` classifiers are constructed.
    `checkAlphabetConsistencyAndLength` - indicates if alphabets and lengths shall be checked for consistency
  Throws:
    `IllegalArgumentException` - if the classifiers have different lengths
    `WrongAlphabetException` - if the classifiers use different alphabets
    `CloneNotSupportedException` - if something went wrong while cloning
    `ClassDimensionException` - if there is something wrong with the class dimension of the classifier
  See Also:
    `ClassifierAssessment.ClassifierAssessment(AbstractClassifier[], TrainableStatisticalModel[][], boolean, boolean)`

### RepeatedHoldOutExperiment

    public RepeatedHoldOutExperiment(AbstractClassifier... aCs)
                              throws IllegalArgumentException,
                                     WrongAlphabetException,
                                     CloneNotSupportedException,
                                     ClassDimensionException

  Creates a new `RepeatedHoldOutExperiment` from a set of `AbstractClassifier`s.

  Parameters:
    `aCs` - contains the classifiers to be assessed.
    If model based classifiers are trained, the order of models in classifiers determines, which model will be trained using which sample in method `assess( ... )`.
    For a two-class problem, it is recommended
    - to initiate the classifiers with models in order (foreground model (positive class), background model (negative class))
    - to initiate an assessment object using models in order (foreground model (positive class), background model (negative class))
    - to give data `s` in order (`s[0]` contains foreground data, `s[1]` contains background data)
  Throws:
    `IllegalArgumentException` - if the classifiers have different lengths
    `WrongAlphabetException` - if not all given classifiers are defined on the same `AlphabetContainer`
    `CloneNotSupportedException` - if something went wrong while cloning
    `ClassDimensionException` - if there is something wrong with the class dimension of the classifier
  See Also:
    `ClassifierAssessment.ClassifierAssessment(AbstractClassifier...)`

### RepeatedHoldOutExperiment

    public RepeatedHoldOutExperiment(boolean buildClassifiersByCrossProduct,
                                     TrainableStatisticalModel[]... aMs)
                              throws IllegalArgumentException,
                                     WrongAlphabetException,
                                     CloneNotSupportedException,
                                     ClassDimensionException

  Creates a new `RepeatedHoldOutExperiment` from a set of `TrainableStatisticalModel`s. The argument `buildClassifiersByCrossProduct` determines how these `TrainableStatisticalModel`s are combined to classifiers.

  Parameters:
    `buildClassifiersByCrossProduct` -
    Determines how classifiers are constructed using the given models. Suppose a k-class problem. In this case, each classifier is supposed to consist of k models, one responsible for each class.
    Let `S_i` be the set of all models in `aMs[i]`. Let `S` be the set `S_1 x S_2 x ... x S_k` (cross-product).
    `true`: all possible classifiers consisting of a subset (set of k models) of `S` are constructed
    `false`: one classifier consisting of the models `aMs[0][i]`,`aMs[1][i]`,..., `aMs[k][i]` for a fixed `i` is constructed. In this case, all second dimensions of `aMs` have to be equal, say `m`. In total `m` classifiers are constructed.
    `aMs` -
    Contains the models in the following way (suppose a k-class problem): the first dimension encodes the class (here it is k), the second dimension (`aMs[i]`) contains the models according to class `i`.
    If models are trained directly (during assessment), the order of given models during initiation of this assessment object determines, which sample will be used for training which model. In general the first model will be trained using the first sample in `s`... .
    For a two-class problem, it is recommended
    - to initiate the classifiers with models in order (foreground model (positive class), background model (negative class))
    - to initiate an assessment object using models in order (foreground model (positive class), background model (negative class))
    - to give data `s` in order (`s[0]` contains foreground data, `s[1]` contains background data)
  Throws:
    `IllegalArgumentException` - if the classifiers have different lengths
    `WrongAlphabetException` - if not all given classifiers are defined on the same `AlphabetContainer`
    `CloneNotSupportedException` - if something went wrong while cloning
    `ClassDimensionException` - if there is something wrong with the class dimension of the classifier
  See Also:
    `ClassifierAssessment.ClassifierAssessment(boolean, TrainableStatisticalModel[][])`

### RepeatedHoldOutExperiment

    public RepeatedHoldOutExperiment(AbstractClassifier[] aCs,
                                     boolean buildClassifiersByCrossProduct,
                                     TrainableStatisticalModel[]... aMs)
                              throws IllegalArgumentException,
                                     WrongAlphabetException,
                                     CloneNotSupportedException,
                                     ClassDimensionException

  This constructor allows to assess a collection of given `AbstractClassifier`s and those constructed using the given `TrainableStatisticalModel`s by a `RepeatedHoldOutExperiment`.

  Parameters:
    `aCs` - contains some `AbstractClassifier` that should be assessed in addition to the `AbstractClassifier`s constructed using the given `TrainableStatisticalModel`s
    `buildClassifiersByCrossProduct` -
    Determines how classifiers are constructed using the given models. Suppose a k-class problem. In this case, each classifier is supposed to consist of k models, one responsible for each class.
    Let `S_i` be the set of all models in `aMs[i]`. Let `S` be the set `S_1 x S_2 x ... x S_k` (cross-product).
    `true`: all possible classifiers consisting of a subset (set of k models) of `S` are constructed
    `false`: one classifier consisting of the models `aMs[0][i]`,`aMs[1][i]`,..., `aMs[k][i]` for a fixed `i` is constructed. In this case, all second dimensions of `aMs` have to be equal, say `m`. In total `m` classifiers are constructed.
    `aMs` -
    Contains the models in the following way (suppose a k-class problem): the first dimension encodes the class (here it is k), the second dimension (`aMs[i]`) contains the models according to class `i`.
    If models are trained directly (during assessment), the order of given models during initiation of this assessment object determines, which sample will be used for training which model. In general the first model will be trained using the first sample in`s`... .
    For a two-class problem, it is recommended
    - to initiate the classifiers with models in order (foreground model (positive class), background model (negative class))
    - to initiate an assessment object using models in order (foreground model (positive class), background model (negative class))
    - to give data `s` in order (`s[0]` contains foreground data, `s[1]` contains background data)
  Throws:
    `IllegalArgumentException` - if the classifiers have different lengths
    `WrongAlphabetException` - if not all given classifiers are defined on the same `AlphabetContainer`
    `CloneNotSupportedException` - if something went wrong while cloning
    `ClassDimensionException` - if there is something wrong with the class dimension of the classifier
  See Also:
    `ClassifierAssessment.ClassifierAssessment(AbstractClassifier[], boolean, TrainableStatisticalModel[][])`

Method Detail

### evaluateClassifier

    protected void evaluateClassifier(NumericalPerformanceMeasureParameterSet mp,
                                      ClassifierAssessmentAssessParameterSet assessPS,
                                      DataSet[] s,
                                      ProgressUpdater pU)
                               throws IllegalArgumentException,
                                      Exception

  Evaluates the classifier.

  Specified by:
    `evaluateClassifier` in class `ClassifierAssessment`
  Parameters:
    `mp` - defines which performance measures are used to assess classifiers
    `pU` - A `KFoldCrossValidation` is not allowed to be aborted. The given `ProgressUpdater` is never used in this method.
    `s` - contains the data to be used for assessment. The order of the samples is important.
    If model based classifiers are trained, the order of the models in the classifiers determines, which model will be trained using which sample. The first model in the classifier will be trained using the first sample in `s`. If the models are trained directly, the order of given models during initiation of this assessment object determines, which sample will be used for training which model. In general the first model will be trained using the first sample in `s`... .
    For a two-class problem, it is recommended
    - to initiate the classifiers with models in order (foreground model (positive class), background model (negative class))
    - to initiate an assessment object using models in order (foreground model (positive class), background model (negative class))
    - to give data `s` in order (`s[0]` contains foreground data, `s[1]` contains background data)
    `assessPS` - contains parameters for a run of this `RepeatedHoldOutExperiment`. Must be of type `RepeatedHoldOutAssessParameterSet`.
  Throws:
    `IllegalArgumentException` - if the given `assessPS` is not of type `RepeatedHoldOutAssessParameterSet`
    `Exception` - if something went wrong

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD