|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object de.jstacs.classifier.assessment.ClassifierAssessment
public abstract class ClassifierAssessment
Class defining an assessment of classifiers.
It should be used as a superclass for specialized classifier assessments like
k-fold-crossvalidation or subsampling-crossvalidation.
Several standard tasks like classifier or model management (testing,
training) are implemented. The method assess( ... )
should be
used as standard method to start a classifier assessment. Subclasses have to
implement the method evaluateClassifier( ... )
. This method
mainly has to execute the construction of test and training subsets of the
given data. These test and training subsets may be used by the methods
test( ... )
and train( ... )
which already are
implemented in a standard way.
Field Summary | |
---|---|
protected AbstractClassifier[] |
myAbstractClassifier
This array contains the internal used classifiers. |
protected Model[][] |
myModel
This array contains for each class the internal used models. |
protected MeanResultSet[] |
myTempMeanResultSets
The temporary result set. |
protected int |
skipLastClassifiersDuringClassifierTraining
Skip last classifier. |
Constructor Summary | |
---|---|
|
ClassifierAssessment(AbstractClassifier... aCs)
Creates a new ClassifierAssessment from a set of
AbstractClassifier s. |
|
ClassifierAssessment(AbstractClassifier[] aCs,
boolean buildClassifiersByCrossProduct,
Model[]... aMs)
This constructor allows to assess a collection of given AbstractClassifier s and, in addition, classifiers that will be
constructed using the given AbstractModel s. |
protected |
ClassifierAssessment(AbstractClassifier[] aCs,
Model[][] aMs,
boolean buildClassifiersByCrossProduct,
boolean checkAlphabetConsistencyAndLength)
Creates a new ClassifierAssessment from an array of
AbstractClassifier s and a two-dimensional array of Model
s, which are combined to additional classifiers. |
|
ClassifierAssessment(boolean buildClassifiersByCrossProduct,
Model[]... aMs)
Creates a new ClassifierAssessment from a set of Model s. |
Method Summary | |
---|---|
ListResult |
assess(MeasureParameters mp,
ClassifierAssessmentAssessParameterSet assessPS,
ProgressUpdater pU,
Sample... s)
Assesses the contained classifiers. |
ListResult |
assess(MeasureParameters mp,
ClassifierAssessmentAssessParameterSet assessPS,
ProgressUpdater pU,
Sample[][]... s)
Assesses the contained classifiers. |
ListResult |
assess(MeasureParameters mp,
ClassifierAssessmentAssessParameterSet assessPS,
Sample... s)
Assesses the contained classifiers. |
protected abstract void |
evaluateClassifier(MeasureParameters mp,
ClassifierAssessmentAssessParameterSet assessPS,
Sample[] s,
ProgressUpdater pU)
This method must be implemented in all subclasses. |
AbstractClassifier[] |
getClassifier()
Returns a deep copy of all classifiers that have been or will be used in this assessment. |
String |
getNameOfAssessment()
Returns the name of this class. |
protected void |
prepareAssessment(Sample... s)
Prepares an assessment. |
protected void |
test(MeasureParameters mp,
boolean exception,
Sample... testS)
Uses the given test samples to call the evaluate( ... )
-methods of the local AbstractClassifier s. |
protected void |
train(Sample... trainS)
Trains the local classifiers using the given training samples. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected AbstractClassifier[] myAbstractClassifier
protected Model[][] myModel
protected MeanResultSet[] myTempMeanResultSets
protected int skipLastClassifiersDuringClassifierTraining
Constructor Detail |
---|
protected ClassifierAssessment(AbstractClassifier[] aCs, Model[][] aMs, boolean buildClassifiersByCrossProduct, boolean checkAlphabetConsistencyAndLength) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
ClassifierAssessment
from an array of
AbstractClassifier
s and a two-dimensional array of Model
s, which are combined to additional classifiers. If
buildClassifiersByCrossProduct
is true
, the
cross product of all Model
s in aMs
is built to
obtain these classifiers.
aCs
- the predefined classifiersaMs
- the Model
s that are used to build additional
classifiersbuildClassifiersByCrossProduct
- Determines how classifiers are constructed using the given
models. Suppose a k-class problem. In this case, each
classifier is supposed to consist of k models, one responsible
for each class. S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.checkAlphabetConsistencyAndLength
- indicates if alphabets and lengths shall be checked for
consistency
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierpublic ClassifierAssessment(AbstractClassifier... aCs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
ClassifierAssessment
from a set of
AbstractClassifier
s.
aCs
- contains the classifiers to be assessed.assess( ... )
.s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment(AbstractClassifier[],
Model[][], boolean, boolean)
public ClassifierAssessment(boolean buildClassifiersByCrossProduct, Model[]... aMs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
ClassifierAssessment
from a set of Model
s.
The argument buildClassifiersByCrossProduct
determines how
these Model
s are combined to classifiers.
buildClassifiersByCrossProduct
- S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.aMs
- aMs[i]
) contains the
models according to class i
.s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment(AbstractClassifier[],
Model[][], boolean, boolean)
public ClassifierAssessment(AbstractClassifier[] aCs, boolean buildClassifiersByCrossProduct, Model[]... aMs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
AbstractClassifier
s and, in addition, classifiers that will be
constructed using the given AbstractModel
s.
aCs
- contains some AbstractClassifier
s that should be
assessed in addition to the AbstractClassifier
s
constructed using the given AbstractModel
sbuildClassifiersByCrossProduct
- Determines how classifiers are constructed using the given
models. Suppose a k-class problem. In this case, each
classifier is supposed to consist of k models, one responsible
for each class. S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.aMs
- Contains the models in the following way (suppose a k-class
problem): the first dimension encodes the class (here it is
k), the second dimension (aMs[i]
) contains the
models according to class i
.s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment(AbstractClassifier[],
Model[][], boolean, boolean)
Method Detail |
---|
public ListResult assess(MeasureParameters mp, ClassifierAssessmentAssessParameterSet assessPS, Sample... s) throws IllegalArgumentException, WrongAlphabetException, Exception
s
- contains the data to be used for assessment. The order of the
samples is important. s
. If
the models are trained directly, the order of given models
during initiation of this assessment object determines, which
sample will be used for training which model. In general the
first model will be trained using the first sample in
s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
mp
- defines which performance measure should be used to assess
classifiersassessPS
- contains some parameters necessary for assessment (depends on
the kind of assessment!)
ListResult
that contains the results (mean and standard
errors) of user specified performance measures. These performance
measures are user specified via the given
MeasureParameters
.
IllegalArgumentException
- if the given assessPS
is not of the right type
(see method evaluateClassifier( ... )
)
WrongAlphabetException
- if the given samples s
do not use the same
AlphabetContainer
as contained classifiers/models
Exception
- forwarded from training/testing of classifiers/modelsassess(MeasureParameters,
ClassifierAssessmentAssessParameterSet, ProgressUpdater, Sample...)
public ListResult assess(MeasureParameters mp, ClassifierAssessmentAssessParameterSet assessPS, ProgressUpdater pU, Sample... s) throws IllegalArgumentException, WrongAlphabetException, Exception
s
- contains the data to be used for assessment. The order of the
samples is important. s
. If
the models are trained directly, the order of given models
during initiation of this assessment object determines, which
sample will be used for training which model. In general the
first model will be trained using the first sample in
s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
mp
- defines which performance measure should be used to assess
classifiersassessPS
- contains some parameters necessary for assessment (depends on
the kind of assessment!)pU
- this ProgressUpdater
may be used to cancel this method
assess()
by setting
pU.isCancelled()=true
. In that case,
assess()
will abort but return results already
computed.KFoldCrossValidation
.
In this case it might be wise to override this method such
that it just returns an error message. pU
is allowed to be null
although in
this case it may be more convenient to use the second method
code()
not requiring a ProgressUpdater
.
ListResult
that contains the results (mean and standard
errors) of user specified performance measures. These performance
measures are user specified via the given
MeasureParameters
.
IllegalArgumentException
- if the given assessPS
is not of the right type
(see method evaluateClassifier( ... )
)
WrongAlphabetException
- if the given samples s
do not use the same
AlphabetContainer
as contained classifiers/models
Exception
- forwarded from training/testing of classifiers/modelspublic ListResult assess(MeasureParameters mp, ClassifierAssessmentAssessParameterSet assessPS, ProgressUpdater pU, Sample[][]... s) throws IllegalArgumentException, WrongAlphabetException, Exception
assess()
-methods this one allows the user to predefine all
train and test data beforehand which should be used for assessment.
s
- contains the data to be used for assessment.s
are important.s[iter][train/test][]
-> the first dimension
codes for which samples (train, test) are used in iteration
iter
. s[iter][0]
or test: s[iter][1]
.
s[iter][0]
contains for each class a training
sample. Analog s[iter][1]
contains the test
samples. The order of the samples is important. For further
details see comment of method
assess(MeasureParameters, ClassifierAssessmentAssessParameterSet, Sample...)
.mp
- defines which performance measure should be used to assess
classifiersassessPS
- contains some parameters necessary for assessment. Must be of
type ClassifierAssessmentAssessParameterSet
pU
- this ProgressUpdater
allows to abort this classifier
assessment. If pU.isCancelled()=true
, all results
already computed will be returned. It is allowed to give a
null reference.
ListResult
that contains the results (mean and standard
errors) of user specified performance measures. These performance
measures are user specified via the given
MeasureParameters
.
IllegalArgumentException
- if the given assessPS
is not of the right type
(see method evaluateClassifier( ... )
)
WrongAlphabetException
- if the given samples s
do not use the same
AlphabetContainer
as contained classifiers/models
Exception
- forwarded from training/testing of classifiers/modelspublic AbstractClassifier[] getClassifier() throws CloneNotSupportedException
CloneNotSupportedException
- if it is impossible to get a deep copy for at least one
classifier (if the classifier could not be cloned)public String getNameOfAssessment()
protected abstract void evaluateClassifier(MeasureParameters mp, ClassifierAssessmentAssessParameterSet assessPS, Sample[] s, ProgressUpdater pU) throws IllegalArgumentException, Exception
train()
to train classifiers/models using
train data test()
to cause evaluation (test) of trained
classifiers
mp
- defines which performance measures are used to assess
classifiersassessPS
- contains assessment specific parameters (like: number of
iterations of a k-fold-crossvalidation)s
- data to be used for assessment (both: test and train data)pU
- a ProgressUpdater
that mainly has to be used to allow
the user to cancel a current running classifier assessment.
This ProgressUpdater
is guaranteed to be not
null
. In certain cases aborting a classifier
assessment will not be allowed for example in case of
KFoldCrossValidation
. In this case the given
ProgressUpdater
should be ignored. pU.setMax()
= number of iterations of the assessment loop
pU.setValue()
=iteration+1;
train()
;
test()
;
pU.isCancelled()
))
IllegalArgumentException
- if the given ClassifierAssessmentAssessParameterSet
is of wrong type
Exception
- that occurred during training or using classifiers/modelsprotected void prepareAssessment(Sample... s) throws IllegalArgumentException, WrongAlphabetException
Sample
may not be used for
this assessment, this method throws an Exception
. MeanResultSet
s are initiated for this assessment (one for
each contained classifier).
s
- the Sample
to be checked
WrongAlphabetException
- if s
is null
or not of required
length (number of classes)AlphabetContainer
s of s
are not
consistent with AlphabetContainer
of local models or
classifiersIllegalArgumentException
- if the given samples are not suitableprotected void test(MeasureParameters mp, boolean exception, Sample... testS) throws SimpleParameter.IllegalValueException, MeanResultSet.InconsistentResultNumberException, MeanResultSet.AdditionImpossibleException, Exception
evaluate( ... )
-methods of the local AbstractClassifier
s. The returned
NumericalResult
s as well as the numerical
characteristics are added to each classifiers MeanResultSet
.
mp
- determines which performance measures are used to assess the
classifiersexception
- whether an Exception
should be thrown if some
MeasureParameters.Measure
could
not be evaluatedtestS
- samples used as test sets (has to contain one Sample
for each class)
SimpleParameter.IllegalValueException
- if a parameter is not valid
MeanResultSet.InconsistentResultNumberException
- if the number of results between the different result sets
differ
MeanResultSet.AdditionImpossibleException
- if added result sets do not match
Exception
- if necessary
IllegalArgumentException
- if the length of testS
is not equal to the
dimension of the classification problem (
testS.length!=this.myAbstractClassifier
[0].getNumberOfClasses()
)AbstractClassifier.evaluate(MeasureParameters, boolean, Sample...)
protected void train(Sample... trainS) throws IllegalArgumentException, Exception
ClassifierAssessment
-object was constructed using
AbstractModel
s.
trainS
- samples used as training sets (has to contain one
Sample
for each class)
IllegalArgumentException
- if the length of trainS
is not equal to the
dimension of the classification problem (
trainS.length!=this.myAbstractClassifier
[0].getNumberOfClasses()
)
Exception
- if necessary
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |