de.jstacs.models.mixture
Class MixtureModel

java.lang.Object
  de.jstacs.models.AbstractModel
      de.jstacs.models.mixture.AbstractMixtureModel
          de.jstacs.models.mixture.MixtureModel

All Implemented Interfaces:

Model, Storable, Cloneable

Direct Known Subclasses:

SharedStructureMixture

public class MixtureModel
extends AbstractMixtureModel

Class for a mixture model of any Models.

If you use Gibbs sampling temporary files will be created in the Java temp folder. These files will be deleted if no reference to the current instance exists and the Garbage collector is called. Therefore it is recommended to call the Garbage collector explicitly at the end of any application.

Author:

Jens Keilwagen, Berit Haldemann

Nested Class Summary

Nested classes/interfaces inherited from class de.jstacs.models.mixture.AbstractMixtureModel

AbstractMixtureModel.Algorithm, AbstractMixtureModel.Parameterization

Field Summary

Fields inherited from class de.jstacs.models.mixture.AbstractMixtureModel

algorithm, algorithmHasBeenRun, alternativeModel, burnInTest, componentHyperParams, compProb, counter, dimension, estimateComponentProbs, file, filereader, filewriter, initialIteration, logWeights, model, optimizeModel, sample, samplingIndex, sostream, starts, stationaryIteration, weights

Fields inherited from class de.jstacs.models.AbstractModel

alphabets, length

Constructor Summary

	MixtureModel(int length, Model[] models, double[] weights, int starts, double alpha, double eps, AbstractMixtureModel.Parameterization parametrization) Creates an instance using EM and fixed component probabilities.
	MixtureModel(int length, Model[] models, double[] weights, int starts, int initialIteration, int stationaryIteration, BurnInTest burnInTest) Creates an instance using Gibbs Sampling and fixed component probabilities.
protected	MixtureModel(int length, Model[] models, int starts, boolean estimateComponentProbs, double[] componentHyperParams, double[] weights, AbstractMixtureModel.Algorithm algorithm, double alpha, double eps, AbstractMixtureModel.Parameterization parametrization, int initialIteration, int stationaryIteration, BurnInTest burnInTest) Creates a new MixtureModel.
	MixtureModel(int length, Model[] models, int starts, double[] componentHyperParams, double alpha, double eps, AbstractMixtureModel.Parameterization parametrization) Creates an instance using EM and estimating the component probabilities.
	MixtureModel(int length, Model[] models, int starts, double[] componentHyperParams, int initialIteration, int stationaryIteration, BurnInTest burnInTest) Creates an instance using Gibbs Sampling and sampling the component probabilities.
	MixtureModel(StringBuffer xml) The constructor for the interface Storable.

Method Summary

protected double[][]	doFirstIteration(double[] dataWeights, MultivariateRandomGenerator m, MRGParams[] params) This method will do the first step in the train algorithm for the current model on the internal sample.
double[][]	doFirstIteration(Sample data, double[] dataWeights, double[][] partitioning) This method enables you to train a mixture model with a fixed start partitioning.
protected Sequence[]	emitSampleUsingCurrentParameterSet(int n, int... lengths) The method returns an array of sequences using the current parameter set.
protected double	getLogProbUsingCurrentParameterSetFor(int component, Sequence s, int start, int end) Returns the log probability for the sequence and the given component using the current parameter set.
protected double	getNewWeights(double[] dataWeights, double[] w, double[][] seqweights) Computes sequence weights and returns the score.
protected void	setTrainData(Sample data) This method is invoked by the train method and set for a given sample the sample that should be used for train.
String	toString() Should give a simple representation (text) of the model as String.

Methods inherited from class de.jstacs.models.mixture.AbstractMixtureModel

algorithmHasBeenRun, checkLength, checkModelsForGibbsSampling, clone, continueIterations, continueIterations, createSeqWeightsArray, doFirstIteration, doFirstIteration, draw, emitSample, extendSampling, extractFurtherInformation, finalize, fromXML, getCharacteristics, getFurtherInformation, getIndexOfMaximalComponentFor, getInstanceName, getLogPriorTerm, getLogPriorTermForComponentProbs, getLogProbFor, getLogProbFor, getLogProbFor, getModel, getModels, getMRG, getMRGParams, getNameOfAlgorithm, getNewComponentProbs, getNewParameters, getNewParametersForModel, getNumberOfComponents, getNumericalCharacteristics, getProbFor, getScoreForBestRun, getWeights, initModelForSampling, initWithPrior, isInSamplingMode, isTrained, iterate, iterate, max, modifyWeights, parseNextParameterSet, parseParameterSet, samplingStopped, set, setAlpha, setOutputStream, setThreshold, setWeights, swap, toXML, train

Methods inherited from class de.jstacs.models.AbstractModel

getAlphabetContainer, getLength, getLogProbFor, getLogProbFor, getLogProbFor, getMaximalMarkovOrder, getPriorTerm, getProbFor, getProbFor, setNewAlphabetContainerInstance, train

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MixtureModel

protected MixtureModel(int length,
                       Model[] models,
                       int starts,
                       boolean estimateComponentProbs,
                       double[] componentHyperParams,
                       double[] weights,
                       AbstractMixtureModel.Algorithm algorithm,
                       double alpha,
                       double eps,
                       AbstractMixtureModel.Parameterization parametrization,
                       int initialIteration,
                       int stationaryIteration,
                       BurnInTest burnInTest)
                throws IllegalArgumentException,
                       WrongAlphabetException,
                       CloneNotSupportedException

Creates a new MixtureModel. This constructor can be used for any algorithm since it takes all necessary values as parameters.

Parameters:

length - the length used in this model

models - the single models building the MixtureModel, if the model is trained using AbstractMixtureModel.Algorithm.GIBBS_SAMPLING the models that will be adjusted have to implement GibbsSamplingComponent

starts - the number of times the algorithm will be started in the train method, at least 1

estimateComponentProbs - the switch for estimating the component probabilities in the algorithm or to hold them fixed; if the component parameters are fixed, the values of weights will be used, otherwise the componentHyperParams will be incorporated in the adjustment

componentHyperParams - the hyperparameters for the component assignment prior,

• will only be used if estimateComponentProbs == true
• the array has to be null or has to have length models.length
• null or an array with all values zero (0) than ML
• otherwise (all values positive) a prior is used (MAP, MP, ...)
• depends on the parameterization

weights - null or the weights for the components (than weights.length == models.length)

algorithm - either AbstractMixtureModel.Algorithm.EM or AbstractMixtureModel.Algorithm.GIBBS_SAMPLING

alpha - only for AbstractMixtureModel.Algorithm.EM
the positive parameter for the Dirichlet which is used when you invoke train to initialize the gammas. It is recommended to use alpha = 1 (uniform distribution on a simplex).

eps - only for AbstractMixtureModel.Algorithm.EM
the non-negative threshold for stopping the EM-algorithm

parametrization - only for AbstractMixtureModel.Algorithm.EM
the type of the component probability parameterization;

• AbstractMixtureModel.Parameterization.THETA or AbstractMixtureModel.Parameterization.LAMBDA
• the parameterization of in a component is determined by the component model
• it is recommended to use the same parameterization for the components and the component assignment probabilities
• it is recommended to use LAMBDA
  initialIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
  the positive length of the initial sampling phase (at least 1, at most stationaryIteration/starts)
  stationaryIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
  the positive length of the stationary phase (at least 1) (summed over all starts), i.e. the number of parameter sets that is used in approximation
  burnInTest - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
  the test that will be used to determine the length of the burn-in phase
  Throws:
  IllegalArgumentException - if
  • the models are not able to score the sequence of length length
  • dimension < 1
  • weights != null && weights.length != dimension
  • weights != null and it exists an i where weights[i] < 0
  • starts < 1
  • componentHyperParams are not correct
  • the algorithm specific parameters are not correct
  WrongAlphabetException - if not all models work on the same alphabet
  CloneNotSupportedException - if the models can not be cloned

MixtureModel

public MixtureModel(int length,
                    Model[] models,
                    int starts,
                    double[] componentHyperParams,
                    double alpha,
                    double eps,
                    AbstractMixtureModel.Parameterization parametrization)
             throws IllegalArgumentException,
                    WrongAlphabetException,
                    CloneNotSupportedException

Creates an instance using EM and estimating the component probabilities.

Parameters:

length - the length used in this model

models - the single models building the MixtureModel, if the model is trained using AbstractMixtureModel.Algorithm.GIBBS_SAMPLING the models that will be adjusted have to implement GibbsSamplingComponent

starts - the number of times the algorithm will be started in the train method, at least 1

componentHyperParams - the hyperparameters for the component assignment prior,

• will only be used if estimateComponentProbs == true
• the array has to be null or has to have length models.length
• null or an array with all values zero (0) than ML
• otherwise (all values positive) a prior is used (MAP, MP, ...)
• depends on the parameterization

alpha - only for AbstractMixtureModel.Algorithm.EM
the positive parameter for the Dirichlet which is used when you invoke train to initialize the gammas. It is recommended to use alpha = 1 (uniform distribution on a simplex).

eps - only for AbstractMixtureModel.Algorithm.EM
the non-negative threshold for stopping the EM-algorithm

parametrization - only for AbstractMixtureModel.Algorithm.EM
the type of the component probability parameterization;

• AbstractMixtureModel.Parameterization.THETA or AbstractMixtureModel.Parameterization.LAMBDA
• the parameterization of in a component is determined by the component model
• it is recommended to use the same parameterization for the components and the component assignment probabilities
• it is recommended to use LAMBDA
  Throws:
  IllegalArgumentException - if
  • the models are not able to score the sequence of length length
  • dimension < 1
  • weights != null && weights.length != dimension
  • weights != null and it exists an i where weights[i] < 0
  • starts < 1
  • componentHyperParams are not correct
  • the algorithm specific parameters are not correct
  WrongAlphabetException - if not all models work on the same alphabet
  CloneNotSupportedException - if the models can not be cloned
  See Also:
  MixtureModel(int, de.jstacs.models.Model[], int, boolean, double[], double[], de.jstacs.models.mixture.AbstractMixtureModel.Algorithm, double, double, de.jstacs.models.mixture.AbstractMixtureModel.Parameterization, int, int, de.jstacs.models.mixture.gibbssampling.BurnInTest), AbstractMixtureModel.Algorithm.EM

MixtureModel

public MixtureModel(int length,
                    Model[] models,
                    double[] weights,
                    int starts,
                    double alpha,
                    double eps,
                    AbstractMixtureModel.Parameterization parametrization)
             throws IllegalArgumentException,
                    WrongAlphabetException,
                    CloneNotSupportedException

Creates an instance using EM and fixed component probabilities.

Parameters:

length - the length used in this model

models - the single models building the MixtureModel, if the model is trained using AbstractMixtureModel.Algorithm.GIBBS_SAMPLING the models that will be adjusted have to implement GibbsSamplingComponent

starts - the number of times the algorithm will be started in the train method, at least 1

weights - null or the weights for the components (than weights.length == models.length)

alpha - only for AbstractMixtureModel.Algorithm.EM
the positive parameter for the Dirichlet which is used when you invoke train to initialize the gammas. It is recommended to use alpha = 1 (uniform distribution on a simplex).

eps - only for AbstractMixtureModel.Algorithm.EM
the non-negative threshold for stopping the EM-algorithm

parametrization - only for AbstractMixtureModel.Algorithm.EM
the type of the component probability parameterization;

• AbstractMixtureModel.Parameterization.THETA or AbstractMixtureModel.Parameterization.LAMBDA
• the parameterization of in a component is determined by the component model
• it is recommended to use the same parameterization for the components and the component assignment probabilities
• it is recommended to use LAMBDA
  Throws:
  IllegalArgumentException - if
  • the models are not able to score the sequence of length length
  • dimension < 1
  • weights != null && weights.length != dimension
  • weights != null and it exists an i where weights[i] < 0
  • starts < 1
  • componentHyperParams are not correct
  • the algorithm specific parameters are not correct
  WrongAlphabetException - if not all models work on the same alphabet
  CloneNotSupportedException - if the models can not be cloned
  See Also:
  MixtureModel(int, de.jstacs.models.Model[], int, boolean, double[], double[], de.jstacs.models.mixture.AbstractMixtureModel.Algorithm, double, double, de.jstacs.models.mixture.AbstractMixtureModel.Parameterization, int, int, de.jstacs.models.mixture.gibbssampling.BurnInTest), AbstractMixtureModel.Algorithm.EM

MixtureModel

public MixtureModel(int length,
                    Model[] models,
                    int starts,
                    double[] componentHyperParams,
                    int initialIteration,
                    int stationaryIteration,
                    BurnInTest burnInTest)
             throws IllegalArgumentException,
                    WrongAlphabetException,
                    CloneNotSupportedException

Creates an instance using Gibbs Sampling and sampling the component probabilities.

Parameters:

length - the length used in this model

models - the single models building the MixtureModel, if the model is trained using AbstractMixtureModel.Algorithm.GIBBS_SAMPLING the models that will be adjusted have to implement GibbsSamplingComponent

starts - the number of times the algorithm will be started in the train method, at least 1

componentHyperParams - the hyperparameters for the component assignment prior,

• will only be used if estimateComponentProbs == true
• the array has to be null or has to have length models.length
• null or an array with all values zero (0) than ML
• otherwise (all values positive) a prior is used (MAP, MP, ...)
• depends on the parameterization

initialIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the positive length of the initial sampling phase (at least 1, at most stationaryIteration/starts)

stationaryIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the positive length of the stationary phase (at least 1) (summed over all starts), i.e. the number of parameter sets that is used in approximation

burnInTest - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the test that will be used to determine the length of the burn-in phase

Throws:

IllegalArgumentException - if

• the models are not able to score the sequence of length length
• dimension < 1
• weights != null && weights.length != dimension
• weights != null and it exists an i where weights[i] < 0
• starts < 1
• componentHyperParams are not correct
• the algorithm specific parameters are not correct

WrongAlphabetException - if not all models work on the same alphabet

CloneNotSupportedException - if the models can not be cloned

See Also:

MixtureModel(int, de.jstacs.models.Model[], int, boolean, double[], double[], de.jstacs.models.mixture.AbstractMixtureModel.Algorithm, double, double, de.jstacs.models.mixture.AbstractMixtureModel.Parameterization, int, int, de.jstacs.models.mixture.gibbssampling.BurnInTest), AbstractMixtureModel.Algorithm.GIBBS_SAMPLING

MixtureModel

public MixtureModel(int length,
                    Model[] models,
                    double[] weights,
                    int starts,
                    int initialIteration,
                    int stationaryIteration,
                    BurnInTest burnInTest)
             throws IllegalArgumentException,
                    WrongAlphabetException,
                    CloneNotSupportedException

Creates an instance using Gibbs Sampling and fixed component probabilities.

Parameters:

length - the length used in this model

models - the single models building the MixtureModel, if the model is trained using AbstractMixtureModel.Algorithm.GIBBS_SAMPLING the models that will be adjusted have to implement GibbsSamplingComponent

starts - the number of times the algorithm will be started in the train method, at least 1

weights - null or the weights for the components (than weights.length == models.length)

initialIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the positive length of the initial sampling phase (at least 1, at most stationaryIteration/starts)

stationaryIteration - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the positive length of the stationary phase (at least 1) (summed over all starts), i.e. the number of parameter sets that is used in approximation

burnInTest - only for AbstractMixtureModel.Algorithm.GIBBS_SAMPLING
the test that will be used to determine the length of the burn-in phase

Throws:

IllegalArgumentException - if

• the models are not able to score the sequence of length length
• dimension < 1
• weights != null && weights.length != dimension
• weights != null and it exists an i where weights[i] < 0
• starts < 1
• componentHyperParams are not correct
• the algorithm specific parameters are not correct

WrongAlphabetException - if not all models work on the same alphabet

CloneNotSupportedException - if the models can not be cloned

See Also:

MixtureModel(int, de.jstacs.models.Model[], int, boolean, double[], double[], de.jstacs.models.mixture.AbstractMixtureModel.Algorithm, double, double, de.jstacs.models.mixture.AbstractMixtureModel.Parameterization, int, int, de.jstacs.models.mixture.gibbssampling.BurnInTest), AbstractMixtureModel.Algorithm.GIBBS_SAMPLING

MixtureModel

public MixtureModel(StringBuffer xml)
             throws NonParsableException

The constructor for the interface Storable.

Parameters:

xml - the StringBuffer

Throws:

NonParsableException - if the StringBuffer is not parsable

Method Detail

emitSampleUsingCurrentParameterSet

protected Sequence[] emitSampleUsingCurrentParameterSet(int n,
                                                        int... lengths)
                                                 throws Exception

Description copied from class: AbstractMixtureModel

The method returns an array of sequences using the current parameter set.

Specified by:

emitSampleUsingCurrentParameterSet in class AbstractMixtureModel

Parameters:

n - the number of sequences to be sampled

lengths - the corresponding lengths

Returns:

an array of sequences

Throws:

Exception - if it was impossible to sample

See Also:

AbstractModel.emitSample(int, int...)

doFirstIteration

protected double[][] doFirstIteration(double[] dataWeights,
                                      MultivariateRandomGenerator m,
                                      MRGParams[] params)
                               throws Exception

Description copied from class: AbstractMixtureModel

This method will do the first step in the train algorithm for the current model on the internal sample. The initialization will be done by randomly setting the component membership. This is usefull when nothing is known about the problem.

Specified by:

doFirstIteration in class AbstractMixtureModel

Parameters:

dataWeights - null or the weights of each element of the sample

m - the multivariate random generator

params - the parameters for the multivariate random generator

Returns:

the weighting array used to initialize, this array can be reused in the following iterations

Throws:

Exception - if something went wrong

doFirstIteration

public double[][] doFirstIteration(Sample data,
                                   double[] dataWeights,
                                   double[][] partitioning)
                            throws Exception

This method enables you to train a mixture model with a fixed start partitioning. This is useful to compare implementations or if one has an hypothesis how the components should look like.

Parameters:

data - the sample of sequences

dataWeights - null or the weights of each element of the sample

partitioning - a kind of partitioning

1. partitioning.length has to be data.getNumberofElements()
2. for all i: partitioning[i].length has to be getNumberOfModels()
3. for all i: \sum_j partitioning[i][j] has to be 1

Returns:

the weighting array used to initialize, this array can be reused in the following iterations

Throws:

Exception - if something went wrong or if the number of components is 1

getLogProbUsingCurrentParameterSetFor

protected double getLogProbUsingCurrentParameterSetFor(int component,
                                                       Sequence s,
                                                       int start,
                                                       int end)
                                                throws Exception

Description copied from class: AbstractMixtureModel

Returns the log probability for the sequence and the given component using the current parameter set.

Specified by:

getLogProbUsingCurrentParameterSetFor in class AbstractMixtureModel

Parameters:

component - the index of the component

s - the sequence

start - the start position in the sequence

end - the end position in the sequence

Returns:

log P(s,component) = log P(s|component) + log P(component)

Throws:

Exception - if not trained yet or something else went wrong

See Also:

AbstractMixtureModel.getNumberOfComponents()

toString

public String toString()

Description copied from interface: Model

Should give a simple representation (text) of the model as String.

Specified by:

toString in interface Model

Overrides:

toString in class Object

Returns:

the representation as String

getNewWeights

protected double getNewWeights(double[] dataWeights,
                               double[] w,
                               double[][] seqweights)
                        throws Exception

Computes sequence weights and returns the score.

Specified by:

getNewWeights in class AbstractMixtureModel

Parameters:

dataWeights - the weights for internal sample (should not be changed)

w - the array for the statistic of the component parameters (shall be filled)

seqweights - an array containing for each component the weights for each sequence (shall be filled)

Returns:

the score

Throws:

Exception - if something went wrong

setTrainData

protected void setTrainData(Sample data)

Description copied from class: AbstractMixtureModel

This method is invoked by the train method and set for a given sample the sample that should be used for train.

Specified by:

setTrainData in class AbstractMixtureModel

Parameters:

data - the given sample