Performs regulatory module prediction (gene expression biclustering plus de novo sequence motif discovery) using the COALESCE algorithm of Huttenhower et al. 2009. More...

#include <coalesce.h>

Inheritance diagram for Sleipnir::CCoalesce:

Public Member Functions
bool	Cluster (const CPCL &PCL, const CFASTA &FASTA, std::vector< CCoalesceCluster > &vecClusters)
	Executes the COALESCE regulatory module prediction algorithm on the given gene expression (and, optionally, sequence) data.
void	SetSeed (const CPCL &PCL)
	Explicitly sets the expression profile used to seed the first module.
void	SetPValueCorrelation (float dPValue)
	Sets the correlation p-value threshhold for genes to be included in a cluster during initialization.
float	GetPValueCorrelation () const
	Returns the correlation p-value threshhold for genes to be included in a cluster during initialization.
void	SetBins (size_t iBins)
	Sets the number of discretization bins used for calculating motif frequency histograms.
size_t	GetBins () const
	Returns the number of discretization bins used for calculating motif frequency histograms.
float	GetZScoreCondition () const
	Returns the z-score effect size threshhold for including significant expression conditions in a cluster.
void	SetZScoreCondition (float dZScore)
	Sets the z-score effect size threshhold for including significant expression conditions in a cluster.
float	GetPValueCondition () const
	Returns the p-value threshhold for including significant expression conditions in a cluster.
void	SetPValueCondition (float dPValue)
	Sets the p-value threshhold for including significant expression conditions in a cluster.
float	GetZScoreMotif () const
	Returns the z-score effect size threshhold for including significant sequence motifs in a cluster.
void	SetZScoreMotif (float dZScore)
	Sets the z-score effect size threshhold for including significant sequence motifs in a cluster.
float	GetPValueMotif () const
	Returns the p-value threshhold for including significant sequence motifs in a cluster.
void	SetPValueMotif (float dPValue)
	Sets the p-value threshhold for including significant sequence motifs in a cluster.
float	GetProbabilityGene () const
	Returns the probability threshhold for including genes in a cluster.
void	SetProbabilityGene (float dProbability)
	Sets the probability threshhold for including genes in a cluster.
bool	IsDirectoryIntermediate () const
	Returns true if a module output directory has been set.
const std::string &	GetDirectoryIntermediate () const
	Returns the output directory for predicted modules.
void	SetDirectoryIntermediate (const std::string &strDirectoryIntermediate)
	Sets the output directory for predicted modules.
void	SetMotifs (CCoalesceMotifLibrary &Motifs)
	Sets the motif library used to manage gene sequences and motifs.
const CCoalesceMotifLibrary *	GetMotifs () const
	Returns the motif library used to manage gene sequences and motifs.
size_t	GetK () const
	Returns the length of k-mer motifs.
void	SetK (size_t iK)
	Sets the length of k-mer motifs.
size_t	GetBasesPerMatch () const
	Returns the granularity in base pairs with which motif frequency histograms are calculated.
void	SetBasesPerMatch (size_t iBasesPerMatch)
	Sets the granularity in base pairs with which motif frequency histograms are calculated.
float	GetPValueMerge () const
	Returns the p-value threshhold at which motifs are merged to build PSTs.
void	SetPValueMerge (float dPValue)
	Sets the p-value threshhold at which motifs are merged to build PSTs.
float	GetCutoffMerge () const
	Returns the edit distance threshhold at which motifs are merged to build PSTs.
void	SetCutoffMerge (float dCutoff)
	Sets the edit distance threshhold at which motifs are merged to build PSTs.
size_t	GetSizeMinimum () const
	Returns the minimum number of genes that must be present in a successful module.
void	SetSizeMinimum (size_t iSizeGenes)
	Sets the minimum number of genes that must be present in a successful module.
size_t	GetSizeMaximum () const
	Returns the maximum number of motifs that may be associated with a converging module.
void	SetSizeMaximum (size_t iSizeMotifs)
	Sets the maximum number of motifs that may be associated with a converging module.
size_t	GetSizeMerge () const
	Returns the maximum number of motifs that are considered for merging into PSTs during module convergence.
void	SetSizeMerge (size_t iSizeMerge)
	Sets the maximum number of motifs that are considered for merging into PSTs during module convergence.
void	ClearDatasets ()
	Removes all currently set dataset blocks.
bool	AddDataset (const std::set< size_t > &setiDataset)
	Adds a block of conditions known to form a non-independent dataset.
void	SetNumberCorrelation (size_t iPairs)
	Sets the maximum number of gene pairs subsampled for seed pair discovery during module initialization.
size_t	GetNumberCorrelation () const
	Returns the maximum number of gene pairs subsampled for seed pair discovery during module initialization.
void	SetThreads (size_t iThreads)
	Sets the maximum number of simultaneous threads used for clustering.
size_t	GetThreads () const
	Returns the maximum number of simultaneous threads used for clustering.
void	AddWiggle (const CFASTA &FASTA)
	Adds a wiggle track of supporting data to be used to weight sequence information.
void	ClearWiggles ()
	Removes all currently active wiggle tracks.
void	AddOutputIntermediate (std::ostream &ostm)
	Adds an output stream to which module information is printed after convergence.
void	RemoveOutputIntermediate (std::ostream &ostm)
	Removes an output stream to which module information was printed after convergence.
void	ClearOutputIntermediate ()
	Removes all currently active intermediate output streams.
void	SetNormalize (bool fNormalize)
	Sets the normalization behavior for automatically detected single channel expression conditions.
bool	GetNormalize () const
	Returns true if automatic detection and normalization of single channel expression data is enabled.
void	ClearSeed ()
	Removes any currently set seed expression profile.

Detailed Description

Performs regulatory module prediction (gene expression biclustering plus de novo sequence motif discovery) using the COALESCE algorithm of Huttenhower et al. 2009.

The COALESCE algorithm consumes gene expression data and, optionally, DNA sequences, to predict regulatory modules. These consist of expression biclusters (subsets of genes and conditions) and putative regulatory motifs. COALESCE predicts modules in a serial manner, seeding each module with a small number of correlated genes. It then iterates between feature selection and Bayesian integration of the selected features to determine which genes should be in the module. Feature selection chooses expression conditions in which the cluster's genes are differentially expressed (i.e. significantly different than the genomic background) and sequence motifs over- or under-enriched in sequences associated with the cluster's genes (also relative to genomic background). Bayesian integration assumes that these features are independent (although prior knowledge of non-independent datasets can be provided and used to incorporate covariance information) and calculates the probability with which each gene in the genome is included in the developing module. These two steps (feature selection and Bayesian integration) are iterated until the module has converged, at which point its average values (expression and motif frequencies) are subtracted from its genes' data, and COALESCE continues with the next module. A variety of options and data can be used to modify this procedure, both at the level of the algorithm itself (e.g. the probability threshhold at which genes are included in a module) and at the level of implementation optimizations (e.g. the granularity with which motif frequencies are discretized).

Remarks:: CCoalesce is tightly coupled to CCoalesceCluster, where many of the details of the COALESCE algorithm are implemented. CCoalesce serves mainly to store configuration state information for clustering, to initialize associated data structures, and to provide the outermost skeleton of the algorithm.

See also:: CCoalesceCluster

Definition at line 61 of file coalesce.h.

Member Function Documentation

bool Sleipnir::CCoalesce::AddDataset ( const std::set< size_t > & setiDataset ) [inline]

Adds a block of conditions known to form a non-independent dataset.

Parameters:

setiDataset Set of condition indices forming a dataset.

Returns:: True if the dataset was added successfully; false if it already existed in the current configuration.

Adds a dataset block to subsequent executions of COALESCE. A dataset block consists of two or more expression conditions known to be non-independent, e.g. multiple conditions belonging to the same time course. Such dataset blocks are treated as units for inclusion in/exclusion from predicted modules, and their covariance is determined and incorporated into significance calculations for differential expression.

Remarks:: Condition indices must correspond to columns in a PCL file subsequently provided to a call to Cluster.

See also:: ClearDatasets

Definition at line 573 of file coalesce.h.

void Sleipnir::CCoalesce::AddOutputIntermediate ( std::ostream & ostm ) [inline]

Adds an output stream to which module information is printed after convergence.

Parameters:

ostm	Output stream to which each module will be printed after it converges.

Remarks:: Usually a single output stream, standard output, is sufficient; this is provided for useless convenience.

See also:: RemoveOutputIntermediate

Definition at line 695 of file coalesce.h.

void Sleipnir::CCoalesce::AddWiggle ( const CFASTA & FASTA ) [inline]

Adds a wiggle track of supporting data to be used to weight sequence information.

Parameters:

FASTA FASTA file containing peudo-wiggle-track formatted per-base weights for gene sequences.

Adds a wiggle track of supporting information used to weight gene sequence positions during COALESCE clustering. A wiggle track as used by COALESCE is not precisely in the wiggle track format as defined by the ENCODE project; instead, it is a FASTA file in which sequence base pairs have been replaced by per-base-pair scores, one floating point value per line. In COALESCE, one or more wiggle tracks can be used to weight the individual base pairs used to determine motif occurrence and frequencies. Lower weights (down to zero) will downweight the base pairs at those positions (and thus the effective frequencies of any motifs that occur there), and higher weights will upweight them. In the absence of wiggle tracks, the default weight of all base pairs is one.

Remarks:: Contents of the provided pseudo-wiggle file must align with the FASTA file of gene sequences provided to subsequent calls to Cluster.

See also:: ClearWiggles | CFASTA

Definition at line 666 of file coalesce.h.

void Sleipnir::CCoalesce::ClearDatasets ( ) [inline]

Removes all currently set dataset blocks.

See also:: AddDataset

Definition at line 547 of file coalesce.h.

void Sleipnir::CCoalesce::ClearOutputIntermediate ( ) [inline]

Removes all currently active intermediate output streams.

See also:: AddOutputIntermediate | RemoveOutputIntermediate

Definition at line 725 of file coalesce.h.

void Sleipnir::CCoalesce::ClearSeed ( ) [inline]

Removes any currently set seed expression profile.

See also:: SetSeed

Definition at line 768 of file coalesce.h.

void Sleipnir::CCoalesce::ClearWiggles ( ) [inline]

Removes all currently active wiggle tracks.

See also:: AddWiggle

Definition at line 677 of file coalesce.h.

bool Sleipnir::CCoalesce::Cluster	(	const CPCL &	PCL,
		const CFASTA &	FASTA,
		std::vector< CCoalesceCluster > &	vecClusters
	)

Executes the COALESCE regulatory module prediction algorithm on the given gene expression (and, optionally, sequence) data.

Parameters:

PCL	PCL file containing genes and expression values with which clustering is performed.
FASTA	FASTA file (possibly empty) containing gene sequences used for motif prediction during clustering.
vecClusters	Output vector of regulatory modules predicted by COALESCE.

Returns:: True if clustering succeeded (possibly without predicting any modules), false otherwise.

Executes the COALESCE algorithm on the given data, predicting zero or more regulatory modules (expression biclusters plus putative sequence motifs). Each predicted module consists of one or more genes, one or more conditions of the given PCL in which those genes are coregulated, and zero or more sequence motifs over- or under-enriched (and thus potentially causal) in the module's genes. For more details, see CCoalesce and Huttenhower et al. 2009.

Remarks:: Cluster interacts heavily with CCoalesceCluster, which performs the major steps of condition, motif, and gene selection. Cluster itself contains mainly the skeleton of the algorithm, including initialization and convergence detection.

See also:: CCoalesce | CCoalesceCluster

Definition at line 539 of file coalesce.cpp.

size_t Sleipnir::CCoalesce::GetBasesPerMatch ( ) const [inline]

Returns the granularity in base pairs with which motif frequency histograms are calculated.

Returns:: Number of base pairs per match used to calculated motif frequency histograms.

See also:: SetBasesPerMatch

Definition at line 373 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetBins ( ) const [inline]

Returns the number of discretization bins used for calculating motif frequency histograms.

Returns:: Number of bins used to discretize motif frequencies.

See also:: SetBins

Definition at line 119 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetCutoffMerge ( ) const [inline]

Returns the edit distance threshhold at which motifs are merged to build PSTs.

Returns:: Edit distance threshhold at which motifs are merged to build PSTs.

See also:: SetCutoffMerge

Definition at line 432 of file coalesce.h.

Referenced by Cluster().

const std::string& Sleipnir::CCoalesce::GetDirectoryIntermediate ( ) const [inline]

Returns the output directory for predicted modules.

Returns:: Output directory in which predicted modules are saved.

See also:: SetDirectoryIntermediate

Definition at line 287 of file coalesce.h.

Referenced by Cluster(), and IsDirectoryIntermediate().

size_t Sleipnir::CCoalesce::GetK ( ) const [inline]

Returns the length of k-mer motifs.

Returns:: K-mer length of predicted motifs; also used as building blocks for more complex motifs.

See also:: SetK

Definition at line 345 of file coalesce.h.

Referenced by Cluster().

const CCoalesceMotifLibrary* Sleipnir::CCoalesce::GetMotifs ( ) const [inline]

Returns the motif library used to manage gene sequences and motifs.

Returns:: Motif library used to manage gene sequences and motifs during clustering; null if none has been set.

See also:: SetMotifs

Definition at line 331 of file coalesce.h.

Referenced by Cluster().

bool Sleipnir::CCoalesce::GetNormalize ( ) const [inline]

Returns true if automatic detection and normalization of single channel expression data is enabled.

Returns:: True if single channel condition detection and normalization is enabled.

See also:: SetNormalize

Definition at line 757 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetNumberCorrelation ( ) const [inline]

Returns the maximum number of gene pairs subsampled for seed pair discovery during module initialization.

Returns:: Maximum number of gene pairs subsampled for module seeding.

See also:: SetNumberCorrelation

Definition at line 611 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetProbabilityGene ( ) const [inline]

Returns the probability threshhold for including genes in a cluster.

Returns:: Probability threshhold for inclusion of genes in a cluster.

See also:: SetProbabilityGene

Definition at line 245 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetPValueCondition ( ) const [inline]

Returns the p-value threshhold for including significant expression conditions in a cluster.

Returns:: P-value threshhold for inclusion of expression conditions in a cluster.

See also:: SetPValueCondition

Definition at line 161 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetPValueCorrelation ( ) const [inline]

Returns the correlation p-value threshhold for genes to be included in a cluster during initialization.

Returns:: P-value threshhold for gene inclusion during module initialization.

See also:: SetPValueCorrelation

Definition at line 91 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetPValueMerge ( ) const [inline]

Returns the p-value threshhold at which motifs are merged to build PSTs.

Returns:: P-value threshhold at which motifs are merged to build PSTs.

See also:: SetPValueMerge

Definition at line 404 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetPValueMotif ( ) const [inline]

Returns the p-value threshhold for including significant sequence motifs in a cluster.

Returns:: P-value threshhold for inclusion of motifs in a cluster.

See also:: SetPValueMotif

Definition at line 217 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetSizeMaximum ( ) const [inline]

Returns the maximum number of motifs that may be associated with a converging module.

Returns:: Maximum number of motifs associated with a converging module.

See also:: SetSizeMaximum

Definition at line 488 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetSizeMerge ( ) const [inline]

Returns the maximum number of motifs that are considered for merging into PSTs during module convergence.

Returns:: Maximum number of motifs considered for PSTs construction during module convergence.

See also:: SetSizeMerge

Definition at line 519 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetSizeMinimum ( ) const [inline]

Returns the minimum number of genes that must be present in a successful module.

Returns:: Minimum number of genes present in a successful module.

See also:: SetSizeMinimum

Definition at line 460 of file coalesce.h.

Referenced by Cluster().

size_t Sleipnir::CCoalesce::GetThreads ( ) const [inline]

Returns the maximum number of simultaneous threads used for clustering.

Returns:: Maximum number of simultaneous threads used during clustering.

See also:: SetThreads

Definition at line 639 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetZScoreCondition ( ) const [inline]

Returns the z-score effect size threshhold for including significant expression conditions in a cluster.

Returns:: Z-score threshhold for inclusion of expression conditions in a cluster.

See also:: SetZScoreCondition

Definition at line 133 of file coalesce.h.

Referenced by Cluster().

float Sleipnir::CCoalesce::GetZScoreMotif ( ) const [inline]

Returns the z-score effect size threshhold for including significant sequence motifs in a cluster.

Returns:: Z-score threshhold for inclusion of motifs in a cluster.

See also:: SetZScoreMotif

Definition at line 189 of file coalesce.h.

Referenced by Cluster().

bool Sleipnir::CCoalesce::IsDirectoryIntermediate ( ) const [inline]

Returns true if a module output directory has been set.

Returns:: True if a module output directory has been set; false if modules are to be output only to standard out.

See also:: GetDirectoryIntermediate | SetDirectoryIntermediate

Definition at line 273 of file coalesce.h.

References GetDirectoryIntermediate().

Referenced by Cluster().

void Sleipnir::CCoalesce::RemoveOutputIntermediate ( std::ostream & ostm ) [inline]

Removes an output stream to which module information was printed after convergence.

Parameters:

ostm	Output stream to which modules were to be printed.

Remarks:: Removal of an output stream not in the current set will be ignored.

See also:: AddOutputIntermediate

Definition at line 712 of file coalesce.h.

void Sleipnir::CCoalesce::SetBasesPerMatch ( size_t iBasesPerMatch ) [inline]

Sets the granularity in base pairs with which motif frequency histograms are calculated.

Parameters:

iBasesPerMatch Number of base pairs per match used to calculated motif frequency histograms.

Remarks:: Each bin in a motif frequency histogram will be of width 1 / iBasesPerMatch.

See also:: GetBasesPerMatch

Definition at line 390 of file coalesce.h.

void Sleipnir::CCoalesce::SetBins ( size_t iBins ) [inline]

Sets the number of discretization bins used for calculating motif frequency histograms.

Parameters:

iBins Number of bins used to discretize motif frequencies.

See also:: GetBins

Definition at line 105 of file coalesce.h.

void Sleipnir::CCoalesce::SetCutoffMerge ( float dCutoff ) [inline]

Sets the edit distance threshhold at which motifs are merged to build PSTs.

Parameters:

dCutoff Edit distance threshhold at which motifs are merged to build PSTs.

See also:: GetCutoffMerge

Definition at line 446 of file coalesce.h.

void Sleipnir::CCoalesce::SetDirectoryIntermediate ( const std::string & strDirectoryIntermediate ) [inline]

Sets the output directory for predicted modules.

Parameters:

strDirectoryIntermediate Output directory in which predicted modules are saved.

See also:: GetDirectoryIntermediate

Definition at line 301 of file coalesce.h.

void Sleipnir::CCoalesce::SetK ( size_t iK ) [inline]

Sets the length of k-mer motifs.

Parameters:

iK	K-mer length of predicted motifs; also used as building blocks for more complex motifs.

See also:: GetK

Definition at line 359 of file coalesce.h.

void Sleipnir::CCoalesce::SetMotifs ( CCoalesceMotifLibrary & Motifs ) [inline]

Sets the motif library used to manage gene sequences and motifs.

Parameters:

Motifs Motif library used to manage gene sequences and motifs during clustering.

See also:: GetMotifs

Definition at line 315 of file coalesce.h.

void Sleipnir::CCoalesce::SetNormalize ( bool fNormalize ) [inline]

Sets the normalization behavior for automatically detected single channel expression conditions.

Parameters:

fNormalize If true, single channel conditions are detected and normalized; otherwise, they are left unchanged.

Remarks:: Single channel normalization is time-consuming and often degrades performance; it should usually be left disabled. However, it can find some interesting clusters given the right input data.

See also:: GetNormalize

Definition at line 743 of file coalesce.h.

void Sleipnir::CCoalesce::SetNumberCorrelation ( size_t iPairs ) [inline]

Sets the maximum number of gene pairs subsampled for seed pair discovery during module initialization.

Parameters:

iPairs Maximum number of gene pairs subsampled for module seeding.

See also:: GetNumberCorrelation

Definition at line 597 of file coalesce.h.

void Sleipnir::CCoalesce::SetProbabilityGene ( float dProbability ) [inline]

Sets the probability threshhold for including genes in a cluster.

Parameters:

dProbability Probability threshhold for inclusion of genes in a cluster.

See also:: GetProbabilityGene

Definition at line 259 of file coalesce.h.

void Sleipnir::CCoalesce::SetPValueCondition ( float dPValue ) [inline]

Sets the p-value threshhold for including significant expression conditions in a cluster.

Parameters:

dPValue P-value threshhold for inclusion of expression conditions in a cluster.

See also:: GetPValueCondition

Definition at line 175 of file coalesce.h.

void Sleipnir::CCoalesce::SetPValueCorrelation ( float dPValue ) [inline]

Sets the correlation p-value threshhold for genes to be included in a cluster during initialization.

Parameters:

dPValue Correlation p-value threshhold for gene inclusion during module initialization.

See also:: GetPValueCorrelation

Definition at line 77 of file coalesce.h.

void Sleipnir::CCoalesce::SetPValueMerge ( float dPValue ) [inline]

Sets the p-value threshhold at which motifs are merged to build PSTs.

Parameters:

dPValue P-value threshhold at which motifs are merged to build PSTs.

See also:: GetPValueMerge

Definition at line 418 of file coalesce.h.

void Sleipnir::CCoalesce::SetPValueMotif ( float dPValue ) [inline]

Sets the p-value threshhold for including significant sequence motifs in a cluster.

Parameters:

dPValue P-value threshhold for inclusion of motifs in a cluster.

See also:: GetPValueMotif

Definition at line 231 of file coalesce.h.

void Sleipnir::CCoalesce::SetSeed ( const CPCL & PCL )

Explicitly sets the expression profile used to seed the first module.

Parameters:

PCL	PCL from which expression profile to be seeded is read.

Forces the first module to be seeded with the given expression profile rather than a randomly chosen significantly correlated gene pair.

Remarks:: Only the first gene from the given PCL is used. The PCL must contain the same number of conditions as the main PCL clustered by COALESCE.

Definition at line 313 of file coalesce.cpp.

References Sleipnir::CPCL::Get(), and Sleipnir::CPCL::GetExperiments().

void Sleipnir::CCoalesce::SetSizeMaximum ( size_t iSizeMotifs ) [inline]

Sets the maximum number of motifs that may be associated with a converging module.

Parameters:

iSizeMotifs Maximum number of motifs associated with a converging module.