FeatureExtraction Class Reference

This class handles all feature extraction issues. More...

Inheritance diagram for FeatureExtraction:

[legend]

List of all members.

Public Member Functions
	FeatureExtraction (const char *audioIn, const char *audioOut, int mfccSize, int useEnergy, int useZeroCross, int useDeltas, double vtln, bool useSqrt10=false, bool offs=false, bool onlyPrepare=false, bool onlyMEL=false, bool pCTS=false)
	~FeatureExtraction ()
int	getVectorSize () const
int	getPoolSize () const
Vector **	getPool () const
Vector *	getOnlineVector (double *silThreshold, bool isSil)
void	setVTLN (double vtln)
void	setBackgroundHistNorm (Vector **hist, int numBins)
bool	createFeaturesUntilFrame (int lastFrame, int normID=0)
void	finishExtraction ()
void	doFeatureExtraction (const char *audioIn, const char *audioOut, bool onlyPrepare=false)
void	setNormalization (FEATURENORM_TYPE normT, int nrClusters, bool doMean, bool doVar, Vector **hist)
void	finishClusterCMVN ()
void	performClusterCMVNUntilFrame (int lastFrame, int cmvnID)
void	performHistNormUntilFrame (int lastFrame, int histID)
void	initializeHistNorm ()
void	setHistNormModel (int normID, Vector **h)
void	setAudioFileOut (const char *fileName)
void	performPCA (Vector **pca, int len)
void	setOnlyMEL ()
void	storeFeatureVectors (FILE *file)
Public Attributes
Vector **	inverseHistogram
Protected Member Functions
bool	readAudioWindow (short int *audioWindow)
void	calculateDelta (int time, int offset)
Vector *	createMfccFrame (int normID, int spectralSubtraction=-1)
Protected Attributes
int	melIndexAboveCTS
FILE *	audioFile
FILE *	audioFileOut
bool	offset
bool	onlyMEL
int	CEP_LIFTER
int	MEL_BANKLENGTH
Vector *	cepLifterFact
float	fea_delta_noemer
FFTReal *	fftProc
float	hammingTemplate [FEA_WINDOWSIZE]
short int	rememberBuffer [FEA_REMEMBERWINSIZE]
int	melTemplate [LARGE_MEL_BANKLENGTH+2]
float	fftWindowSpectralSubtract [FEA_FREQWINDOWSIZE]
Vector **	featureVector
Vector **	gaussianization
int	nrFrames
int	mfccSize
int	vectorSize
int	useDeltas
int	useZeroCross
int	useEnergy
int	nonDeltaSize
int	numberOfFeaturesProcessed
int	nrNormClusters
NormData *	normData
bool	doCMN
bool	doCVN
bool	doSqrt10
bool	performCTS
FEATURENORM_TYPE	normType

---

Detailed Description

This class handles all feature extraction issues.

The following feature extraction steps are taken:

• Create overlapping windows: default: 32 ms windows, every 10 ms.
• DC offset removal. (is this needed? Does the pre-emphesis solve this?)
• Pre-emphesis
• Apply Hamming window
• Calculate energy
• Fast Fourier Transformation (magnitude)
• Vocal Tract Length Normalization
• Mel bank filtering (default 20 banks, logaritmic)
• DCT transformation creating MFCC coefficients
• Cepstrum liftering
• Normalize the coefficients: mean substraction
• Calculate delta's and delta-delta's

This class can do feature extraction for a single audio file (16K16, raw PCM audio) or for a batch file.

---

Constructor & Destructor Documentation

FeatureExtraction::FeatureExtraction	(	const char *	audioIn,
		const char *	audioOut,
		int	mfccS,
		int	useE,
		int	useZC,
		int	useD,
		double	vtln,
		bool	useSqrt10 = false,
		bool	offs = false,
		bool	onlyPrepare = false,
		bool	onlyM = false,
		bool	pCTS = false
	)

Todo:

Add documentation

References audioFile, audioFileOut, CEP_LIFTER, cepLifterFact, doCMN, doCVN, doFeatureExtraction(), doSqrt10, fea_delta_noemer, FEATURENORM_SEGMENT, featureVector, fftProc, WriteFileLittleBigEndian::freadEndianSafe(), gaussianization, hammingTemplate, inverseHistogram, MEL_BANKLENGTH, mfccSize, nonDeltaSize, normData, normType, nrFrames, offset, onlyMEL, performCTS, Vector::setAllValues(), Vector::setValue(), setVTLN(), useDeltas, useEnergy, useZeroCross, and vectorSize.

Here is the call graph for this function:

FeatureExtraction::~FeatureExtraction

(

)

Todo:

Add documentation

References audioFile, cepLifterFact, NormData::cmvnCluster, featureVector, fftProc, NormData::gaussianizationDest, NormData::gaussianizationSource, NormData::histogram, inverseHistogram, normData, nrFrames, and nrNormClusters.

---

Member Function Documentation

void FeatureExtraction::calculateDelta	(	int	time,
		int	offset
	)			[protected]

Todo:

Add documentation

References fea_delta_noemer, featureVector, Vector::getValue(), nonDeltaSize, and Vector::setValue().

Referenced by doFeatureExtraction(), finishExtraction(), and getOnlineVector().

Here is the call graph for this function:

bool FeatureExtraction::createFeaturesUntilFrame	(	int	lastFrame,
		int	normID = 0
	)

Todo:

Add documentation

References NormData::cmvnCluster, createMfccFrame(), Vector::divideElements(), doCMN, doCVN, FEATURENORM_CLUSTER, FEATURENORM_SEGMENT, featureVector, Gaussian::meanVector, mfccSize, normData, normType, numberOfFeaturesProcessed, Vector::setValue(), Vector::sqrtElements(), Vector::substractElements(), Gaussian::train(), Gaussian::trainFinish(), useEnergy, useZeroCross, and Gaussian::varianceVector.

Referenced by FeaturePool::createNewPool(), doFeatureExtraction(), and getOnlineVector().

Here is the call graph for this function:

Vector * FeatureExtraction::createMfccFrame	(	int	normID,
		int	spectralSubtraction = -1
	)			[protected]

Todo:

Add documentation

References cepLifterFact, FFTReal::do_fft(), doSqrt10, fftProc, fftWindowSpectralSubtract, Vector::getValue(), hammingTemplate, NormData::histogram, inverseHistogram, MEL_BANKLENGTH, melIndexAboveCTS, melTemplate, mfccSize, Vector::multiplyElements(), normData, onlyMEL, readAudioWindow(), FFTReal::rescale(), Vector::setValue(), useEnergy, useZeroCross, and vectorSize.

Referenced by createFeaturesUntilFrame().

Here is the call graph for this function:

void FeatureExtraction::doFeatureExtraction	(	const char *	audioIn,
		const char *	audioOut,
		bool	onlyPrepare = false
	)

Todo:

Add documentation

References audioFile, calculateDelta(), createFeaturesUntilFrame(), Vector::divideElements(), doCMN, doCVN, featureVector, finishExtraction(), Gaussian::meanVector, nonDeltaSize, nrFrames, numberOfFeaturesProcessed, readAudioWindow(), rememberBuffer, Vector::setValue(), StringFunctions::splitList(), Vector::sqrtElements(), Vector::storeFloatData(), Vector::substractElements(), Gaussian::trainFinish(), useDeltas, Gaussian::varianceVector, and vectorSize.

Referenced by FeaturePool::createNewPool(), FeatureExtraction(), FeaturePool::retrieveEnergy(), and ShoutOnline::ShoutOnline().

Here is the call graph for this function:

void FeatureExtraction::finishClusterCMVN

(

)

Todo:

Add documentation

References NormData::cmvnCluster, FEATURENORM_CLUSTER, Gaussian::getMean(), Gaussian::getVariance(), mfccSize, normData, normType, nrNormClusters, numberOfFeaturesProcessed, Vector::setValue(), Vector::sqrtElements(), Gaussian::trainFinish(), useEnergy, and useZeroCross.

Referenced by FeaturePool::createNewPool().

Here is the call graph for this function:

void FeatureExtraction::finishExtraction

(

)

Todo:

Add documentation

References audioFile, calculateDelta(), Vector::divideElements(), doCMN, doCVN, FEATURENORM_ALL, featureVector, Gaussian::meanVector, mfccSize, nonDeltaSize, normType, nrFrames, numberOfFeaturesProcessed, Vector::setValue(), Vector::sqrtElements(), Vector::substractElements(), Gaussian::trainFinish(), useDeltas, useEnergy, useZeroCross, and Gaussian::varianceVector.

Referenced by FeaturePool::createNewPool(), and doFeatureExtraction().

Here is the call graph for this function:

Vector * FeatureExtraction::getOnlineVector	(	double *	silThreshold,
		bool	isSil
	)

Todo:

Add documentation

References audioFile, calculateDelta(), createFeaturesUntilFrame(), Vector::divideElements(), doCMN, doCVN, featureVector, Gaussian::meanVector, numberOfFeaturesProcessed, Vector::sqrtElements(), Vector::substractElements(), Gaussian::trainFinish(), useDeltas, and Gaussian::varianceVector.

Referenced by ShoutOnline::ShoutOnline().

Here is the call graph for this function:

Vector** FeatureExtraction::getPool

(

)

const [inline]

References featureVector.

Referenced by FeaturePool::createNewPool(), FeaturePool::fillBuffer_fromFeatureExtraction(), and FeaturePool::retrieveEnergy().

int FeatureExtraction::getPoolSize

(

)

const [inline]

References nrFrames.

Referenced by FeaturePool::addFeatureFile(), and FeaturePool::createNewPool().

int FeatureExtraction::getVectorSize

(

)

const [inline]

References vectorSize.

Referenced by FeaturePool::addFeatureFile(), and FeaturePool::createNewPool().

void FeatureExtraction::initializeHistNorm

(

)

Todo:

Add documentation

References numberOfFeaturesProcessed.

void FeatureExtraction::performClusterCMVNUntilFrame	(	int	lastFrame,
		int	normID
	)

Todo:

Add documentation

References Vector::divideElements(), doCMN, doCVN, featureVector, normData, numberOfFeaturesProcessed, and Vector::substractElements().

Referenced by FeaturePool::createNewPool().

Here is the call graph for this function:

void FeatureExtraction::performHistNormUntilFrame	(	int	lastFrame,
		int	normID
	)

Todo:

Add documentation

References featureVector, Vector::getValue(), NormData::histogram, inverseHistogram, Vector::len(), normData, numberOfFeaturesProcessed, and Vector::setValue().

Here is the call graph for this function:

void FeatureExtraction::performPCA	(	Vector **	pca,
		int	len
	)

Todo:

Add documentation

References featureVector, nrFrames, Vector::setValue(), and vectorSize.

Referenced by FeaturePool::createNewPool().

Here is the call graph for this function:

bool FeatureExtraction::readAudioWindow

(

short int *

audioWindow

)

[protected]

Todo:

Add documentation

References audioFile, audioFileOut, WriteFileLittleBigEndian::freadEndianSafe(), and rememberBuffer.

Referenced by createMfccFrame(), and doFeatureExtraction().

Here is the call graph for this function:

void FeatureExtraction::setAudioFileOut

(

const char *

fileName

)

Todo:

Add documentation

References audioFileOut.

Referenced by ShoutOnline::ShoutOnline().

void FeatureExtraction::setBackgroundHistNorm	(	Vector **	hist,
		int	numBins
	)

void FeatureExtraction::setHistNormModel	(	int	normID,
		Vector **	h
	)

Todo:

Add documentation

References NormData::histogram, and normData.

void FeatureExtraction::setNormalization	(	FEATURENORM_TYPE	normT,
		int	nrClusters,
		bool	doMean,
		bool	doVar,
		Vector **	hist
	)

Todo:

Add documentation

References NormData::cmvnCluster, Gaussian::dim(), doCMN, doCVN, FEATURENORM_CLUSTER, Gaussian::Gaussian(), gaussianization, NormData::gaussianizationDest, NormData::gaussianizationSource, Vector::getValue(), NormData::histogram, inverseHistogram, Vector::len(), normData, normType, nrFrames, nrNormClusters, and Vector::setValue().

Referenced by FeaturePool::createNewPool(), and ShoutOnline::ShoutOnline().

Here is the call graph for this function:

void FeatureExtraction::setOnlyMEL

(

)

Todo:

Add documentation

References onlyMEL.

Referenced by FeaturePool::setOnlyMEL().

void FeatureExtraction::setVTLN

(

double

vtln

)

Todo:

Add documentation

References MEL_BANKLENGTH, melIndexAboveCTS, melTemplate, and performCTS.

Referenced by FeaturePool::createNewPool(), FeatureExtraction(), and ShoutOnline::ShoutOnline().

void FeatureExtraction::storeFeatureVectors

(

FILE *

file

)

/

Todo:

Docs

References featureVector, MEL_BANKLENGTH, nrFrames, onlyMEL, and Vector::storeFloatData().

Referenced by FeaturePool::storeFeatureVectors().

Here is the call graph for this function:

---

Member Data Documentation

FILE* FeatureExtraction::audioFile [protected]

Referenced by doFeatureExtraction(), FeatureExtraction(), finishExtraction(), getOnlineVector(), readAudioWindow(), and ~FeatureExtraction().

FILE* FeatureExtraction::audioFileOut [protected]

Referenced by FeatureExtraction(), readAudioWindow(), and setAudioFileOut().

int FeatureExtraction::CEP_LIFTER [protected]

Referenced by FeatureExtraction().

Vector* FeatureExtraction::cepLifterFact [protected]

Referenced by createMfccFrame(), FeatureExtraction(), and ~FeatureExtraction().

bool FeatureExtraction::doCMN [protected]

Referenced by createFeaturesUntilFrame(), doFeatureExtraction(), FeatureExtraction(), finishExtraction(), getOnlineVector(), performClusterCMVNUntilFrame(), and setNormalization().

bool FeatureExtraction::doCVN [protected]

Referenced by createFeaturesUntilFrame(), doFeatureExtraction(), FeatureExtraction(), finishExtraction(), getOnlineVector(), performClusterCMVNUntilFrame(), and setNormalization().

bool FeatureExtraction::doSqrt10 [protected]

Referenced by createMfccFrame(), and FeatureExtraction().

float FeatureExtraction::fea_delta_noemer [protected]

Referenced by calculateDelta(), and FeatureExtraction().

Vector** FeatureExtraction::featureVector [protected]

Referenced by calculateDelta(), createFeaturesUntilFrame(), doFeatureExtraction(), FeatureExtraction(), finishExtraction(), getOnlineVector(), getPool(), performClusterCMVNUntilFrame(), performHistNormUntilFrame(), performPCA(), storeFeatureVectors(), and ~FeatureExtraction().

FFTReal* FeatureExtraction::fftProc [protected]

Referenced by createMfccFrame(), FeatureExtraction(), and ~FeatureExtraction().

float FeatureExtraction::fftWindowSpectralSubtract[FEA_FREQWINDOWSIZE] [protected]

Referenced by createMfccFrame().

Vector** FeatureExtraction::gaussianization [protected]

Referenced by FeatureExtraction(), and setNormalization().

float FeatureExtraction::hammingTemplate[FEA_WINDOWSIZE] [protected]

Referenced by createMfccFrame(), and FeatureExtraction().

Vector** FeatureExtraction::inverseHistogram

Referenced by createMfccFrame(), FeatureExtraction(), performHistNormUntilFrame(), setNormalization(), and ~FeatureExtraction().

int FeatureExtraction::MEL_BANKLENGTH [protected]

Referenced by createMfccFrame(), FeatureExtraction(), setVTLN(), and storeFeatureVectors().

int FeatureExtraction::melIndexAboveCTS [protected]

Referenced by createMfccFrame(), and setVTLN().

int FeatureExtraction::melTemplate[LARGE_MEL_BANKLENGTH+2] [protected]

Referenced by createMfccFrame(), and setVTLN().

int FeatureExtraction::mfccSize [protected]

Referenced by createFeaturesUntilFrame(), createMfccFrame(), FeatureExtraction(), finishClusterCMVN(), and finishExtraction().

int FeatureExtraction::nonDeltaSize [protected]

Referenced by calculateDelta(), doFeatureExtraction(), FeatureExtraction(), and finishExtraction().

NormData* FeatureExtraction::normData [protected]

Referenced by createFeaturesUntilFrame(), createMfccFrame(), FeatureExtraction(), finishClusterCMVN(), performClusterCMVNUntilFrame(), performHistNormUntilFrame(), setHistNormModel(), setNormalization(), and ~FeatureExtraction().

FEATURENORM_TYPE FeatureExtraction::normType [protected]

Referenced by createFeaturesUntilFrame(), FeatureExtraction(), finishClusterCMVN(), finishExtraction(), and setNormalization().

int FeatureExtraction::nrFrames [protected]

Referenced by doFeatureExtraction(), FeatureExtraction(), finishExtraction(), getPoolSize(), performPCA(), setNormalization(), storeFeatureVectors(), and ~FeatureExtraction().

int FeatureExtraction::nrNormClusters [protected]

Referenced by finishClusterCMVN(), setNormalization(), and ~FeatureExtraction().

int FeatureExtraction::numberOfFeaturesProcessed [protected]

Referenced by createFeaturesUntilFrame(), doFeatureExtraction(), finishClusterCMVN(), finishExtraction(), getOnlineVector(), initializeHistNorm(), performClusterCMVNUntilFrame(), and performHistNormUntilFrame().

bool FeatureExtraction::offset [protected]

Referenced by FeatureExtraction().

bool FeatureExtraction::onlyMEL [protected]

Referenced by createMfccFrame(), FeatureExtraction(), setOnlyMEL(), and storeFeatureVectors().

bool FeatureExtraction::performCTS [protected]

Referenced by FeatureExtraction(), and setVTLN().

short int FeatureExtraction::rememberBuffer[FEA_REMEMBERWINSIZE] [protected]

Referenced by doFeatureExtraction(), and readAudioWindow().

int FeatureExtraction::useDeltas [protected]

Referenced by doFeatureExtraction(), FeatureExtraction(), finishExtraction(), and getOnlineVector().

int FeatureExtraction::useEnergy [protected]

Referenced by createFeaturesUntilFrame(), createMfccFrame(), FeatureExtraction(), finishClusterCMVN(), and finishExtraction().

int FeatureExtraction::useZeroCross [protected]

Referenced by createFeaturesUntilFrame(), createMfccFrame(), FeatureExtraction(), finishClusterCMVN(), and finishExtraction().

int FeatureExtraction::vectorSize [protected]

Referenced by createMfccFrame(), doFeatureExtraction(), FeatureExtraction(), getVectorSize(), and performPCA().