cz.muni.fi.mias.math

Class MathTokenizer

java.lang.Object

org.apache.lucene.util.AttributeSource

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.Tokenizer

cz.muni.fi.mias.math.MathTokenizer

All Implemented Interfaces:

Closeable, AutoCloseable

public class MathTokenizer
extends org.apache.lucene.analysis.Tokenizer

Implementation of Lucene Tokenizer. Provides math formulae contained in the input as string tokens and their weight. These attributes are held in TermAttribute and PayloadAttribute and carried over the stream.

Since:

14.5.2010

Author:

Martin Liska

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MathTokenizer.MathMLType

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

Field Summary

Fields inherited from class org.apache.lucene.analysis.Tokenizer

input

Constructor Summary

Constructors

Constructor and Description
MathTokenizer(Reader input, boolean subformulae, String mmldtd, MathTokenizer.MathMLType type)

Method Summary

Methods

Modifier and Type	Method and Description
Map<Integer,List<Formula>>	getFormulae()
Map<String,Float>	getQueryFormulae()
boolean	incrementToken()
static void	printFormulaeCount() Prints numbers of processed formulae to standard output
void	reset()
void	reset(Reader input)
void	setFormulaPosition(int formulaPosition) Overrides the position attribute for all processed formulae

Methods inherited from class org.apache.lucene.analysis.Tokenizer

close, correctOffset

Methods inherited from class org.apache.lucene.analysis.TokenStream

end

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

MathTokenizer

public MathTokenizer(Reader input,
             boolean subformulae,
             String mmldtd,
             MathTokenizer.MathMLType type)

Parameters:

input - Reader containing the input to process

subformulae - if true, subformulae will be extracted

mmldtd - Path to MathML DTD file

type - type of MathML that should be processed

Method Detail

setFormulaPosition

public void setFormulaPosition(int formulaPosition)

Overrides the position attribute for all processed formulae

Parameters:

formulaPosition - Position number to be used for all processed formulae

incrementToken

public boolean incrementToken()

Specified by:

incrementToken in class org.apache.lucene.analysis.TokenStream

reset

public void reset()
           throws IOException

Overrides:

reset in class org.apache.lucene.analysis.TokenStream

Throws:

IOException

reset

public void reset(Reader input)
           throws IOException

Overrides:

reset in class org.apache.lucene.analysis.Tokenizer

Throws:

IOException

getQueryFormulae

public Map<String,Float> getQueryFormulae()

Returns:

Processed formulae to be used for a query. No subformulae are extracted.

printFormulaeCount

public static void printFormulaeCount()

Prints numbers of processed formulae to standard output

getFormulae

public Map<Integer,List<Formula>> getFormulae()

Returns:

A map with formulae as if they are indexed. Key of the map is the original document position of the extracted formulae contained in the value of the map.