org.apache.accumulo.examples.wikisearch.logic
Class AbstractQueryLogic

java.lang.Object
  extended by org.apache.accumulo.examples.wikisearch.logic.AbstractQueryLogic
Direct Known Subclasses:
QueryLogic

public abstract class AbstractQueryLogic
extends Object

 

Overview

Query implementation that works with the JEXL grammar. This uses the metadata, global index, and partitioned table to return results based on the query. Example queries: Single Term Query 'foo' - looks in global index for foo, and if any entries are found, then the query is rewritten to be field1 == 'foo' or field2 == 'foo', etc. This is then passed down the optimized query path which uses the intersecting iterators on the partitioned table. Boolean expression field == 'foo' - For fielded queries, those that contain a field, an operator, and a literal (string or number), the query is parsed and the set of eventFields in the query that are indexed is determined by querying the metadata table. Depending on the conjunctions in the query (or, and, not) and the eventFields that are indexed, the query may be sent down the optimized path or the full scan path. We are not supporting all of the operators that JEXL supports at this time. We are supporting the following operators: ==, !=, >, ≥, <, ≤, =~, and !~ Custom functions can be created and registered with the Jexl engine. The functions can be used in the queries in conjunction with other supported operators. A sample function has been created, called between, and is bound to the 'f' namespace. An example using this function is : "f:between(LATITUDE,60.0, 70.0)"

Constraints on Query Structure

Queries that are sent to this class need to be formatted such that there is a space on either side of the operator. We are rewriting the query in some cases and the current implementation is expecting a space on either side of the operator. If an error occurs in the evaluation we are skipping the event.

Notes on Optimization

Queries that meet any of the following criteria will perform a full scan of the events in the partitioned table: 1. An 'or' conjunction exists in the query but not all of the terms are indexed. 2. No indexed terms exist in the query 3. An unsupported operator exists in the query


Nested Class Summary
static class AbstractQueryLogic.IndexRanges
          Object that is used to hold ranges found in the index.
static class AbstractQueryLogic.UnionIndexRanges
          Object that computes the ranges by unioning all of the ranges for all of the terms together.
 
Field Summary
static String DATATYPE_FILTER_SET
          Set of datatypes to limit the query to.
protected static org.apache.log4j.Logger log
           
 
Constructor Summary
AbstractQueryLogic()
           
 
Method Summary
 Document createDocument(Key key, Value value)
           
protected  Map<String,com.google.common.collect.Multimap<String,Class<? extends Normalizer>>> findIndexedTerms(Connector c, Authorizations auths, Set<String> queryLiterals, Set<String> datatypes)
          Queries metadata table to determine which terms are indexed.
protected abstract  Collection<Range> getFullScanRange(Date begin, Date end, com.google.common.collect.Multimap<String,QueryParser.QueryTerm> terms)
           
 String getIndexTableName()
           
 String getMetadataTableName()
           
 int getQueryThreads()
           
 String getReadAheadQueueSize()
           
 String getReadAheadTimeOut()
           
 String getResultsKey(Map.Entry<Key,Value> key)
           
 String getReverseIndexTableName()
           
 String getTableName()
           
protected abstract  RangeCalculator getTermIndexInformation(Connector c, Authorizations auths, com.google.common.collect.Multimap<String,Normalizer> indexedTerms, com.google.common.collect.Multimap<String,QueryParser.QueryTerm> terms, String indexTableName, String reverseIndexTableName, String queryString, int queryThreads, Set<String> datatypes)
          Performs a lookup in the global index / reverse index and returns a RangeCalculator
protected abstract  AbstractQueryLogic.IndexRanges getTermIndexInformation(Connector c, Authorizations auths, String value, Set<String> datatypes)
          Performs a lookup in the global index for a single non-fielded term.
 List<String> getUnevaluatedFields()
           
 boolean isUseReadAheadIterator()
           
 Results runQuery(Connector connector, List<String> authorizations, String query, Date beginDate, Date endDate, Set<String> types)
           
 void setIndexTableName(String indexTableName)
           
 void setMetadataTableName(String metadataTableName)
           
 void setQueryThreads(int queryThreads)
           
 void setReadAheadQueueSize(String readAheadQueueSize)
           
 void setReadAheadTimeOut(String readAheadTimeOut)
           
 void setReverseIndexTableName(String reverseIndexTableName)
           
 void setTableName(String tableName)
           
 void setUnevaluatedFields(List<String> unevaluatedFields)
           
 void setUnevaluatedFields(String unevaluatedFieldList)
           
 void setUseReadAheadIterator(boolean useReadAheadIterator)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

log

protected static org.apache.log4j.Logger log

DATATYPE_FILTER_SET

public static final String DATATYPE_FILTER_SET
Set of datatypes to limit the query to.

See Also:
Constant Field Values
Constructor Detail

AbstractQueryLogic

public AbstractQueryLogic()
Method Detail

findIndexedTerms

protected Map<String,com.google.common.collect.Multimap<String,Class<? extends Normalizer>>> findIndexedTerms(Connector c,
                                                                                                              Authorizations auths,
                                                                                                              Set<String> queryLiterals,
                                                                                                              Set<String> datatypes)
                                                                                                       throws TableNotFoundException,
                                                                                                              InstantiationException,
                                                                                                              IllegalAccessException
Queries metadata table to determine which terms are indexed.

Parameters:
c -
auths -
queryLiterals -
datatypes - - optional list of types
Returns:
map of indexed field names to types to normalizers used in this date range
Throws:
TableNotFoundException
IllegalAccessException
InstantiationException

getTermIndexInformation

protected abstract AbstractQueryLogic.IndexRanges getTermIndexInformation(Connector c,
                                                                          Authorizations auths,
                                                                          String value,
                                                                          Set<String> datatypes)
                                                                   throws TableNotFoundException
Performs a lookup in the global index for a single non-fielded term.

Parameters:
c -
auths -
value -
datatypes - - optional list of types
Returns:
ranges that fit into the date range.
Throws:
TableNotFoundException

getTermIndexInformation

protected abstract RangeCalculator getTermIndexInformation(Connector c,
                                                           Authorizations auths,
                                                           com.google.common.collect.Multimap<String,Normalizer> indexedTerms,
                                                           com.google.common.collect.Multimap<String,QueryParser.QueryTerm> terms,
                                                           String indexTableName,
                                                           String reverseIndexTableName,
                                                           String queryString,
                                                           int queryThreads,
                                                           Set<String> datatypes)
                                                    throws TableNotFoundException,
                                                           org.apache.commons.jexl2.parser.ParseException
Performs a lookup in the global index / reverse index and returns a RangeCalculator

Parameters:
c - Accumulo connection
auths - authset for queries
indexedTerms - multimap of indexed field name and Normalizers used
terms - multimap of field name and QueryTerm object
indexTableName -
reverseIndexTableName -
queryString - original query string
queryThreads -
datatypes - - optional list of types
Returns:
range calculator
Throws:
TableNotFoundException
org.apache.commons.jexl2.parser.ParseException

getFullScanRange

protected abstract Collection<Range> getFullScanRange(Date begin,
                                                      Date end,
                                                      com.google.common.collect.Multimap<String,QueryParser.QueryTerm> terms)

getMetadataTableName

public String getMetadataTableName()

getIndexTableName

public String getIndexTableName()

getTableName

public String getTableName()

setMetadataTableName

public void setMetadataTableName(String metadataTableName)

setIndexTableName

public void setIndexTableName(String indexTableName)

setTableName

public void setTableName(String tableName)

getQueryThreads

public int getQueryThreads()

setQueryThreads

public void setQueryThreads(int queryThreads)

getReadAheadQueueSize

public String getReadAheadQueueSize()

getReadAheadTimeOut

public String getReadAheadTimeOut()

isUseReadAheadIterator

public boolean isUseReadAheadIterator()

setReadAheadQueueSize

public void setReadAheadQueueSize(String readAheadQueueSize)

setReadAheadTimeOut

public void setReadAheadTimeOut(String readAheadTimeOut)

setUseReadAheadIterator

public void setUseReadAheadIterator(boolean useReadAheadIterator)

getReverseIndexTableName

public String getReverseIndexTableName()

setReverseIndexTableName

public void setReverseIndexTableName(String reverseIndexTableName)

getUnevaluatedFields

public List<String> getUnevaluatedFields()

setUnevaluatedFields

public void setUnevaluatedFields(List<String> unevaluatedFields)

setUnevaluatedFields

public void setUnevaluatedFields(String unevaluatedFieldList)

createDocument

public Document createDocument(Key key,
                               Value value)

getResultsKey

public String getResultsKey(Map.Entry<Key,Value> key)

runQuery

public Results runQuery(Connector connector,
                        List<String> authorizations,
                        String query,
                        Date beginDate,
                        Date endDate,
                        Set<String> types)


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.