The Search Query Language

When you search a Verity collection, you use the CFSEARCH tag in a Cold Fusion application page. Use the CRITERIA attribute to specify the query expression you want to pass to the search engine. You can build two types of query expressions: simple and explicit. A simple query expression is typically a word or words. An explicit query expression can employ a number of operators and modifiers to refine the search. Although an explicit query can employ operators and modifiers, all aspects of the search must be explicitly invoked. A simple query expression is somewhat more powerful since it employs operators by default. You can assemble an explicit query expression programmatically or simply pass a simple query expression to the search engine directly from an HTML input form.

The Verity query language provides many operators and modifiers for composing queries. The following search techniques can be used in searching a Verity collection:

• Word searches
• Proximity searches
• Concept-based
• Field searches in which documents are match based on matching predefined custom attributes
• Scoring operators

Simple query expressions

Simple queries allow end users to enter simple, comma-delimited strings and use wildcard characters. By default, a simple query searches for words, not strings. For example, entering the word "All" will find documents containing the word "all" but not "allegorical." You can use wildcards, however to broaden the scope of the search. "All*" will return documents containing both "all" and "alliterate." Case is ignored.

You can enter multiple words separated by commas: software, Microsoft, Oracle. The comma in a Simple query expression is treated like a logical OR. If you omit the commas, the query expression is treated as a phrase, so documents would be searched for the phrase "software Microsoft Oracle."

Ordinarily, operators are employed in explicit query expressions. Operators are normally surrounded by angle brackets < >. However, you can use the AND, OR, and NOT operators in a simple query without using angle brackets: software AND (Microsoft OR Oracle). To include an operator in a search, you surround it with double quotation marks: software "and" Microsoft. This expression searches for the phrase "software and Microsoft."

A simple query employs the STEM operator and the MANY modifier. STEM searches for words that derive from those entered in the query expression, so that entering "find" will return documents that contain "find," "finding," "finds," etc. The MANY modifier forces the documents returned in the search to be presented in a list based on a relevancy score.

Explicit query expressions

Explicit queries can be constructed using a variety of operators, including evidence, proximity, relational, concept, and score operators. Most operators in an explicit query expression are surrounded by angle brackets < >. You can use the AND, OR, and NOT operators without angle brackets.

Simple and explicit syntax

You can use either simple or explicit syntax when stating simple query syntax. The syntax you use determines whether the search words you enter will be stemmed, and whether the words that are found will contribute to relevance-ranked scoring.

Simple syntax

When you use simple syntax, the search engine implicitly interprets single words as if they were modified by the MANY and STEM operators. By implicitly applying the MANY modifier, the search engine calculates each document’s score based on the density of the search term in the searched documents. The more frequent the occurrence of a word in a document, the higher the document's score.

As a result, the search engine ranks documents according to word density as it searches for the word you specify, as well as words that have the same stem. For example, "films," "filmed," and "filming" are stemmed variations of the word "film." To search for documents containing the word "film" and its stem words, you can enter the word "film" without modification. When documents are ranked by relevance, they appear in a list with the most relevant documents at the top.

Explicit syntax

When you use explicit syntax, the search engine interprets the search terms you enter as literals. For example, by entering the word "film" (including quotation marks) using explicit syntax, the stemmed versions of the word "film", "films," "filmed," and "filming" are ignored.

Operator summary

An operator represents logic to be applied to a search element. This logic defines the qualifications a document must meet to be retrieved. Operator types are as follows:

• Wildcards
• Evidence operators
• Proximity operators
• Relational operators
• Concept operators
• Score operators

Ordinarily, you use operators in explicit searches. They are used in the following manner:

"<operator>search_string"

Search operations

The following table shows all operators available for conducting searches of Cold Fusion Verity collections.

Query expressions

Query expressions pass to the search engine in the CRITERIA attribute of the CFSEARCH tag. Expressions are assembled with a combination of search words, operators, and modifiers.

Special characters

A number of characters are handled in particular ways by the search engine.

A backslash (\) removes special meaning from whatever character follows it. To enter a literal backslash in a query, use two in succession. Examples:

<FREETEXT>("\"Hello\", said Packard.")
"backslash (\\)"

Precedence evaluation

The following rules apply for composing search expressions.

Precedence rules

While an expression is read from left to right, some operators carry more weight than others. For example, AND operators take precedence over OR operators. To ensure that an OR operator is interpreted prior to an AND operator, you can use parentheses to enclose the OR operator:

(a OR b) AND c

Terms enclosed by parentheses are read first.

There must be at least one space between operators and words used in the expression.

When the search engine encounters nested parentheses, it starts with the innermost term:

(a AND (b OR c)) OR d

This expression means: Look for documents that contain b or c as well as a, or that contain d.

Prefix and infix notation

Search strings that use any operator other than evidence operators can be defined in prefix notation or infix notation.

Prefix notation specifies that the operator comes before the search string:

AND (a,b)

When prefix notation is used, precedence is handled explicitly within the expression. The following example means: "Look for documents that contain b and c first, then documents that contain a":

OR (a, AND (b,c))

Infix notation specifies that the operator is to be specified between each term within the expression. The following example means: "Look for documents that contain a and b or documents that contain c":

a AND b OR c

When infix notation is used, precedence is implicit in the expression. For example, the AND operator takes precedence over the OR operator.

Commas in expressions

If an expression includes two or more search terms within parentheses, a comma is required as a separator between each element. The following example means: Look for documents that contain any combination of a and b together. Note that in this example, angle brackets are used with the OR operator.

<OR> (a, b)

Delimiters in expressions

Angle brackets < >, double quotation marks " ", and backslashes \ are used to delimit various elements in a query expression.

Angle brackets for operators

Left and right angle brackets < > are reserved for designating operators and modifiers. They are optional for the AND, OR, and NOT operators, but required for all other operators.

Double quotation marks in expressions

You use double quotation marks to search for a word that is otherwise reserved as an operator, such as AND, OR, and NOT.

Backslashes in expressions

To include a backslash \ in a search, insert two backslashes for each backslash character you want to search for:

C:\\CFUSION\\BIN

Wildcards

The following wildcard characters are available for searching Verity collections:

Searching for wildcards as literals

To search for a wildcard character in your collection, you need to escape the character with a backslash (\). For example:

To match a literal asterisk, you precede the * with two backslashes: "a\\*"

To match a question mark or other wildcard character: "Checkers\?"

Searching for special characters as literals

The following non-alphanumeric characters must be preceded by a backslash character (\) in a search string:

• comma (,)
• left and right parentheses ( )
• Double quotation mark (")
• backslash (\)
• at sign (@)
• left curly brace ({)
• left bracket ([)
• less than sign (<)
• backquote (`)

In addition to the backslash character, you can use paired backquotes (` `) to interpret special characters as literals. For example, to search for the wildcard string "a{b" you can surround the string with backquotes, as follows:

`a{b`

To search for a wildcard string that includes the literal backquote character (`) you must use two backquotes together and surround the whole string in backquotes:

`*n``t`

Note that you can use either paired backquotes or backslashes to escape special characters. There is no functional difference in the use of one or the other. For example, you can query for the term: <DDA> in the following ways:

\<DDA\> or `<DDA>`

Evidence operators

Evidence operators can be used to specify either a basic word search or an intelligent word search. A basic word search finds documents that contain only the word or words specified in the query. An intelligent word search expands the query terms to create an expanded word list so that the search returns documents that contain variations of the query terms.

Documents retrieved using evidence operators are not ranked by relevance unless you use the MANY modifier.

<STEM>believe

spam*

Proximity operators

Proximity operators specify the relative location of specific words in the document. Specified words must be in the same phrase, paragraph, or sentence for a document to be retrieved. In the case of NEAR and NEAR/N operators, retrieved documents are ranked by relevance based on the proximity of the specified words. Proximity operators can be nested; phrases or words can appear within SENTENCE or PARAGRAPH operators, and SENTENCE operators can appear within PARAGRAPH operators.

The following table describes each operator:

commute <NEAR/10> bicycle <NEAR/10>
train <NEAR/10>

mission oak
"mission oak"
mission <PHRASE> oak
<PARAGRAPH> (mission, oak)

jazz <SENTENCE> musician
<SENTENCE> (jazz, musician)

Relational operators

Relational operators search document fields that have been defined in the collection. Documents containing specified field values are returned. Documents retrieved using relational operators are not ranked by relevance, and you cannot use the MANY modifier with relational operators.

There are two types of relational operators to perform numeric and date comparisons. Text comparison operators match words and parts of words.

Numeric and date relational operators

The following operators are used for numeric and date comparisons.

Text comparison operators

The following operators are used for text comparisons.

SUBSTRING example

You can use the SUBSTRING operator to match a character string with data stored in a specified data source. In the following example, a data source called TEST1 contains the table YearPlaceText, which itself contains three columns: Year, Place, and Text. Year and Place make up the primary key. This is what the table looks like:

The following application page matches records that have 1990 in the TEXT column and are in the Place Utah. The search is performed against the collection that contains the TEXT column and then is narrowed further by searching the string "Utah" in the CF_TITLE document field. Recall that document fields are defaults defined in every collection corresponding to the values you define for URL, TITLE, and KEY in the CFINDEX tag.

<CFQUERY NAME="GetText" DATASOURCE="TEST1">
   Select Year+Place AS Identifier, text from YearPlaceText
</CFQUERY>
 
<CFINDEX COLLECTION="testcollection"
   ACTION="UPDATE"
   TYPE="CUSTOM"
   TITLE="Identifier"
   KEY="Identifier"
   BODY="TEXT"
   QUERY="GetText">
 
<CFSEARCH NAME="GetText_Search"
   COLLECTION="testcollection"
   TYPE="EXPLICIT"
   CRITERIA="1990 and CF_TITLE <SUBSTRING> Utah" >
 
<CFOUTPUT> 
   Record Counts: <br>
   #GetText.RecordCount# <BR>
   #GetText_Search.RecordCount# <BR>
</CFOUTPUT>
 
<CFOUTPUT> 
   Query Results --- Should be 5 rows <BR> 
</CFOUTPUT>
 
<CFOUTPUT QUERY="Gettext">
   #Identifier# <BR>
</CFOUTPUT>
 
<CFOUTPUT> 
   Search Results -- should be 1 row <BR> 
</CFOUTPUT>
 
<CFOUTPUT QUERY="GetText_Search">
   #GetText_Search.TITLE# <BR>
</CFOUTPUT>

Document fields

There are three document fields predefined in any Cold Fusion Verity collection you create and populate:

• CF_KEY — Defined in the KEY attribute of the CFINDEX tag
• CF_TITLE — Defined in the TITLE attribute of the CFINDEX tag
• CF_URL — Defined in the URLPATH attribute of the CFINDEX tag (where applicable)

Document fields are referenced in text comparison operators. These fields can contain alphanumeric characters. You define these fields when you generate a Verity collection using the CFINDEX tag.

Concept operators

Concept operators combine the meaning of search elements to identify a concept in a document. Documents retrieved using concept operators are ranked by relevance. The following table describes each concept operator.

Score operators

Score operators govern how the search engine calculates scores for retrieved documents. The maximum score a returned search element can have is 1. When a score operator is used, the search engine first calculates a separate score for each search element found in a document, and then performs a mathematical operation on the individual element scores to arrive at the final score for each document.

Note that the document's score is available as a result column. The SCORE result column can be referenced to trap the relevancy score of any document retrieved. For example:

<CFOUTPUT>
   <A HREF="#Search1.URL#">#Search1.Title#</A><BR>
   Document Score=#Search1.SCORE#<BR>
</CFOUTPUT>

<YESNO>mainframe

<PRODUCT>(computers, laptops)

<SUM>(computers, laptops)

<COMPLEMENT>computers

Search Modifiers

Modifiers are combined with operators to change the standard behavior of an operator in some way. For example, you can use the CASE modifier with an operator to specify that you want to match the case of the search word.

Modifiers are as follows:

<PARAGRAPH><MANY>javascript <AND> vbscript

Java <AND> programming <NOT> coffee

<ORDER><PARAGRAPH>("server", "Java")