Monday, October 1, 2012

What’s New In SharePoint 2013 Search (Developers Perspective) Part Three

Technorati Tags: ,,

This is my third post in a series which is covering the new capabilities in SharePoint 2013 Search and how you can leverage them through the server API. In the last post I wrote about the new SearchExecutor class and how it can be used to execute multiple queries. The reasoning behind putting this new class into SharePoint Search was to enable the federation of multiple searchable sources into one search experience. The ability to easily set up remote SharePoint farms as search sources makes federation useful for searching across large geographically distributed farms. In this post I will write about the new search Query Rules that are available in SharePoint 2013. The new Query Rule is an incredibly powerful tool which enables the manipulation of search results when a search is executed. Query rules encompass three components, result sources, conditions and actions. First I will show you the different types of conditions that can be used to trigger actions. Second I will show you the relationship between result sources and query rules. Finally I will show what kind of actions can be triggered by query rule conditions and how they affect the search results the user sees. As a developer you will interested in knowing how to interpret the query rule actions using the object model so you can leverage these in your own solutions. As always search entities can defined at the search service application and the site collection level, this also applies to query rules.

 

Query Rule Conditions

Query rule conditions are the conditions that will trigger any query rule actions that take place when a search is executed. A rule can have multiple conditions and can only be “OR” together. The conditions can be as simple as matching a keyword term or as sophisticated as the query matching on a regular expression. The wide range allows you tailor the search results on the intent and audience of the search user rather just matching keyword terms.  Below is a brief description on what each type of condition can do:

Condition Type Description Example
Query Matches Keyword Exactly If the query matches any of the semi-colon separated list of phrases then the rule is triggered Query= “Document”

Document;Invoice;Purchase Order
Query Matches Action Term Triggered when the query contains verbs or commands that match a semi-colon separated list of phrases or a taxonomy dictionary. Query= “Downloaded documents from Saint Louis”

Downloaded;Saint Louis;Approved
Query Matches Dictionary Exactly Triggered when the query matches a dictionary entry exactly. The dictionary can be an imported taxonomy such as peoples names. Query=”John Doe”
Query Commonly Used in Source A popular query that has been logged numerous times for a given result source. You can choose which result source.  
Result Type Commonly Clicked If the results from the query contain result types that have been clicked on and opened for viewing numerous times, then the rule is triggered. Email, Microsoft Office files and PDF
Advanced Query Text Match Most powerful of all conditions. Includes query matching on a regular expression, semi-colon separated list of phrases or an entry in a taxonomy dictionary. Depending on which you choose additional options are included. For example, exact query match, query starts with or ends with. The matching can be applied to either the subject terms or the action terms in the query. Query=”Legal documents” Microsoft

Triggered when starts with Legal

 

Additional restrictions when a rule is triggered can be targeted to certain topic pages or user segments (Audiences). These restrictions can be set in the “Show more conditions” section.

Result Sources

Query rules can be assigned to multiple result sources or all result sources. I explained what result sources are in my previous posts. The association between sources and rules is required in order for the search engine to evaluate triggered rules for a given query. Any  query submitted by the user can only be associated with one source. This is accomplished by an administrator setting what result source a search web part will be associated with or a developer setting the SourceID property of the KeywordQuery class. When the query is submitted to the query server, the query server checks which result source the query is searching. Next the query server checks to see if query rules should be evaluated. If rules are not being evaluated then the query server executes the query and returns the results with no further action, else the conditions of query rules associated with the result source are evaluated. If conditions are met, then the actions associated with the conditions are executed and any additional results are combined and returned with original search results. Below is a flow of how queries, result sources and rules work together:

 

Query Rule Actions

The three types of query rule actions are Promoted Results, Result Blocks and Change rank results by changing the query. The query rule actions is where most of the power of query rules resides. As a developer you will want to know what the triggered rules are for a given query. You can then interrogate the triggered rules in order to know how to access and display any additional results. There are three types of actions. I am going to describe these and show some code on how to access them using  the server search API. Below is a simple execution of a KeywordQuery using the new SearchExecutor class. Your first step is to access the triggered rules for the search by using the returned ResultTableCollection’s TriggeredRules collection. This is a collection of query rule GUIDS.

public static DataTable ExecuteKeyWordSearch(string queryText)
{
    ResultTableCollection rtc = null;
    DataTable retResults = new DataTable();

    using (SPSite site = new SPSite("http://basesmc15"))
    {
        using (KeywordQuery query = new KeywordQuery(site))
        {
            query.QueryText = queryText;
            query.KeywordInclusion = KeywordInclusion.AllKeywords;
            query.RowLimit = 500;
            query.SelectProperties.Add("Path");
            query.SelectProperties.Add("IsDocument");
            query.EnableQueryRules = true;

            SearchExecutor se = new SearchExecutor();

            rtc = se.ExecuteQuery(query);

           List<Guid> triggeredRuleIds = rtc.TriggeredRules;


            if (rtc.Count > 0)
            {

                var results = rtc.Filter("TableType", KnownTableTypes.RelevantResults);
                if (results != null && results.Count() > 0)
                {
                    foreach (ResultTable result in results)
                    {
                        retResults.Load(result, LoadOption.OverwriteChanges);
                    }

                }

            }

        }

    }

    return retResults;

}

Once you have retrieved the triggered rule IDs for the query you will need to get the associated QueryRule objects. The code below shows how to use the QueryRuleManager to obtain the query rules triggered from the search. Please note this code only searches for query rules defined at the search service application level.

public static List<QueryRule> GetQueryTriggeredRules(List<Guid> queryRuleIds)
{
    SPServiceContext serviceContext =
    SPServiceContext.GetContext(SPServiceApplicationProxyGroup.Default,
    SPSiteSubscriptionIdentifier.Default);

    var searchProxy =
        serviceContext.GetDefaultProxy(typeof(SearchServiceApplicationProxy))
        as SearchServiceApplicationProxy;

    QueryRuleManager qrm = new QueryRuleManager(searchProxy);
    SearchObjectFilter filter = new SearchObjectFilter(SearchObjectLevel.Ssa);
    filter.IncludeLowerLevels = true;

    QueryRuleCollection rules = qrm.GetQueryRules(filter);

    var triggeredRules = from r in rules join sourceIds
                             in queryRuleIds on r.Id equals sourceIds select r;

    return triggeredRules.ToList();
}

 

Promoted Result Actions

A promoted result action is very similar to the “Best Bet” capability in SharePoint 2010. It gives you the ability to promote URLs to the top of the search results. The only difference in SharePoint 2013 is that the promoted results can be defined based on the more powerful query rule conditions, whereas in SharePoint 2010 you were limited to a particular keyword. As a developer you will be interested in getting the title, URL and description of the promoted result. In order to get at the returned results of a promoted result action you will need the ID of the query rule. The code below will check the triggered rules for promoted results and then access the promoted results using a linq query against the ResultTableCollection where the QueryRuleId property of the ResultTable matches the ID of the promoted result query rule. Normally you should use the built-in Filter method of the ResultTableCollection but it has a bug where it will not work using a GUID or object property. The Title, URL, Description and whether the promoted result should be rendered as a banner instead of a hyperlink is contained in the IsVisualBestBet value. You can get the appropriate rendering template from the RenderTemplateId value. Make sure to render the promoted results in the order they are returned.

SearchExecutor se = new SearchExecutor();
   rtc = se.ExecuteQuery(query);

   List<Guid> triggeredRuleIds = rtc.TriggeredRules;

   var promotedResultRules = from r in GetQueryQueryRules(triggeredRuleIds)
                                       where r.AssignBestBetsAction != null select r;

   if (promotedResultRules != null && promotedResultRules.Count() > 0)
   {
       var results = from r in rtc join pr in promotedResultRules on r.QueryRuleId equals pr.Id select r;

       if (results != null && results.Count() > 0)
       {
           foreach (ResultTable result in results)
           {
               retResults.Load(result, LoadOption.OverwriteChanges);
           }

           foreach (DataRow dr in retResults.Rows)
           {
               string title = dr["Title"].ToString();
               string url = dr["URL"].ToString();
               string description = dr["Description"].ToString();

               //if you should render the promoted result as a banner
               bool isVisualBestBet = dr["IsVisualBestBet"] != null ?
                   bool.Parse(dr["IsVisualBestBet"].ToString()): false;

              //template to render the promoted result
               string template = dr["RenderTemplateId"].ToString();
           }

       }

   }

Change ranked results by changing the query action

The “Change ranked results by changing the query action” action  is the most powerful action. Only one can be defined per query rule. Based on conditions this action can substitute or append another query and boost the rankings of other results.  As a developer you don’t need to know much about the effects of this action. However since this action can fundamentally change the results of the query you may want to inform the user what changes were made. For example, if the action has changed the sorting and any type rank reordering. All the information you need will be contained in the query rule’s ChangeQueryAction object and it’s QueryTranform property. The code below get query rules that have a ChangedQueryAction and then accesses the QueryTransform.OverrideProperties collection to obtain information on sorting and rank boosting.

SearchExecutor se = new SearchExecutor();
rtc = se.ExecuteQuery(query);

List<Guid> triggeredRuleIds = rtc.TriggeredRules;

var changedQueryRules = from r in GetQueryQueryRules(triggeredRuleIds)
                            where r.ChangeQueryAction != null
                            select r;

if (changedQueryRules != null && changedQueryRules.Count() > 0)
{
    foreach (QueryRule rule in changedQueryRules)
    {
        ChangeQueryAction action = rule.ChangeQueryAction;
        SortCollection sorts = action.QueryTransform.OverrideProperties["SortList"] as SortCollection;

        foreach (Sort sort in sorts)
        {
            string sortProp = sort.Property;
            SortDirection sortDirection = sort.Direction;
        }

        ReorderingRuleCollection rankRules =
            action.QueryTransform.OverrideProperties["RankRules"] as ReorderingRuleCollection;

        foreach (ReorderingRule rankRule in rankRules)
        {
            string matchValue = rankRule.MatchValue;
            ReorderingRuleMatchType matchType = rankRule.MatchType;
            int boostValue = rankRule.Boost;
        }
    }

}

Result Block Actions

The result block actions are similar to promoted results. It is like “Best Bets” but uses additional  blocks of results instead of titles and URLs. Your solution may be interested in knowing the title, title language, what source the result block originated from, “More Link Behavior” (URL for more results), whether to always display the result block above the core results, group display template, item display template and the label for routing to a content search web part. The difficult part comes when you want to determine which ResultTable belongs to which CreateResultBlockAction. Remember that you can have multiple of these actions associated with one query rule. There seems to be no way to select the corresponding ResultTable from the returned ResultTableCollection. I tried selecting on the ResultTile but the action’s ResultTitle.DefaulLanguageString will not match since it uses the place holder “{subjectTerms}” which holds the keywords from the query. You can modify this in the UI to make sure they all have unique titles, however that could be very unreliable. I also tried using the  routing label but this of course cannot be guaranteed to be unique. Better approach would be to add the acton’s ID to the ResultTable.

SearchExecutor se = new SearchExecutor();
rtc = se.ExecuteQuery(query);

List<Guid> triggeredRuleIds = rtc.TriggeredRules;

var blockResultRules = from r in GetQueryQueryRules(triggeredRuleIds)
                            where r.CreateResultBlockActions != null
                            select r;

var blockResults = from r in rtc join br in blockResultRules
                   on r.QueryRuleId equals br.Id select r;

if (blockResultRules != null && blockResultRules.Count() > 0)
{
    foreach (QueryRule qr in blockResultRules)
    {
        foreach (CreateResultBlockAction crba in qr.CreateResultBlockActions)
        {
            //always show results on top
            bool showOnTop = crba.AlwaysShow;

            //template ids
            string grpTemplateId = crba.GroupTemplateId;
            string itemTemplateId = crba.ItemTemplateId;

            //title and what is the default language for the title
            MultiLingualString<CreateResultBlockAction> titleInfo = crba.ResultTitle;
            int defalutLanguageId = titleInfo.DefaultLanguageLcid;
            string defaultTitle = titleInfo.DefaultLanguageString;

            //link for more results
            string showMoreResultsLink = crba.ResultTitleUrl;
           //label for a content search web part
            string searchLabel = crba.ResultTableType;

            //id to the source of the block results
            Guid? blockResultSource = crba.QueryTransform.SourceId;

            //MISSING which resulttable belongs to which CreateResultBlockAction?
            var blockResultForAction = from r in blockResults where r.ResultTitle == defaultTitle select r;
        }             
    }

}

Query Rules are the Rubik’s Cube of SharePoint 2013 Search

The new SharePoint 2013 Search query rules add a great deal of extensibility to search. The ability to add numerous types of conditions and actions that can pull data from multiple sources into one search experience is a big step forward from SharePoint 2010 search. Combining query rules with different result types and display templates makes it easier for administrators to customize the look and feel of search. One question is whether there is a limit on the number of rules you can associate with a result source, or if there is a limit on how many triggered rules will be executed given a query?  The query rules UI gives you the ability to group rules and set where the evaluation can stop or continue. I hope there is a limit because you could set up a chain of rules that could basically run away and timeout.  I have not tested this but it would be good in the future to understand the best practices and limitations of evaluating and executing query rules.