Setup and Configurations

 

Configure ElasticSearch

 

To set up Elasticsearch correctly, please adhere to the following guidance:

• Install the latest version of Elasticsearch (minimum supported version 5.6).
• Configure settings to match your environment’s needs.
• Define user roles and configure heap size when running as a service.
• Install the Ingest Attachment Processor Plugin for file indexing.
• Refer to the official documentation for detailed setup instructions.

 

Configure Your Project for .NET Core

 

To integrate EPiInnovate.ElasticSearch.EPiServer with your .NET Core application, configure the necessary services within the Startup.cs file. This process involves registering the service so that it is available for your application to use. Below is a step-by-step guide for adding this configuration to your Startup.cs file:

1. Within the `ConfigureServices` method, you must register the EPiElasticSearch services using the `AddEpiElasticSearch()` extension method.
2. In the `Configure` method, you must invoke the `UseEpiElasticSearch()` to initialize the EPiElasticSearch service.

    using EPiInnovate.ElasticSearch.EPiServer.CMS.Extensions;

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddEpiElasticSearch();
    }

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        app.UseEpiElasticSearch();
    }

 

Configure .NET Core appsettings.json

  {
      "EpiElasticSearch": {
        "License": "",
        "Host": "http://localhost:9200",
        "BulkSize": 1000,
        "ProviderMaxResults": 100,
        "CloseIndexDelay": 500,
        "IgnoreXhtmlStringContentFragments": false,
        "ClientTimeoutSeconds": 100,
        "Username": "",
        "Password": "",
        "Shards": 5,
        "Replicas": 1,
        "TrackingConnectionStringName": "EPiServerDB",
        "Indices": [
          {
            "Name": "pages",
            "DisplayName": "Pages"
          }
        ],
        "Files": {
          "MaxSize": "50MB",
          "Enabled": true,
          "Extensions": [
            "doc",
            "txt"
          ]
        }
      }
  }

The only essential configuration parameter is `Host`; the others are optional and pre-populated with default values as specified:
Configuration Parameters:

• License: Replace with your demo or purchased license key from www.epiinnovate.com
• BulkSize: Specifies the batch size for bulk document updates.
• ProviderMaxResults: Determines the number of search results displayed in the UI.
• CloseIndexDelay: Time in milliseconds to wait between index open/close actions. Slower servers may require a longer delay.
• IgnoreXhtmlStringContentFragments: Indicates whether to disregard fragments in XhtmlStrings.
• ClientTimeoutSeconds: The HttpClient timeout duration in seconds
• Username: For basic authentication, the username.
• Password: For basic authentication, the password.
• Shards: The default shard count for new indices.
• Replicas: The default replica count for new indices.
• TrackingConnectionStringName: The identifier for the SQL connection string used for activity tracking.

In the `Indices` array, define your indices, typically one is sufficient. If multiple are used, the primary index for IContent should be identified by setting `Default` to `true`.
Example:

    {
        "Name": "my-index-name",
        "DisplayName": "My Index",
        "SynonymsFile": "synonyms/mysynonyms.txt"
    }

• Name: The actual name of the index, which will be appended with a language code for each supported language.
• DisplayName: The friendly name of the index, which appears in user-facing scenarios.
• SynonymsFile: The friendly name of the index, which appears in user-facing scenarios.
• DisplayName: An optional setting for specifying a path to a pre-existing synonyms file relative to the server’s config-folder. This file should be available upon index creation and replicated across all cluster nodes, named with the language code as a prefix. For example, if Norwegian and Swedish are active languages, `SynonymsFile` set to `synonyms/mysynonyms.txt` would require `no_mysynonyms.txt` and `se_mysynonyms.txt` to be in the config-folder.

Example:

    {
        "Files": {
            "Enabled": true or false,
            "MaxSize": "Maximum size in bytes or suffix, KB, MB, GB.",
            "Extensions": ["Array of file extensions to be indexed"]
        }
    }

• Enabled: A boolean indicating whether file indexing is active.
• MaxSize: The maximum file size allowed for indexing, specified in bytes or with a size suffix.
• Extensions: A list of file extensions that should be indexed.

 

Create the “ElasticsearchAdmins” Group

 

You must create a group named `ElasticsearchAdmins` in Optimizely/Episerver which will be used for [explain purpose of the group briefly].
To create the group, follow these steps:

1. Log in to your Optimizely/Episerver CMS.
2. Navigate to the admin panel.
3. Go to the “Groups” section.
4. Click on “Create new group”.
5. Name the group `ElasticsearchAdmins` and save it.

Add Users to the Group
Once the group is created, you’ll need to add users to it. Any user that is supposed to work with EPiInnovate.ElasticSearch.EPiServer should be a member of this group.
Here’s how you add users to the `ElasticsearchAdmins` group:

1. Still in the admin panel, find the `ElasticsearchAdmins` group you just created.
2. Click on the group to open its details.
3. Navigate to the “Users” tab within the group details.
4. Add the relevant users by selecting them and confirming the action.

After you have added the users to the group, it is important to ensure the changes are properly applied to your user account. To do this, log out of the system and then log back in. This action will register the new group memberships to your user claims, updating your access rights accordingly.
Once logged back in, you should be able to see a new menu titled “Search Engine” on your dashboard or main navigation panel. Clicking on this menu will take you to the corresponding page where you can manage settings or perform tasks related to the Elasticsearch functionality.

 

For those working with existing Optimizely/Episerver databases, it is recommended to select the option “Create or update indices with mapping” to ensure your indices are configured with the correct mappings. Otherwise, if you are setting up a new system, simply click on “Create or update indices” to create the necessary indices from scratch.
Finally, initiate the “Run indexing job” to index all necessary data into Elasticsearch. This process is crucial for enabling search capabilities on your data within the application.

Guidelines for Precision Indexing

 

To tailor the indexing configuration for a .NET Core application, you can harness the capabilities of the EPiInnovate.ElasticSearch.EPiServer.Core.Conventions.Indexing singleton. This should be set up during the application’s startup sequence, specifically within the Configure method of the Startup class.

 

Here’s an enhanced version of the configuration class:

    using EPiInnovate.ElasticSearch.EPiServer.Core.Conventions;

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {

        Indexing.Instance
            .ExcludeType()
            .ExcludeType()
            .ExcludeRoot(50)
            .IncludeFileType("pdf")
            .IncludeFileType("docx")
            .ForType()
            .IncludeProperty(x => x.Changed)
            .ForType()
            .IncludeField(x => x.GetFoo())
            .ForType()
            .IncludeField("TwoYearsAgo", x => DateTime.Now.AddYears(-2))
            .ForType().EnableSuggestions(x => x.Title)
            .ForType().EnableSuggestions()
            .ForType()
            .EnableHighlighting(x => x.MyField)
            .ForType()
            .StemField(x => x.MyField);
    }

 

• ExcludeType: Utilize this to prevent specific types from being indexed, ensuring efficiency. Similar outcomes are attainable by using the ExcludeFromSearchAttribute on your desired class or interface.
• ExcludeRoot: Apply this to exclude an entire node, along with its descendants, from the indexing process, which is particularly useful for nodes in the Episerver page tree.
• IncludeFileType: This parameter allows the specification of file extensions that should be included in the indexing, ensuring a comprehensive search capability over various file types.
• ForType: This serves as the foundational rule for indexing a particular data type. While it doesn’t perform any action by itself, it grants access to a suite of indexing options.
• IncludeField: With this, you can define which custom properties are indexed, including extension methods that pull external data or execute complex calculations.
• EnableSuggestions: Activate autosuggest features for a data type, enhancing user search experience by providing suggestions either from all properties or specific ones using a lambda function.
• IncludeProperty: Achieve the same indexing effect as the [Searchable] attribute on a property, useful when you’re unable to modify the source model directly.
• EnableHighlighting: This feature allows you to highlight additional fields in search results, drawing user attention to relevant information.
• StemField: Use this to ensure a property is analyzed according to the current language’s nuances, with specific properties like MainIntro, MainBody, and Description receiving automatic analysis.

 

By strategically implementing these options, the indexing process becomes more targeted, efficient, and aligned with the specific search needs of your application.

 

What will be indexed?

In adherence to Episerver’s established protocols, the indexing mechanism is designed to automatically capture and make searchable all text-based properties, including those of the string and XhtmlString types. To opt out any specific property from this process, it can be flagged with a [Searchable(false)] annotation. Should there be a need to include additional properties in the index, one can utilize the [Searchable] attribute. This is also applicable to ContentArea properties, which when marked [Searchable], have their content indexed, thus enhancing search capabilities with richer content accessibility.

 

Indexing or Re-indexing content

To initiate the indexing of all content or to replace previously indexed data, please navigate to the ‘Administration and status’ section under ‘Search Engine’ and click on the “Run indexing job” option. This action will trigger the indexing process, ensuring that all content is current and searchable within your system.

 

 

Whenever you publish, move, or delete content, the system will automatically re-index it to keep the search results up to date. If you need to re-index a particular piece of content, you can do so anytime by going to the Tools menu and selecting the re-indexing option  or via the context menu in the page tree.

 

Exploring the Full Spectrum of Search Capabilities

 

The EPiInnovate.ElasticSearch.EPiServer integration provides a seamless connection between EPiServer applications and Elasticsearch services. By leveraging Dependency Injection (DI), developers can effortlessly incorporate powerful search capabilities into their EPiServer-based projects.

Getting Started with DI
To begin using the ElasticSearch integration, the IElasticSearchService interface must be injected into the controllers where search operations are required. This is accomplished by adding the interface as a parameter to the constructor of the controller, as shown in the provided code example.

Constructor Injection
Here’s how to set up the HomeController to utilize the ElasticSearch service:

    private readonly IElasticSearchService _searchService;

    public HomeController(IElasticSearchService searchService)
    {
        _searchService = searchService;
    }

In this snippet, the HomeController receives an instance of IElasticSearchService through its constructor, and the instance is assigned to a private read-only field. This pattern ensures that the search service is readily available throughout the controller.

 

Performing a Search
Initiating a search operation is straightforward with the injected service. Here is an example of how to perform a basic search:

In the Index action method, we call the Search method on the _searchService field with a simple query string (in this case, “alloy”). The GetResults method is then invoked to execute the search and retrieve the results.

    public IActionResult Index()
    {

        SearchResult result = _searchService
            .Search("alloy")
            .GetResults();

        return View();
    }

 

Typed Search
A more refined search that filters results to a specified content type, ensuring that the results are strongly typed and relevant:
This method harnesses the plugin’s capability to provide results that are not only relevant but also adhere to the structure and types defined by your application’s data model, ensuring consistency and reliability in the search outcomes.

    SearchResult result = _searchService
        .Search<StartPage>("alloy")
        .GetResults();

 

Excluding Specific Types per Query
To enhance search precision, you can exclude certain content types from your query results. This targeted exclusion is beneficial when you want to omit irrelevant result types and focus solely on the content that meets your specific criteria:

    SearchResult result = _searchService
       .Search("alloy")
       .Exclude<StartPage>()
       .GetResults();

    

 

 

Excluding Entire Sections or Nodes per Query
Similarly, you can exclude entire sections or nodes from the search to narrow down the results. This is particularly useful for omitting large segments of content, such as administrative sections that are not relevant to end-user search queries:

    SearchResult result = _searchService
       .Search<StartPage>("alloy")
       .Exclude(42)
       .Exclude(contentInstance)
       .Exclude(contentReference)
       .GetResults();
   

 

Precise Property Queries
For more precise and relevant search results, it’s beneficial to direct your searches toward particular aspects of your data. For instance, when you search for a term like “alloy,” the system can be set up to look through relevant fields only, such as titles of recipes or ingredients. This targeted search strategy improves the likelihood that the results are closely aligned with what you’re looking for.

    SearchResult result = _searchService
        .Search<StartPage>("alloy")
        .InField(x => x.Title)
        .InField(x => x.Description)
        .InField(x => x.Keywords)
        .GetResults();

 

Autocomplete Suggestions
Utilize predictive text features to provide instant suggestions while users type, streamlining the search process by reducing keystrokes and leading them closer to their desired search term.

    string[] result = _searchService
        .GetSuggestions("alloy");

 

Wildcard Search Patterns
Incorporate wildcard characters in your search queries to create adaptable search patterns. This technique is particularly handy when the full term is uncertain or to retrieve a wider array of results that conform to a given pattern. For instance, using a wildcard with the base “cook*” can yield diverse results like “cookie,” “cooking,” or “cookware.

    SearchResult result = _searchService
        .WildcardSearch("cook*")
        .GetResults();

 

Simplified Query Strings
Apply simplified query strings to enable users to conduct searches with basic logical operators. This approach simplifies the construction of complex queries, making it accessible to users who may not be familiar with advanced search syntax:

    SearchResult result = _searchService
        .SimpleQuerystringSearch("bacon AND eggs NOT burned")
        .GetResults();

 

With limited operators:

    SearchResult result = _searchService
        .SimpleQuerystringSearch("\"bacon melt\" melt sandwi*",
            defaultOperator: Operator.And,
            allowedOperators:
            SimpleQuerystringOperators.Prefix |
            SimpleQuerystringOperators.Phrase)
        .GetResults();

 

Fuzzy Search
Implement a fuzzy search algorithm to accommodate for misspellings or approximate keyword entries. This technique, based on the Levenshtein distance metric, allows for a more forgiving search experience, capturing relevant results that may have minor discrepancies from the search term:

    SearchResult result = _searchService
        .Search("aloy")
        .Fuzzy()
        .GetResults();

 

More Like This
Enhance the user’s search journey with the ‘More Like This’ feature, designed to recommend content that aligns with their current interest. This tool examines the attributes of documents to surface others with similar properties, thus boosting the discoverability of related content.

    SearchResult result = _searchService
        .MoreLikeThis(contentId)
        .GetResults();

 

Did You Mean
The ‘Did You Mean’ feature provides users with corrective suggestions when they mistype a search term. This capability is integral to maintaining a fluid and user-friendly search experience, helping to prevent fruitless search results due to common spelling errors:

    SearchResult result = _searchService
        .Search("smartphne")
        .GetResults();
    IEnumerable suggestedCorrections = result.DidYouMean; //["smartphone"]

 

Advanced Filtering
Advanced filtering enables the creation of sophisticated queries through the amalgamation of numerous filter criteria. This precise control helps tailor search results to meet exact user requirements and preferences:

    SearchResult result = _searchService
        .Search<ProductPage>("outdoor gear")
        .FilterGroup(f => f
            .Or(p => p.Category, "camping")
            .And(p => p.Availability, "in-stock")
            .Or(p => p.Brand, new[] {"BrandA", "BrandB"})
        );

 

Exclusion Filters with ‘Not-Filter’
The ‘Not-Filter’ is applied when you need to explicitly exclude certain terms from your search results. This exclusion helps in scenarios where the presence of a term would make the content less relevant to the searcher’s intent:

    SearchResult result = _searchService
        .Search<ProductPage>("smartphones")
        .FilterMustNot(p => p.Brand, "BrandX")
        .GetResults();

 

ACL-Filter
To filter on the current users ACL, use FilterByACL

    SearchResult result = _searchService
        .Search<ProductPage>("smartphones")
        .FilterMustNot(p => p.Brand, "BrandX")
        .FilterByACL()
        .GetResults();

 

Pagination
Pagination is a crucial feature that structures the search results into manageable segments, allowing users to navigate through them efficiently. By defining the starting point and the size of the result set, users can access specific portions of the search results without overwhelming load times:

    
    SearchResult result = _searchService
       .Search<ProductPage<("foo")
       .From(10)
       .Size(20)
       .GetResults();

 

Scroll API
Currently under development, the Scroll API provides an optimized solution for accessing extensive sets of search results. This method is essential for navigating through vast datasets where conventional search queries fall short due to efficiency constraints. The Scroll API supersedes traditional pagination in scenarios where retrieving large volumes of data is required, offering a more resource-effective approach to data retrieval.

 

Sorting
Sorting enables the organization of search results in a meaningful order, such as by relevance, date, price, or any other specified criterion. This ordered structuring is essential for enhancing user experience, as it allows users to quickly find the most pertinent results:

    SearchResult result = _searchService
        .Get<ProductPage>()
        .SortBy(p => p.ReleaseDate)
        .ThenBy(p => p.Popularity)
        .ThenBy(p => p.Rating)
        .GetResults();

 

Boosting
Boosting in search queries is a technique used to fine-tune the importance of certain terms or properties in the search results. By assigning a higher boost value, you can influence the ranking algorithm to give more weight to the preferred attributes, making them more prominent in the search results:

    [Boost(13)]
    public string Title { get; set; }

… or at query time:

    SearchResult result = _searchService
       .Search("alloy")
       .Boost(x => x.MyProp, 3)
       .GetResults();

 

A type can be given either a positive or a negative boost:

    SearchResult result = _searchService
       .Search("alloy")
       .Boost<ProductPage>(2)
       .GetResults();

    SearchResult result = _searchService
       .Search("alloy")
       .Boost(typeof(ProductPage), 2)
       .GetResults();

    SearchResult result = _searchService
       .Search("alloy")
       .Boost<ProductPage>(-5)
       .GetResults();

You can also boost hits depending on their location in the page tree:

SearchResult result = _searchService
   .Search("alloy")
   .BoostByAncestor(new ContentReference(42), 2)
   .GetResults();

 

Date properties can be scored lower depending on their value, by using the Decay function. This way you can promote newer articles over older ones.
Every time the date interval (2nd argument) occurs, 0.5 points will be subtracted from the score. In the example below, 0.5 points will be subtracted after 7 days, 1 points after 14 days, and so on.

SearchResult result = _searchService
   .Search("alloy")
   .Decay(x => x.StartPublish, TimeSpan.FromDays(7))
   .GetResults();