Elasticsearch Request Failed Issue in Sitecore Content Hub

Problem Statement:

After setting the default values in the content entity, it gives me an error when I attempt to create a new content type. However, it also creates a content entity after an error, and it works if I create a content entity, but while creating a content entity, it returns me an Elasticsearch request failed error.

Error:

Solution:

Sitecore has investigated and identified this issue. It involves an issue with the Elasticsearch request and requires a graph rebuild to resolve. See the documentation for more information: Schema changes, inheritance, and the graph server

FAQ:

1. Duration of Graph Rebuild and Downtime: How long does the graph rebuild process typically take, and how much downtime will be required for the PROD CH environment?

The graph rebuild activity is estimated to take up to 2 hours to complete. During the process, Sitecore will back up the main index in the Elasticsearch deployment, as well as the graph snapshot to facilitate rollback if needed. During the operation, the environment will not be accessible, and users will be shown the “Down for maintenance” page.

2. Preferred Schedule time to rebuild Graph

The operation will involve some downtime, and Sitecore recommends scheduling it outside of business hours.

3. Impact on Custom Entities: Will the graph rebuild have any effect on other custom entities, content types, or structures? I want to avoid a situation where addressing this issue might accidentally create another one.

The graph rebuild process involves Elasticsearch reindex which will rectify all out-of-sync search issues. There should be no changes to your custom definitions and entities. However, there may be additional impact on Experience Edge from clearing the graph queue. If there were any updates to entities that are being exported when the graph rebuild happens, these changes would be lost. To recover from this, a publish-all operation would be required. If there was an ingestion of data for a specific schema, it's recommended to publish only the entities that belong to that schema. You may also refer here for more details on the Schema changes and graph server behavior:

The primary goal of the graph rebuild operation is to align the mapping structure in Elasticsearch and re-index the data that is already present in the database. Since the database serves as the single source of truth, this process simply reconstructs the search index, resolving any existing mapping and search-related issues. During the operation, the environment will not be accessible, and users will be shown the “Down for maintenance” page.

4. Reason Behind this issue:

This is related to an infrastructure update and is not connected to any product software updates. Earlier this year, Sitecore updated the underlying infrastructure of Content Hub, specifically upgrading to Elasticsearch version 8. This maintenance was done without downtime and was seamless for end-users. The upgrade to ES8 brought several benefits, including:

- Visual Search: This new feature required ES8.

- Support: ES7 was nearing the end of support, so upgrading ensures we're using the latest stable and supported version.

- Stability and Performance: Our engineering teams continually improve the stability and performance of search functionalities.

Pre-production and production maintenance were carried out separately, and it appears the mapping templates for date-time properties differ between the two environments. We have reported this issue internally to our engineering teams for further analysis, and the solution is a full index rebuild.

Happy Sitecoreing 😊