Overview

Change the structure of Apache Ignite’s QueryEntities with no need to restart the cluster.

Features:

• Ensures system availability when changes are implemented on the structure of the QueryEntities defined on Apache Ignite caches.

• Detected changes in data structure are propagated automatically, in the background and over all cluster nodes. No need to restart.

• Optimizes indexes rebuild process.

Importing maven dependency:

Add repository and the Maven dependency below to your pom.xml file to make sure that the module is included into your application:

Maven dependency

<dependency>
    <groupId>com.hawkore.libs.ignite</groupId>
    <artifactId>hk-indexing</artifactId>
    <version>2.8.1-hk</version>
</dependency>

Hawkore's Repository

<repositories>
    <repository>
        <id>Hawkore Repository</id>
        <url>https://repository.hawkore.com/maven2/</url>
    </repository>
</repositories>

---

Optimized indexes rebuild

---

Rebuild QueryEntity's indexes over cluster.

Features:

• Simultaneously rebuild all provided indexes within the same population step (full cache iterator). This will improve dramatically indexes rebuild because you don't need to do it one by one, as done with standard DDL DROP/CREATE INDEX statements. Just think on a cache with billions of records...

• Ensures system availability as indexes are rebuilt in the background and do not need to be dropped.

Java

import com.hawkore.ignite.cache.IgniteCacheLoader;
(...)

IgniteCacheLoader.dynamicIndexesRebuild(ignite, cacheName, tableName, async, parallel, indexNames);   

1. ignite (org.apache.ignite.Ignite)
   • Ignite instance
2. cacheName (java.lang.String)
   • cache containing table (QueryEntity)
3. tableName (java.lang.String)
   • Name of table containing indexes to rebuild
4. async (boolean)
   • Whether to wait until process is completed
5. parallel (int)
   • Indexes creation/rebuild parallelism level
6. indexNames (String...)
   • Optional index names to rebuild. If not provided all indexes defined on table will be rebuilt.

---

Update QueryEntities over cluster

---

Updates cache's QueryEntities over cluster. This is normally do it from client nodes as they have QueryEntities class definitions.

Features:

• Auto register new QueryEntities defined on caches

• Auto create new columns

• Auto create/update and populate (only if required) indexes

• Supports full QueryEntity recreation, needs to be explicitly requested by forceMutateQueryEntity parameter

• Supports force rebuild new QueryEntity indexes, needs to be explicitly requested by forceIndexRebuild parameter

Java

import com.hawkore.ignite.cache.IgniteCacheLoader;
(...)
IgniteCacheLoader.updateQueryEntities(ignite, oldConf, newConf, forceIndexRebuild, forceMutateQueryEntity, async, parallel)

Parameters:

1. ignite (org.apache.ignite.Ignite)

   • Ignite instance

2. oldConf (org.apache.ignite.configuration.CacheConfiguration)

   • Old CacheConfiguration

3. newConf (org.apache.ignite.configuration.CacheConfiguration)

   • New CacheConfiguration

4. forceIndexRebuild (boolean)

   • whether to force new QueryEntity indexes to be re-build

5. forceMutateQueryEntity (boolean). WARNING use with precaution

   • Whether to force QueryEntity recreation. By default QueryEntity update is applied in a "grow up mode", this means that only is allowed to create new columns and/or new indexes. Note that if you set this flag to true, indexes and mapped sql columns that do not exist on QueryEntities defined on New CacheConfiguration will be deleted.

6. async (boolean)

   • Whether to wait until process is completed

7. parallel (int)

   • Indexes creation/rebuild parallelism level

Java sample

import com.hawkore.ignite.cache.IgniteCacheLoader;
(...)

IgniteCache<?, ?> cache = ignite.cache("myCache");

CacheConfiguration<?, ?> currentCacheConf = ((CacheConfiguration<?, ?>) cache
    .getConfiguration(CacheConfiguration.class));

CacheConfiguration<?, ?> newCacheConf = ...;

IgniteCacheLoader.updateQueryEntities(ignite, currentCacheConf, newCacheConf, false, false, true, 10);

Spring XML based configurations

<!-- 
Alternative mechanism for caches registration in order to update/create QueryEntities over Cluster 
-->
<bean id="ignite_caches_registration" name="ignite_caches_registration"
class="com.hawkore.ignite.cache.IgniteCacheLoader">
    <property name="igniteGridName" value="${ignite.this.node.gridname}"/>          
    <property name="startIgniteAfterPropertiesSet" value="false" />
    <property name="updateEntityConfigsOverCluster" value="true" />
    <property name="autoLoadCache" value="false" />
    <property name="parallel" value="4"/>
    <property name="loadCacheConfigurations">
        <list>
            <ref bean="tweetsCache" />
        </list>
    </property>     
</bean>

<!-- Annotation Based Configuration cache -->
<bean id="tweetsCache" class="org.apache.ignite.configuration.CacheConfiguration">
    <property name="name" value="tweets" />
    <property name="rebalanceMode" value="ASYNC"/> 
    <property name="cacheMode" value="PARTITIONED" />
    <property name="indexedTypes">
        <array>
            <value>com.hawkore.ignite.examples.entities.tweets.TweetKey</value>
            <value>com.hawkore.ignite.examples.entities.tweets.Tweet</value>
        </array>
    </property> 
</bean>

<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" >

    <property name="gridName" value="${ignite.this.node.gridname}"/>

    <property name="lifecycleBeans">
        <list>
             <ref bean="ignite_caches_registration"/>
        </list>
    </property>             

(...)

</bean>