Chapter 7. Migrating data between cache stores
Chapter 7. Migrating data between cache stores Data Grid provides a Java utility for migrating persistent data between cache stores. In the case of upgrading Data Grid, functional differences between major versions do not allow backwards compatibility between cache stores. You can use StoreMigrator to convert your data so that it is compatible with the target version. For example, upgrading to Data Grid 8.0 changes the default marshaller to Protostream. In Data Grid versions, cache stores use a binary format that is not compatible with the changes to marshalling. This means that Data Grid 8.0 cannot read from cache stores with Data Grid versions. In other cases Data Grid versions deprecate or remove cache store implementations, such as JDBC Mixed and Binary stores. You can use StoreMigrator in these cases to convert to different cache store implementations. 7.1. Cache store migrator Data Grid provides the CLI migrate store command that recreates data for the latest Data Grid cache store implementations. The store migrator takes a cache store from a version of Data Grid as source and uses a cache store implementation as target. When you run the store migrator, it creates the target cache with the cache store type that you define using the EmbeddedCacheManager interface. The store migrator then loads entries from the source store into memory and then puts them into the target cache. The store migrator also lets you migrate data from one type of cache store to another. For example, you can migrate from a JDBC string-based cache store to a SIFS cache store. Important The store migrator cannot migrate data from segmented cache stores to: Non-segmented cache store. Segmented cache stores that have a different number of segments. 7.2. Configuring the cache store migrator Use the migrator.properties file to configure properties for source and target cache stores. Procedure Create a migrator.properties file. Configure properties for source and target cache store using the migrator.properties file. Add the source. prefix to all configuration properties for the source cache store. Example source cache store Important For migrating data from segmented cache stores, you must also configure the number of segments using the source.segment_count property. The number of segments must match clustering.hash.numSegments in your Data Grid configuration. If the number of segments for a cache store does not match the number of segments for the corresponding cache, Data Grid cannot read data from the cache store. Add the target. prefix to all configuration properties for the target cache store. Example target cache store 7.2.1. Configuration properties for the cache store migrator Configure source and target cache stores in a StoreMigrator properties. Table 7.1. Cache Store Type Property Property Description Required/Optional type Specifies the type of cache store for a source or target cache store. .type=JDBC_STRING .type=JDBC_BINARY .type=JDBC_MIXED .type=LEVELDB .type=ROCKSDB .type=SINGLE_FILE_STORE .type=SOFT_INDEX_FILE_STORE .type=JDBC_MIXED Required Table 7.2. Common Properties Property Description Example Value Required/Optional cache_name The name of the cache that you want to back up. .cache_name=myCache Required segment_count The number of segments for target cache stores that can use segmentation. The number of segments must match clustering.hash.numSegments in the Data Grid configuration. If the number of segments for a cache store does not match the number of segments for the corresponding cache, Data Grid cannot read data from the cache store. .segment_count=256 Optional marshaller.class Specifies a custom marshaller class. Required if using custom marshallers. marshaller.allow-list.classes Specifies a comma-separated list of fully qualified class names that are allowed to be deserialized. Optional marshaller.allow-list.regexps Specifies a comma-separated list of regular expressions that determine which classes are allowed be deserialized. Optional marshaller.externalizers Specifies a comma-separated list of custom AdvancedExternalizer implementations to load in this format: [id]:<Externalizer class> Optional Table 7.3. JDBC Properties Property Description Required/Optional dialect Specifies the dialect of the underlying database. Required version Specifies the marshaller version for source cache stores. Set one of the following values: * 8 for Data Grid 7.2.x * 9 for Data Grid 7.3.x * 10 for Data Grid 8.0.x * 11 for Data Grid 8.1.x * 12 for Data Grid 8.2.x * 13 for Data Grid 8.3.x Required for source stores only. connection_pool.connection_url Specifies the JDBC connection URL. Required connection_pool.driver_class Specifies the class of the JDBC driver. Required connection_pool.username Specifies a database username. Required connection_pool.password Specifies a password for the database username. Required db.disable_upsert Disables database upsert. Optional db.disable_indexing Specifies if table indexes are created. Optional table.string.table_name_prefix Specifies additional prefixes for the table name. Optional table.string.<id|data|timestamp>.name Specifies the column name. Required table.string.<id|data|timestamp>.type Specifies the column type. Required key_to_string_mapper Specifies the TwoWayKey2StringMapper class. Optional Note To migrate from Binary cache stores in older Data Grid versions, change table.string.* to table.binary.\* in the following properties: source.table.binary.table_name_prefix source.table.binary.<id\|data\|timestamp>.name source.table.binary.<id\|data\|timestamp>.type Table 7.4. RocksDB Properties Property Description Required/Optional location Sets the database directory. Required compression Specifies the compression type to use. Optional Table 7.5. SingleFileStore Properties Property Description Required/Optional location Sets the directory that contains the cache store .dat file. Required Table 7.6. SoftIndexFileStore Properties Property Description Value Required/Optional location Sets the database directory. Required index_location Sets the database index directory. 7.3. Migrating Data Grid cache stores Run the store migrator to migrate data from one cache store to another. Prerequisites Get the Data Grid CLI. Create a migrator.properties file that configures the source and target cache stores. Procedure Run the migrate store -p /path/to/migrator.properties CLI command
source.type=SOFT_INDEX_FILE_STORE source.cache_name=myCache source.location=/path/to/source/sifs source.version=<version> target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/target/sfs.dat Example configuration for migrating to a JDBC String-Based cache store target.type=STRING target.cache_name=myCache target.dialect=POSTGRES target.marshaller.class=org.infinispan.commons.marshall.JavaSerializationMarshaller target.marshaller.allow-list.classes=org.example.Person,org.example.Animal target.marshaller.allow-list.regexps="org.another.example.*" target.marshaller.externalizers=25:Externalizer1,org.example.Externalizer2 target.connection_pool.connection_url=jdbc:postgresql:postgres target.connection_pool.driver_class=org.postrgesql.Driver target.connection_pool.username=postgres target.connection_pool.password=redhat target.db.disable_upsert=false target.db.disable_indexing=false target.table.string.table_name_prefix=tablePrefix target.table.string.id.name=id_column target.table.string.data.name=datum_column target.table.string.timestamp.name=timestamp_column target.table.string.id.type=VARCHAR target.table.string.data.type=bytea target.table.string.timestamp.type=BIGINT target.key_to_string_mapper=org.infinispan.persistence.keymappers. DefaultTwoWayKey2StringMapper Example configuration for migrating from a RocksDB cache store. source.type=ROCKSDB source.cache_name=myCache source.location=/path/to/rocksdb/database source.compression=SNAPPY Example configuration for migrating to a Single File cache store. target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/sfs.dat Example configuration for migrating to a Soft-Index File cache store. target.type=SOFT_INDEX_FILE_STORE target.cache_name=myCache target.location=path/to/sifs/database target.location=path/to/sifs/index