anchor.diff:  // specific language governing permissions and limitations
   // under the License.
   
  -Many search applications store the content to be indexed in a structured data store, such as a relational database. The Data Import Handler (DIH) provides a mechanism for importing content from a data store and indexing it. In addition to relational databases, DIH can index content from HTTP based data sources such as RSS and ATOM feeds, e-mail repositories, and structured XML where an XPath processor is used to generate fields.
  -
  -The `example/example-DIH` directory contains several collections many of the features of the data import handler. To run this "```dih```" example:
  +Many search applications store the content to be indexed in a structured data store, such as a relational database. The Data Import Handler (DIH) provides a mechanism for importing content from a data store and indexing it.
  +
  +In addition to relational databases, DIH can index content from HTTP based data sources such as RSS and ATOM feeds, e-mail repositories, and structured XML where an XPath processor is used to generate fields.
  +
  +== DIH Concepts and Terminology
  +
  +Descriptions of the Data Import Handler use several familiar terms, such as entity and processor, in specific ways, as explained in the table below.
  +
  +Datasource::
  +As its name suggests, a datasource defines the location of the data of interest. For a database, it's a DSN. For an HTTP datasource, it's the base URL.
  +
  +Entity::
  +Conceptually, an entity is processed to generate a set of documents, containing multiple fields, which (after optionally being transformed in various ways) are sent to Solr for indexing. For a RDBMS data source, an entity is a view or table, which would be processed by one or more SQL statements to generate a set of rows (documents) with one or more columns (fields).
  +
  +Processor::
  +An entity processor does the work of extracting content from a data source, transforming it, and adding it to the index. Custom entity processors can be written to extend or replace the ones supplied.
  +
  +Transformer::
  +Each set of fields fetched by the entity may optionally be transformed. This process can modify the fields, create new fields, or generate multiple rows/documents form a single row. There are several built-in transformers in the DIH, which perform functions such as modifying dates and stripping HTML. It is possible to write custom transformers using the publicly available interface.
  +
  +== Solr's DIH Examples
  +
  +The `example/example-DIH` directory contains several collections to demonstrate many of the features of the data import handler. These are available with the `dih` example from the <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script>>:
   
   [source,bash]
   ----
   bin/solr -e dih
   ----
   
  -For more information about the Data Import Handler, see https://wiki.apache.org/solr/DataImportHandler.
  -
  -[[UploadingStructuredDataStoreDatawiththeDataImportHandler-ConceptsandTerminology]]
  -== Concepts and Terminology
  -
  -Descriptions of the Data Import Handler use several familiar terms, such as entity and processor, in specific ways, as explained in the table below.
  -
  -// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
  -
  -[cols="30,70",options="header"]
  -|===
  -|Term |Definition
  -|Datasource |As its name suggests, a datasource defines the location of the data of interest. For a database, it's a DSN. For an HTTP datasource, it's the base URL.
  -|Entity |Conceptually, an entity is processed to generate a set of documents, containing multiple fields, which (after optionally being transformed in various ways) are sent to Solr for indexing. For a RDBMS data source, an entity is a view or table, which would be processed by one or more SQL statements to generate a set of rows (documents) with one or more columns (fields).
  -|Processor |An entity processor does the work of extracting content from a data source, transforming it, and adding it to the index. Custom entity processors can be written to extend or replace the ones supplied.
  -|Transformer |Each set of fields fetched by the entity may optionally be transformed. This process can modify the fields, create new fields, or generate multiple rows/documents form a single row. There are several built-in transformers in the DIH, which perform functions such as modifying dates and stripping HTML. It is possible to write custom transformers using the publicly available interface.
  -|===
  -
  -[[UploadingStructuredDataStoreDatawiththeDataImportHandler-Configuration]]
  -== Configuration
  -
  -[[UploadingStructuredDataStoreDatawiththeDataImportHandler-Configuringsolrconfig.xml]]
  -=== Configuring `solrconfig.xml`
  +This launches a standalone Solr instance with several collections that correspond to detailed examples. The available examples are `atom`, `db`, `mail`, `solr`, and `tika`.
  +
  +All examples in this section assume you are running the DIH example server.
  +
  +== Configuring DIH
  +
  +=== Configuring solrconfig.xml for DIH
   
   The Data Import Handler has to be registered in `solrconfig.xml`. For example:
   
  
positive.code: </entity>
  ----
  
  There is one attribute for this transformer, `stripHTML`, which is a boolean value (true/false) to signal if the HTMLStripTransformer should process the field or not.
  
  [[UploadingStructuredDataStoreDatawiththeDataImportHandler-TheLogTransformer]]
  === The LogTransformer
  
  You can use this transformer to log data to the console or log files. For example:
  
negative.code: 
  The URLDataSource type accepts these optional parameters:
  
  baseURL::
  Specifies a new baseURL for pathnames. You can use this to specify host/port changes between Dev/QA/Prod environments. Using this attribute isolates the changes to be made to the `solrconfig.xml`
  
  connectionTimeout::
  Specifies the length of time in milliseconds after which the connection should time out. The default value is 5000ms.
  
  encoding::
  By default the encoding in the response header is used. You can use this property to override the default encoding.
  
  readTimeout::
  Specifies the length of time in milliseconds after which a read operation should time out. The default value is 10000ms.
  
  
  == Entity Processors
  
  Entity processors extract data, transform it, and add it to a Solr index. Examples of entities include views or tables in a data store.
  
  Each processor has its own set of attributes, described in its own section below. In addition, there are several attributes common to all entities which may be specified:
  
  dataSource::
  The name of a data source. If there are multiple data sources defined, use this attribute with the name of the data source for this entity.
  
  name::
  Required. The unique name used to identify an entity.
  
  pk::
  The primary key for the entity. It is optional, and required only when using delta-imports. It has no relation to the uniqueKey defined in `schema.xml` but they can both be the same.
  +
  This attribute is mandatory if you do delta-imports and then refer to the column name in `${dataimporter.delta.<column-name>}` which is used as the primary key.
  
  processor::
  Default is <<The SQL Entity Processor,SqlEntityProcessor>>. Required only if the datasource is not RDBMS.
  
  onError::
  Defines what to do if an error is encountered.
  +
  Permissible values are:
  
  abort::: Stops the import.
  
  skip::: Skips the current document.
  
  continue::: Ignores the error and processing continues.
  
  preImportDeleteQuery::
  Before a `full-import` command, use this query this to cleanup the index instead of using `\*:*`. This is honored only on an entity that is an immediate sub-child of `<document>`.
  
  postImportDeleteQuery::
  Similar to `preImportDeleteQuery`, but it executes after the import has completed.
  
  rootEntity::
  By default the entities immediately under `<document>` are root entities. If this attribute is set to false, the entity directly falling under that entity will be treated as the root entity (and so on). For every row returned by the root entity, a document is created in Solr.
  
  transformer::
  Optional. One or more transformers to be applied on this entity.
  
  cacheImpl::
  Optional. A class (which must implement `DIHCache`) to use for caching this entity when doing lookups from an entity which wraps it. Provided implementation is `SortedMapBackedCache`.
  
  cacheKey::
  The name of a property of this entity to use as a cache key if `cacheImpl` is specified.
  
  cacheLookup::
  An entity + property name that will be used to lookup cached instances of this entity if `cacheImpl` is specified.
  
  where::
  An alternative way to specify `cacheKey` and `cacheLookup` concatenated with '='.
  +
  For example, `where="CODE=People.COUNTRY_CODE"` is equivalent to `cacheKey="CODE" cacheLookup="People.COUNTRY_CODE"`
  
  child="true"::
  Enables indexing document blocks aka <<uploading-data-with-index-handlers.adoc#uploading-data-with-index-handlers,Nested Child Documents>> for searching with <<other-parsers.adoc#other-parsers,Block Join Query Parsers>>. It can be only specified on the `<entity>` element under another root entity. It switches from default behavior (merging field values) to nesting documents as children documents.
  +
  Note: parent `<entity>` should add a field which is used as a parent filter in query time.
  
  join="zipper"::
  Enables merge join, aka "zipper" algorithm, for joining parent and child entities without cache. It should be specified at child (nested) `<entity>`. It implies that parent and child queries return results ordered by keys, otherwise it throws an exception. Keys should be specified either with `where` attribute or with `cacheKey` and `cacheLookup`.
  
  === Entity Caching
  Caching of entities in DIH is provided to avoid repeated lookups for same entities again and again. The default `SortedMapBackedCache` is a `HashMap` where a key is a field in the row and the value is a bunch of rows for that same key.
  
  In the example below, each `manufacturer` entity is cached using the `id` property as a cache key. Cache lookups will be performed for each `product` entity based on the product's `manu` property. When the cache has no data for a particular key, the query is run and the cache is populated
  
  [source,xml]
  ----
  