preparation

Environmental preparation

JAVA version

Copy
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

ES version

Copy
{
 "name": "pYaFJhZ",
 "cluster_name": "my-cluster",
 "cluster_uuid": "oC28y-cNQduGItC7qq5W8w",
 "version": {
 "number": "6.8.2",
 "build_flavor": "oss",
 "build_type": "tar",
 "build_hash": "b506955",
 "build_date": "2019-07-24T15:24:41.545295Z",
 "build_snapshot": false,
 "lucene_version": "7.7.0",
 "minimum_wire_compatibility_version": "5.6.0",
 "minimum_index_compatibility_version": "5.0.0"
 },
 "tagline": "You Know, for Search"
}

SpringBoot version

Copy
2.1.7.RELEASE

Development tools use IDEA

Install ES

Elasticsearch introduction and installation: getting started with elasticsearch - Introduction to basic concepts and installation

start

Create a SpringBoot project

1. Open IDEA and click in the menu

File > New > Project...

Select Spring Initializr in the pop-up box

Then Next

2. Fill in the project name, etc., and then Next,

3. Select the dependent jar package (generally, I only choose Lombok, and others are added manually), and then Next.

4. Finally, select the project path and click Finish.

Finish the work. At this point, a new SpringBoot project is fresh.

POM file

Of course, the specific dependent jar packages must be more than those selected in step 2, including those provided by SpringBoot Domain name trading platform Of course, spring boot starter data elasticsearch, a jar package for ES, is also essential.

The final pom file is posted here:

Copy
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <parent>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-parent</artifactId>
 <version>2.1.7.RELEASE</version>
 <relativePath/> <!-- lookup parent from repository -->
 </parent>
 <groupId>com.lifengdi</groupId>
 <artifactId>search</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>search</name>
 <description>elasticsearch</description>
 <properties>
 <java.version>1.8</java.version>
 <testng.version>6.14.2</testng.version>
 <spring-cloud-dependencies.version>Greenwich.RELEASE</spring-cloud-dependencies.version>
 <kibana-logging-spring-boot-starter.version>1.2.4</kibana-logging-spring-boot-starter.version>
 <fastjson.version>1.2.47</fastjson.version>
 <alarm-spring-boot-starter.version>1.0.15-SNAPSHOT</alarm-spring-boot-starter.version>
 </properties>
 <dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.springframework.cloud</groupId>
 <artifactId>spring-cloud-dependencies</artifactId>
 <version>${spring-cloud-dependencies.version}</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>
 <!--elasticsearch-->
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-configuration-processor</artifactId>
 <optional>true</optional>
 </dependency>
 <!--lombok-->
 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
 </dependency>
 <!--test-->
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
 <dependency>
 <groupId>org.testng</groupId>
 <artifactId>testng</artifactId>
 <version>${testng.version}</version>
 <scope>test</scope>
 </dependency>
 <!-- Date processing -->
 <dependency>
 <groupId>joda-time</groupId>
 <artifactId>joda-time</artifactId>
 </dependency>
 <!--FastJson-->
 <dependency>
 <groupId>com.alibaba</groupId>
 <artifactId>fastjson</artifactId>
 <version>${fastjson.version}</version>
 </dependency>
 <!--feign-->
 <dependency>
 <groupId>org.springframework.cloud</groupId>
 <artifactId>spring-cloud-starter-openfeign</artifactId>
 </dependency>
 <dependency>
 <groupId>org.apache.commons</groupId>
 <artifactId>commons-lang3</artifactId>
 </dependency>
 </dependencies>
 <build>
 <plugins>
 <plugin>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-maven-plugin</artifactId>
 </plugin>
 </plugins>
 </build>
</project>

application.yml file

The application.yml file is configured as follows:

Copy
server:
 port: 8080
 servlet:
 context-path: /search
spring:
 application:
 name: search
 data:
 elasticsearch:
 cluster-name: my-cluster
 cluster-nodes: localhost:9300
 jackson:
 default-property-inclusion: non_null
logging:
 file: application.log
 path: .
 level:
 root: info
 com.lifengdi.store.client: DEBUG
index-entity:
 configs:
 - docCode: store
 indexName: store
 type: base
 documentPath: com.lifengdi.document.StoreDocument

spring.data.elasticsearch.cluster-name: cluster name

spring.data.elasticsearch.cluster-nodes: address list of cluster nodes. Multiple nodes are separated by English commas (,)

Create ES documents and maps

First create a JAVA object, and then declare the mapping properties of the field through annotations.

The annotations provided by spring include @ Document, @ id and @ Field, where @ Document acts on the class, @ id and @ Field act on the member variable, and @ id marks a Field as the id primary key.

Copy
package com.lifengdi.document;
import com.lifengdi.document.store.*;
import com.lifengdi.search.annotation.DefinitionQuery;
import com.lifengdi.search.enums.QueryTypeEnum;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import java.util.List;
/**
 * Store Document
 *
 * @author Li Fengdi
 * @date Create at 19:31 2019/8/22
 */
@Document(indexName = "store", type = "base")
@Data
@DefinitionQuery(key = "page", type = QueryTypeEnum.IGNORE)
@DefinitionQuery(key = "size", type = QueryTypeEnum.IGNORE)
@DefinitionQuery(key = "q", type = QueryTypeEnum.FULLTEXT)
public class StoreDocument {
 @Id
 @DefinitionQuery(type = QueryTypeEnum.IN)
 @DefinitionQuery(key = "id", type = QueryTypeEnum.IN)
 @Field(type = FieldType.Keyword)
 private String id;
 /**
 * Basic information
 */
 @Field(type = FieldType.Object)
 private StoreBaseInfo baseInfo;
 /**
 * label
 */
 @Field(type = FieldType.Nested)
 @DefinitionQuery(key = "tagCode", mapped = "tags.key", type = QueryTypeEnum.IN)
 @DefinitionQuery(key = "tagValue", mapped = "tags.value", type = QueryTypeEnum.AND)
 @DefinitionQuery(key = "_tagValue", mapped = "tags.value", type = QueryTypeEnum.IN)
 private List<StoreTags> tags;
}

Create index

ElasticsearchTemplate provides four createIndex() methods to create indexes, which can be generated automatically according to class information, or you can manually specify indexName and Settings

Copy
@Override
public <T> boolean createIndex(Class<T> clazz) {
 return createIndexIfNotCreated(clazz);
}
@Override
public boolean createIndex(String indexName) {
 Assert.notNull(indexName, "No index defined for Query");
 return client.admin().indices().create(Requests.createIndexRequest(indexName)).actionGet().isAcknowledged();
}
@Override
public boolean createIndex(String indexName, Object settings) {
 CreateIndexRequestBuilder createIndexRequestBuilder = client.admin().indices().prepareCreate(indexName);
 if (settings instanceof String) {
 createIndexRequestBuilder.setSettings(String.valueOf(settings), Requests.INDEX_CONTENT_TYPE);
 } else if (settings instanceof Map) {
 createIndexRequestBuilder.setSettings((Map) settings);
 } else if (settings instanceof XContentBuilder) {
 createIndexRequestBuilder.setSettings((XContentBuilder) settings);
 }
 return createIndexRequestBuilder.execute().actionGet().isAcknowledged();
}
@Override
public <T> boolean createIndex(Class<T> clazz, Object settings) {
 return createIndex(getPersistentEntityFor(clazz).getIndexName(), settings);
}

Create mapping

ElasticsearchTemplate provides three putMapping() methods to create mappings

Copy
@Override
public <T> boolean putMapping(Class<T> clazz) {
 if (clazz.isAnnotationPresent(Mapping.class)) {
 String mappingPath = clazz.getAnnotation(Mapping.class).mappingPath();
 if (!StringUtils.isEmpty(mappingPath)) {
 String mappings = readFileFromClasspath(mappingPath);
 if (!StringUtils.isEmpty(mappings)) {
 return putMapping(clazz, mappings);
 }
 } else {
 LOGGER.info("mappingPath in @Mapping has to be defined. Building mappings using @Field");
 }
 }
 ElasticsearchPersistentEntity<T> persistentEntity = getPersistentEntityFor(clazz);
 XContentBuilder xContentBuilder = null;
 try {
 ElasticsearchPersistentProperty property = persistentEntity.getRequiredIdProperty();
 xContentBuilder = buildMapping(clazz, persistentEntity.getIndexType(),
 property.getFieldName(), persistentEntity.getParentType());
 } catch (Exception e) {
 throw new ElasticsearchException("Failed to build mapping for " + clazz.getSimpleName(), e);
 }
 return putMapping(clazz, xContentBuilder);
}
@Override
public <T> boolean putMapping(Class<T> clazz, Object mapping) {
 return putMapping(getPersistentEntityFor(clazz).getIndexName(), getPersistentEntityFor(clazz).getIndexType(),
 mapping);
}
@Override
public boolean putMapping(String indexName, String type, Object mapping) {
 Assert.notNull(indexName, "No index defined for putMapping()");
 Assert.notNull(type, "No type defined for putMapping()");
 PutMappingRequestBuilder requestBuilder = client.admin().indices().preparePutMapping(indexName).setType(type);
 if (mapping instanceof String) {
 requestBuilder.setSource(String.valueOf(mapping), XContentType.JSON);
 } else if (mapping instanceof Map) {
 requestBuilder.setSource((Map) mapping);
 } else if (mapping instanceof XContentBuilder) {
 requestBuilder.setSource((XContentBuilder) mapping);
 }
 return requestBuilder.execute().actionGet().isAcknowledged();
}

The test code is as follows

Copy
@Test
public void testCreate() {
 System.out.println(elasticsearchTemplate.createIndex(StoreDocument.class));
 System.out.println(elasticsearchTemplate.putMapping(StoreDocument.class));
}

Delete index

ElasticsearchTemplate provides two deleteIndex() methods to delete indexes

Copy
@Override
public <T> boolean deleteIndex(Class<T> clazz) {
 return deleteIndex(getPersistentEntityFor(clazz).getIndexName());
}
@Override
public boolean deleteIndex(String indexName) {
 Assert.notNull(indexName, "No index defined for delete operation");
 if (indexExists(indexName)) {
 return client.admin().indices().delete(new DeleteIndexRequest(indexName)).actionGet().isAcknowledged();
 }
 return false;
}

Add & modify document

In Elasticsearch, documents are immutable and cannot be modified. Conversely, if you want to update an existing document, you need to re index or replace it.

Therefore, you can use the same interface as the new interface to modify the document. The distinction is based on id.

Two methods for adding & modifying documents are provided below. One is the index() method provided by ElasticsearchTemplate:

Copy
@Override
public String index(IndexQuery query) {
 String documentId = prepareIndex(query).execute().actionGet().getId();
 // We should call this because we are not going through a mapper.
 if (query.getObject() != null) {
 setPersistentEntityId(query.getObject(), documentId);
 }
 return documentId;
}

The example code is as follows:

Copy
/**
 * Update index
 * @param indexName Index name
 * @param type Index type
 * @param id ID
 * @param jsonDoc JSON Formatted document
 * @param refresh Refresh index
 * @return ID
 */
public String index(String indexName, String type, String id, JsonNode jsonDoc, boolean refresh)
 throws JsonProcessingException {
 log.info("AbstractDocumentIndexService Update index.indexName:{},type:{},id:{},jsonDoc:{}", indexName, type, id, jsonDoc);
 IndexQuery indexQuery = new IndexQueryBuilder()
 .withIndexName(indexName)
 .withType(type)
 .withId(id)
 .withSource(objectMapper.writeValueAsString(jsonDoc))
 .build();
 try {
 if (elasticsearchTemplate.indexExists(indexName)) {
 String index = elasticsearchTemplate.index(indexQuery);
 if (refresh) {
 elasticsearchTemplate.refresh(indexName);
 }
 return index;
 }
 } catch (Exception e) {
 log.error("Failed to update index,Refresh ES retry ", e);
 elasticsearchTemplate.refresh(indexName);
 return elasticsearchTemplate.index(indexQuery);
 }
 throw BaseException.INDEX_NOT_EXISTS_EXCEPTION.build();
 }

The other is through the Repository interface. The Repository interface of ES provided by Spring is ElasticsearchCrudRepository, so we can directly define a new interface and implement ElasticsearchCrudRepository:

Copy
package com.taoche.docindex.repo;
import com.taoche.document.StoreDocument;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
/**
 * Store Repository
 * @author Li Fengdi
 * @date Create at 09:30 2019/8/23
 */
public interface StoreRepository extends ElasticsearchRepository<StoreDocument, String> { }

The example code is as follows:

Copy
@Test
public void testSave() {
 StoreDocument storeDocument = new StoreDocument();
 storeDocument.setId("1");
 StoreBaseInfo baseInfo = new StoreBaseInfo();
 baseInfo.setStoreId("1");
 baseInfo.setCreatedTime(DateTime.now());
 storeDocument.setBaseInfo(baseInfo);
 storeRepository.save(storeDocument);
}

query

The main function of ES is query. ElasticsearchRepository also provides basic query interfaces, such as findById(), findAll(), findAllById(), search(); Of course, you can also use another function provided by Spring Data: Spring Data JPA - to create a query by method name, of course, you need to follow certain rules. For example, your method name is findByTitle(), then it will know that you query according to the title, and then automatically help you complete it. I won't talk about it carefully here.

The above can basically meet the general query, but a more complex query can't help. This requires custom query. Here you can check another blog of mine. SpringBoot uses annotation to build Elasticsearch query statements to realize multi condition complex query. Here's a detailed description.

In addition, there is a powerful function, Elasticsearch aggregation; Aggregation mainly realizes the statistics and analysis of data. This is not used for the time being, so the partners who want to see the aggregation function may be disappointed ~ hahaha~~~

The aggregation function will be said separately when there is time ~ there will be.

So far, SpringBoot integration Elasticsearch has basically ended. Please leave a message if you don't understand~