preparation
Environmental preparation
JAVA version
Copy java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
ES version
Copy { "name": "pYaFJhZ", "cluster_name": "my-cluster", "cluster_uuid": "oC28y-cNQduGItC7qq5W8w", "version": { "number": "6.8.2", "build_flavor": "oss", "build_type": "tar", "build_hash": "b506955", "build_date": "2019-07-24T15:24:41.545295Z", "build_snapshot": false, "lucene_version": "7.7.0", "minimum_wire_compatibility_version": "5.6.0", "minimum_index_compatibility_version": "5.0.0" }, "tagline": "You Know, for Search" }
SpringBoot version
Copy 2.1.7.RELEASE
Development tools use IDEA
Install ES
Elasticsearch introduction and installation: getting started with elasticsearch - Introduction to basic concepts and installation
start
Create a SpringBoot project
1. Open IDEA and click in the menu
File > New > Project...
Select Spring Initializr in the pop-up box
Then Next
2. Fill in the project name, etc., and then Next,
3. Select the dependent jar package (generally, I only choose Lombok, and others are added manually), and then Next.
4. Finally, select the project path and click Finish.
Finish the work. At this point, a new SpringBoot project is fresh.
POM file
Of course, the specific dependent jar packages must be more than those selected in step 2, including those provided by SpringBoot Domain name trading platform Of course, spring boot starter data elasticsearch, a jar package for ES, is also essential.
The final pom file is posted here:
Copy <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.7.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.lifengdi</groupId> <artifactId>search</artifactId> <version>0.0.1-SNAPSHOT</version> <name>search</name> <description>elasticsearch</description> <properties> <java.version>1.8</java.version> <testng.version>6.14.2</testng.version> <spring-cloud-dependencies.version>Greenwich.RELEASE</spring-cloud-dependencies.version> <kibana-logging-spring-boot-starter.version>1.2.4</kibana-logging-spring-boot-starter.version> <fastjson.version>1.2.47</fastjson.version> <alarm-spring-boot-starter.version>1.0.15-SNAPSHOT</alarm-spring-boot-starter.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud-dependencies.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!--elasticsearch--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <!--lombok--> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <!--test--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>${testng.version}</version> <scope>test</scope> </dependency> <!-- Date processing --> <dependency> <groupId>joda-time</groupId> <artifactId>joda-time</artifactId> </dependency> <!--FastJson--> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>${fastjson.version}</version> </dependency> <!--feign--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-openfeign</artifactId> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
application.yml file
The application.yml file is configured as follows:
Copy server: port: 8080 servlet: context-path: /search spring: application: name: search data: elasticsearch: cluster-name: my-cluster cluster-nodes: localhost:9300 jackson: default-property-inclusion: non_null logging: file: application.log path: . level: root: info com.lifengdi.store.client: DEBUG index-entity: configs: - docCode: store indexName: store type: base documentPath: com.lifengdi.document.StoreDocument
spring.data.elasticsearch.cluster-name: cluster name
spring.data.elasticsearch.cluster-nodes: address list of cluster nodes. Multiple nodes are separated by English commas (,)
Create ES documents and maps
First create a JAVA object, and then declare the mapping properties of the field through annotations.
The annotations provided by spring include @ Document, @ id and @ Field, where @ Document acts on the class, @ id and @ Field act on the member variable, and @ id marks a Field as the id primary key.
Copy package com.lifengdi.document; import com.lifengdi.document.store.*; import com.lifengdi.search.annotation.DefinitionQuery; import com.lifengdi.search.enums.QueryTypeEnum; import lombok.Data; import org.springframework.data.annotation.Id; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType; import java.util.List; /** * Store Document * * @author Li Fengdi * @date Create at 19:31 2019/8/22 */ @Document(indexName = "store", type = "base") @Data @DefinitionQuery(key = "page", type = QueryTypeEnum.IGNORE) @DefinitionQuery(key = "size", type = QueryTypeEnum.IGNORE) @DefinitionQuery(key = "q", type = QueryTypeEnum.FULLTEXT) public class StoreDocument { @Id @DefinitionQuery(type = QueryTypeEnum.IN) @DefinitionQuery(key = "id", type = QueryTypeEnum.IN) @Field(type = FieldType.Keyword) private String id; /** * Basic information */ @Field(type = FieldType.Object) private StoreBaseInfo baseInfo; /** * label */ @Field(type = FieldType.Nested) @DefinitionQuery(key = "tagCode", mapped = "tags.key", type = QueryTypeEnum.IN) @DefinitionQuery(key = "tagValue", mapped = "tags.value", type = QueryTypeEnum.AND) @DefinitionQuery(key = "_tagValue", mapped = "tags.value", type = QueryTypeEnum.IN) private List<StoreTags> tags; }
Create index
ElasticsearchTemplate provides four createIndex() methods to create indexes, which can be generated automatically according to class information, or you can manually specify indexName and Settings
Copy @Override public <T> boolean createIndex(Class<T> clazz) { return createIndexIfNotCreated(clazz); } @Override public boolean createIndex(String indexName) { Assert.notNull(indexName, "No index defined for Query"); return client.admin().indices().create(Requests.createIndexRequest(indexName)).actionGet().isAcknowledged(); } @Override public boolean createIndex(String indexName, Object settings) { CreateIndexRequestBuilder createIndexRequestBuilder = client.admin().indices().prepareCreate(indexName); if (settings instanceof String) { createIndexRequestBuilder.setSettings(String.valueOf(settings), Requests.INDEX_CONTENT_TYPE); } else if (settings instanceof Map) { createIndexRequestBuilder.setSettings((Map) settings); } else if (settings instanceof XContentBuilder) { createIndexRequestBuilder.setSettings((XContentBuilder) settings); } return createIndexRequestBuilder.execute().actionGet().isAcknowledged(); } @Override public <T> boolean createIndex(Class<T> clazz, Object settings) { return createIndex(getPersistentEntityFor(clazz).getIndexName(), settings); }
Create mapping
ElasticsearchTemplate provides three putMapping() methods to create mappings
Copy @Override public <T> boolean putMapping(Class<T> clazz) { if (clazz.isAnnotationPresent(Mapping.class)) { String mappingPath = clazz.getAnnotation(Mapping.class).mappingPath(); if (!StringUtils.isEmpty(mappingPath)) { String mappings = readFileFromClasspath(mappingPath); if (!StringUtils.isEmpty(mappings)) { return putMapping(clazz, mappings); } } else { LOGGER.info("mappingPath in @Mapping has to be defined. Building mappings using @Field"); } } ElasticsearchPersistentEntity<T> persistentEntity = getPersistentEntityFor(clazz); XContentBuilder xContentBuilder = null; try { ElasticsearchPersistentProperty property = persistentEntity.getRequiredIdProperty(); xContentBuilder = buildMapping(clazz, persistentEntity.getIndexType(), property.getFieldName(), persistentEntity.getParentType()); } catch (Exception e) { throw new ElasticsearchException("Failed to build mapping for " + clazz.getSimpleName(), e); } return putMapping(clazz, xContentBuilder); } @Override public <T> boolean putMapping(Class<T> clazz, Object mapping) { return putMapping(getPersistentEntityFor(clazz).getIndexName(), getPersistentEntityFor(clazz).getIndexType(), mapping); } @Override public boolean putMapping(String indexName, String type, Object mapping) { Assert.notNull(indexName, "No index defined for putMapping()"); Assert.notNull(type, "No type defined for putMapping()"); PutMappingRequestBuilder requestBuilder = client.admin().indices().preparePutMapping(indexName).setType(type); if (mapping instanceof String) { requestBuilder.setSource(String.valueOf(mapping), XContentType.JSON); } else if (mapping instanceof Map) { requestBuilder.setSource((Map) mapping); } else if (mapping instanceof XContentBuilder) { requestBuilder.setSource((XContentBuilder) mapping); } return requestBuilder.execute().actionGet().isAcknowledged(); }
The test code is as follows
Copy @Test public void testCreate() { System.out.println(elasticsearchTemplate.createIndex(StoreDocument.class)); System.out.println(elasticsearchTemplate.putMapping(StoreDocument.class)); }
Delete index
ElasticsearchTemplate provides two deleteIndex() methods to delete indexes
Copy @Override public <T> boolean deleteIndex(Class<T> clazz) { return deleteIndex(getPersistentEntityFor(clazz).getIndexName()); } @Override public boolean deleteIndex(String indexName) { Assert.notNull(indexName, "No index defined for delete operation"); if (indexExists(indexName)) { return client.admin().indices().delete(new DeleteIndexRequest(indexName)).actionGet().isAcknowledged(); } return false; }
Add & modify document
In Elasticsearch, documents are immutable and cannot be modified. Conversely, if you want to update an existing document, you need to re index or replace it.
Therefore, you can use the same interface as the new interface to modify the document. The distinction is based on id.
Two methods for adding & modifying documents are provided below. One is the index() method provided by ElasticsearchTemplate:
Copy @Override public String index(IndexQuery query) { String documentId = prepareIndex(query).execute().actionGet().getId(); // We should call this because we are not going through a mapper. if (query.getObject() != null) { setPersistentEntityId(query.getObject(), documentId); } return documentId; }
The example code is as follows:
Copy /** * Update index * @param indexName Index name * @param type Index type * @param id ID * @param jsonDoc JSON Formatted document * @param refresh Refresh index * @return ID */ public String index(String indexName, String type, String id, JsonNode jsonDoc, boolean refresh) throws JsonProcessingException { log.info("AbstractDocumentIndexService Update index.indexName:{},type:{},id:{},jsonDoc:{}", indexName, type, id, jsonDoc); IndexQuery indexQuery = new IndexQueryBuilder() .withIndexName(indexName) .withType(type) .withId(id) .withSource(objectMapper.writeValueAsString(jsonDoc)) .build(); try { if (elasticsearchTemplate.indexExists(indexName)) { String index = elasticsearchTemplate.index(indexQuery); if (refresh) { elasticsearchTemplate.refresh(indexName); } return index; } } catch (Exception e) { log.error("Failed to update index,Refresh ES retry ", e); elasticsearchTemplate.refresh(indexName); return elasticsearchTemplate.index(indexQuery); } throw BaseException.INDEX_NOT_EXISTS_EXCEPTION.build(); }
The other is through the Repository interface. The Repository interface of ES provided by Spring is ElasticsearchCrudRepository, so we can directly define a new interface and implement ElasticsearchCrudRepository:
Copy package com.taoche.docindex.repo; import com.taoche.document.StoreDocument; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; /** * Store Repository * @author Li Fengdi * @date Create at 09:30 2019/8/23 */ public interface StoreRepository extends ElasticsearchRepository<StoreDocument, String> { }
The example code is as follows:
Copy @Test public void testSave() { StoreDocument storeDocument = new StoreDocument(); storeDocument.setId("1"); StoreBaseInfo baseInfo = new StoreBaseInfo(); baseInfo.setStoreId("1"); baseInfo.setCreatedTime(DateTime.now()); storeDocument.setBaseInfo(baseInfo); storeRepository.save(storeDocument); }
query
The main function of ES is query. ElasticsearchRepository also provides basic query interfaces, such as findById(), findAll(), findAllById(), search(); Of course, you can also use another function provided by Spring Data: Spring Data JPA - to create a query by method name, of course, you need to follow certain rules. For example, your method name is findByTitle(), then it will know that you query according to the title, and then automatically help you complete it. I won't talk about it carefully here.
The above can basically meet the general query, but a more complex query can't help. This requires custom query. Here you can check another blog of mine. SpringBoot uses annotation to build Elasticsearch query statements to realize multi condition complex query. Here's a detailed description.
In addition, there is a powerful function, Elasticsearch aggregation; Aggregation mainly realizes the statistics and analysis of data. This is not used for the time being, so the partners who want to see the aggregation function may be disappointed ~ hahaha~~~
The aggregation function will be said separately when there is time ~ there will be.
So far, SpringBoot integration Elasticsearch has basically ended. Please leave a message if you don't understand~