【Spring Boot 2.0学习项目】SpringBoot+ElasticSearch博客检索系统

【Spring Boot 2.0学习项目】SpringBoot+ElasticSearch博客检索系统 SpringBoot+ElasticSearch博客检索系统 一、初识ElasticSearch 1.ElasticSearch概念和适用场景

分布式；全文检索；实时快速；Restful API； 2.ElasticSearch VS MySQL

例子：

MySql：select * from user.user_info where name = '张三';

ES：GET /user/user_info/_search?q=name:张三

3.ElasticSearch、kibana安装

ElasticSearch下载

启动：/bin/elasticsearch.bat

展示：

Kibana下载地址

启动：/bin/kibana.bat

展示：

4.演示postman、kibana对es的交互

postman对es的交互API：

查看所有索引： GET localhost:9200/_all创建索引-test：PUT localhost:9200/test删除索引-test：DELETe localhost:9200/test创建索引-person： PUT localhost:9200/person新增数据-person-1：PUT localhost:9200/person/_doc/1

{
    "first_name":"John",
    "last_name":"Smith",
    "age":25,
    "about":"i love  to go rock climbing",
    "interests":["sports","music"]
}

新增数据-person-2：PUT localhost:9200/person/_doc/2

{
    "first_name":"Eric",
    "last_name":"Smith",
    "age":23,
    "about":"i love  basketball",
    "interests":["sports","reading"]
}

搜索数据-person-id： GET localhost:9200/person/_doc/1搜索数据-person-name：GET localhost:9200/person/_doc/_search?q=first_name:john

kibana对es的交互

/dev tools/console下

查看所有索引: GET _all

查看id为1的数据：GET /person/_doc/1

搜索查询数据：

POST /person/_doc/_search
{
  "query":{
    "bool": {
      "should": [
        {"match": {
          "first_name": "Eric"
        }}
      ]
    }
  }
}

二、博客网站全文检索 1.MySql与ElasticSearch语句测试检索速度 1.1 Mysql建表测试检索速度

SET FOREIGN_KEY_CHECKS=0;

-- ----------------------------
-- Table structure for `t_blog`
-- ----------------------------
DROP TABLE IF EXISTS `t_blog`;
CREATE TABLE `t_blog` (
  `id` int(11) NOT NULL AUTO_INCREMENT COMMENT '自增id',
  `title` varchar(60) DEFAULT NULL COMMENT '博客标题',
  `author` varchar(60) DEFAULT NULL COMMENT '博客作者',
  `content` mediumtext COMMENT '博客内容',
  `create_time` datetime DEFAULT NULL COMMENT '创建时间',
  `update_time` datetime DEFAULT NULL COMMENT '更新时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=12 DEFAULT CHARSET=utf8mb4;

-- ----------------------------
-- Records of t_blog
-- ----------------------------
INSERT INTO `t_blog` VALUES ('1', 'Springboot 为什么这', 'bywind', '没错 Springboot ', '2019-12-08 01:44:29', '2019-12-08 01:44:34');
INSERT INTO `t_blog` VALUES ('3', 'Springboot 中 Redis', 'bywind', 'Spring Boot', '2019-12-08 01:44:29', '2019-12-08 01:44:29');
INSERT INTO `t_blog` VALUES ('4', 'Springboot 中如何优化', 'bywind', null, '2019-12-08 01:44:29', '2019-12-08 01:44:29');
INSERT INTO `t_blog` VALUES ('5', 'Springboot 消息队列', 'bywind', null, '2019-12-08 01:44:29', '2019-12-08 01:44:29');
INSERT INTO `t_blog` VALUES ('6', 'Docker Compose + Springboot', 'bywind', null, '2019-12-08 01:44:29', '2019-12-08 01:44:29');

查询语句：

select * from t_blog where title like "%spring%" or content like "%spring%";

1.2 ElastciSearch为什么搜索快呢？

底层基于倒排索引

分布式

2.Mysql与ElasticSearch同步中间件 2.1 开源的中间件介绍

binlog订阅

alibaba/canal：阿里巴巴开源组件。MySQL binlog增量订阅&消费组件go-mysql-elasticsearch：go语言的组件。logstash:官方提供的组件

2.2 logstash来增量、全量同步数据解决方案

Logstash下载地址

mysql-connector-java下载地址

在下载的logstash中放入mysql-connector-java

在config中新建一个mysql.conf文件，内容如下：

input {
	jdbc {
		# jdbc驱动包位置
		jdbc_driver_library => "D:softwareelasticsearchlogstash-6.3.2logstash-6.3.2\mysql-connector-java-5.1.31.jar"
		# 要使用的驱动包类
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        # mysql数据库的连接信息
        jdbc_connection_string => "jdbc:mysql://localhost:3306/blog?serverTimezone=UTC&characterEncoding=utf8"
        # mysql用户
        jdbc_user => "root"
        # mysql密码
        jdbc_password => "123"

        # 定时任务，默认一分钟，"* * * * *"代表设置为无延迟
        schedule => "* * * * *"

        # 清空上一次sql_last_value记录
        clean_run => true
        # 要执行的sql语句
        statement => "SELECt * FROM t_blog WHERe update_time > date_add(:sql_last_value, interval 8 hour) AND update_time ["127.0.0.1:9200"]
        #索引
        index => "blog"
        # ——id
        document_id => "%{id}"

    }

}

启动方式：

D:softwareelasticsearchlogstash-6.3.2logstash-6.3.2bin>logstash -f ../config/mysql.conf

在kibana中验证：

GET /blog/_stats, 查看其_all底下的count字段。

3.ElasticSearch内置分词器 3.1 内置分词器

standard;

standard : ES默认分词器，将单词转换为小写，去除停用词与符号，支持中文——单字切分

simple;

simple ：通过非字母字符进行切分，统一化为小写，去除数字类型字符

whitespace;

whitespace ：不支持中文，不转换为小写，只去除空格，

language;

language ：特定语言的分词器，不支持中文

测试分词效果：

3.2 引入elasticsearch-analysis-ik分词器

从github下载对应版本的分词器。解压之后，在elasticsearch中的plugins中新建一个ik文件夹，放入解压之后的文件即可。重启之后

三、springboot+elasticsearch实现博客检索功能 1.环境配置以及项目结构

（1）pom文件

    4.0.0
    
        org.springframework.boot
        spring-boot-starter-parent
        2.1.1.RELEASE
         
    
    com.lcz
    elasticsearch_blog
    0.0.1-SNAPSHOT
    elasticsearch_blog
    Demo project for Spring Boot
    
        1.8
    
    
        
            org.springframework.boot
            spring-boot-starter-data-elasticsearch
        

        
            org.springframework.boot
            spring-boot-starter-data-jpa
        
        
            org.springframework.boot
            spring-boot-starter-web
        

        
            org.springframework.boot
            spring-boot-devtools
            runtime
            true
        
        
            mysql
            mysql-connector-java
            runtime
        
        
            org.springframework.boot
            spring-boot-configuration-processor
            true
        
        
            org.projectlombok
            lombok
            true
        
        
            org.springframework.boot
            spring-boot-starter-test
            test
        
    

    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
                
                    
                        
                            org.projectlombok
                            lombok
                        
                    
                
            
        
    

（2）application.properties配置选项

通用数据源配置
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/blog?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=123
#Hikari数据源专用配置
spring.datasource.hikari.maximum-pool-size=20
spring.datasource.hikari.minimum-idle=5
#JPA相关配置
spring.jpa.database-platform=org.hibernate.dialect.MySQLDialect
#es
spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300
spring.data.elasticsearch.cluster-name=elasticsearch
#mVc
spring.mvc.static-path-pattern=


import lombok.Data;

import javax.persistence.*;
import java.util.Date;



@Data
@Table(name = "t_blog")
@Entity
public class MySqlBlog {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;
    private String title;
    private String author;
    @Column(columnDefinition = "mediumtext")
    private String content;
    private Date createTime;
    private Date updateTime;

}

package com.lcz.elasticsearch_blog.respository.mysql;

import com.lcz.elasticsearch_blog.entity.mysql.MySqlBlog;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;

import java.util.List;


public interface MySqlBlogRespository extends JpaRepository {
    @Query("select e from MySqlBlog e order by e.createTime desc")
    List queryAll();
    @Query("select e from MySqlBlog e where e.title like concat('%',:keyword,'%') " +
            "or e.content like concat('%',:keyword,'%') order by e.createTime desc")
    List queryBlogs(@Param("keyword") String keyword);
}

ES出发

package com.lcz.elasticsearch_blog.entity.es;

import com.fasterxml.jackson.annotation.JsonAlias;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

import javax.persistence.*;
import java.util.Date;



import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

import java.util.Date;


@Data
@document(indexName = "blog", type = "doc",
        useServerConfiguration = true, createIndex = false)
public class EsBlog {
    @Id
    private Integer id;
    @Field(type = FieldType.Text, analyzer = "ik_max_work")
    private String title;
    @Field(type = FieldType.Text, analyzer = "ik_max_work")
    private String author;
    @Field(type = FieldType.Text, analyzer = "ik_max_work")
    private String content;
    @Field(type = FieldType.Date, format = DateFormat.custom,
            pattern = "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis")
    @JsonAlias(value = "create_time")
    private Date createTime;
    @Field(type = FieldType.Date, format = DateFormat.custom,
            pattern = "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis")
    @JsonAlias(value = "update_time")
    private Date updateTime;

    public String getTitle() {
        return title;
    }
}

package com.lcz.elasticsearch_blog.respository.es;

import com.lcz.elasticsearch_blog.entity.es.EsBlog;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

public interface EsBlogRepository extends ElasticsearchRepository {

}

controller

package com.lcz.elasticsearch_blog.controller;

import com.lcz.elasticsearch_blog.entity.mysql.MySqlBlog;
import com.lcz.elasticsearch_blog.respository.mysql.MySqlBlogRespository;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;

import java.util.List;



@Controller
@Slf4j
public class IndexController {
    @Autowired
    private MySqlBlogRespository mySqlBlogRespository;

    @RequestMapping("/")
    public String index(){
        List all = mySqlBlogRespository.findAll();
        System.out.println(all.size());
        return "index.html";
    }
}

package com.lcz.elasticsearch_blog.controller;

import com.lcz.elasticsearch_blog.entity.es.EsBlog;
import com.lcz.elasticsearch_blog.entity.mysql.MySqlBlog;
import com.lcz.elasticsearch_blog.respository.es.EsBlogRepository;
import com.lcz.elasticsearch_blog.respository.mysql.MySqlBlogRespository;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.util.StopWatch;
import org.springframework.web.bind.annotation.*;

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;



@RestController
@Slf4j
public class DataController {

    @Autowired
    private MySqlBlogRespository mysqlBlogRepository;
    @Autowired
    private EsBlogRepository esBlogRepository;

    @GetMapping("/blogs")
    public Object blog(){
        List mysqlBlogs = mysqlBlogRepository.queryAll();
        return mysqlBlogs;
    }

    @PostMapping("/search")
    public Object search(@RequestBody Param param){
        Map map = new HashMap<>();
        String type = param.getType();
        StopWatch watch = new StopWatch();
        watch.start();
        if(type.equalsIgnoreCase("mysql")){
            List mysqlBlogs = mysqlBlogRepository.queryBlogs(param.getKeyword());
            map.put("list",mysqlBlogs);
        }else if(type.equalsIgnoreCase("es")){
            BoolQueryBuilder builder = QueryBuilders.boolQuery();
            builder.should(QueryBuilders.matchPhraseQuery("title",param.getKeyword()));
            builder.should(QueryBuilders.matchPhraseQuery("content",param.getKeyword()));
            String s = builder.toString();
            System.out.println("======");
            System.out.println(s);
            System.out.println("======");
            Page esBlogs = (Page) esBlogRepository.search(builder);
            List content = esBlogs.getContent();
            map.put("list",content);
        }else {
            return ">>> 不知道 <<<";
        }
        watch.stop();
        long totalTimeMillis = watch.getTotalTimeMillis();
        map.put("duration",totalTimeMillis);
        return map;
    }

    @GetMapping("/blog/{id}")
    public Object blog(@PathVariable Integer id){
        Optional byId = mysqlBlogRepository.findById(id);
        return byId.get();
    }


    @Data
    public static class Param{
        // String,es
        private String type;
        private String keyword;

        public String getType() {
            return type;
        }

        public void setType(String type) {
            this.type = type;
        }

        public String getKeyword() {
            return keyword;
        }

        public void setKeyword(String keyword) {
            this.keyword = keyword;
        }
    }
}

3.源码

github下载地址