Compare commits

...

29 Commits

Author SHA1 Message Date
xxy
9975faed01 点击量计算优化 2020-05-26 02:27:39 +08:00
xxy
8fde3a3725 点击量计算优化 2020-05-26 02:13:28 +08:00
xxy
430504ee28 点击量计算优化 2020-05-26 02:10:37 +08:00
xxy
16447c60ec 引入redisson 2020-05-26 00:28:12 +08:00
xxy
be7cbe2d6f 欲引入Redisson框架实现分布式锁 2020-05-26 00:04:00 +08:00
xxy
8f1ed88b07 es采集优化 2020-05-25 23:38:46 +08:00
xxy
9b9851e7ab 文档更新 2020-05-25 23:16:08 +08:00
a55edf0408 es优化 2020-05-25 21:37:00 +08:00
856c4c0667 es优化 2020-05-25 21:26:47 +08:00
e4dd5bcb71 引入rabbitmq流量削峰,累积点击量后统一更新 2020-05-25 21:06:51 +08:00
5dbddbdd96 es导入优化 2020-05-25 17:51:05 +08:00
73be43e1c5 优化es采集 2020-05-25 17:11:03 +08:00
ce2a3b4647 集成redis 2020-05-25 16:54:39 +08:00
68abdeca93 文档更新 2020-05-25 10:59:24 +08:00
0b505a3922 v2.1.2发布 2020-05-25 10:51:22 +08:00
353cb8c536 修复网路图片保存时文件损坏的情况 2020-05-25 10:40:50 +08:00
xxy
e1e1310b9e 解决book_index的主键生成冲突 2020-05-24 17:59:03 +08:00
4c42ac0d29 文档更新 2020-05-24 09:17:49 +08:00
d025d3d514 优化列表页显示 2020-05-24 02:04:02 +08:00
a0fb8e481a 1.解决爬虫线程停止失败的bug,2新增新笔趣阁源,兼容更多源站 2020-05-24 00:54:27 +08:00
80b933db8d v2.1.1发布 2020-05-22 11:58:42 +08:00
87a060280a 1.解决小说/章节发布表单重复提交问题,2.解决小说章节发布字数计算问题,3.解决个人发布小说字体更改问题 2020-05-22 11:49:18 +08:00
9c78f400dc v2.1.0发布 2020-05-21 09:15:47 +08:00
c2cdf73893 1优化网络图片转换策略 2020-05-21 08:46:46 +08:00
a13ea78c3f 1优化es导入策略,2增加默认图片,在读取网络图片失败时设置,防止一直失败重试 2020-05-21 08:16:37 +08:00
0144b77983 v2.1.0发布 2020-05-20 22:13:53 +08:00
07b9ffde96 修改版本号 2020-05-20 21:30:19 +08:00
002a0723f7 索引+搜索优化 2020-05-20 21:23:59 +08:00
8a628f081f 集成elasticsearch搜索引擎 2020-05-20 18:08:29 +08:00
40 changed files with 1214 additions and 212 deletions

View File

@ -40,7 +40,28 @@ novel-plus -- 父工程
```
#### 技术选型
Springboot+Mybatis+Mysql+Ehcache+Thymeleaf+Layui
| 技术 | 说明
| -------------------- | ---------------------------
| SpringBoot | Spring应用快速开发脚手架
| MyBatis | 持久层ORM框架
| MyBatis Dynamic SQL | Mybatis动态sql
| PageHelper | MyBatis分页插件
| MyBatisGenerator | 持久层代码生成插件
| JJWT | JWT登录支持
| SpringSecurity | 安全框架
| Shiro | 安全框架
| Ehcache | Java进程内缓存框架(默认缓存)
| Redis | 分布式缓存(缓存替换方案默认关闭一行配置开启)
| ElasticSearch | 搜索引擎(搜索增强方案默认关闭一行配置开启)
| RabbitMq | 消息队列(流量削峰默认关闭一行配置开启)
| Redisson | 实现分布式锁
| Lombok | 简化对象封装工具
| Docker | 应用容器引擎
| Mysql | 数据库服务
| Thymeleaf | 模板引擎
| Layui | 前端UI
#### PC站截图
@ -52,69 +73,67 @@ Springboot+Mybatis+Mysql+Ehcache+Thymeleaf+Layui
![img](https://oscimg.oschina.net/oscnet/up-d0b2e03129bfae47b8bb96a491b73d383c5.png)
3. 排行榜
3. 搜索页
![img](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.0/assets/QQ20200520-215756.png)
![img](https://oscimg.oschina.net/oscnet/up-ed5f689557718924acac76bc3ebca36afcb.png)
4. 排行榜
![img](https://oscimg.oschina.net/oscnet/up-78d5a68586cd92a86c669311f414508f922.png)
4. 详情页
5. 详情页
![img](https://oscimg.oschina.net/oscnet/up-8be2495a2869f93626b0c9c1df6f329747a.png)
5. 阅读页
6. 阅读页
![img](https://oscimg.oschina.net/oscnet/up-517c84148d2db8e11717a8bbecc57fa1be7.png)
6. 用户中心
7. 用户中心
![img](https://oscimg.oschina.net/oscnet/up-805a30e7a663a3fd5cb39a7ea26bc132a01.png)
7. 充值
8. 充值
![img](https://oscimg.oschina.net/oscnet/up-5a601b0b3af3224d0bebcfe12fc15075d34.png)
![img](https://oscimg.oschina.net/oscnet/up-face25d02c07b05b2ce954cc4bf4ee6a0cc.png)
8.作家专区
9. 作家专区
![img](https://oscimg.oschina.net/oscnet/up-30766372cc7f56480ff1d7d55198204f6ea.png)
![img](https://oscimg.oschina.net/oscnet/up-9737995e465b86f3bee3211221f6c3b8a56.png)
9.购买
10. 购买
![img](https://oscimg.oschina.net/oscnet/up-ce0f585efd82a9670335f118cf5897c85e9.png)
![img](https://oscimg.oschina.net/oscnet/up-f849960f4c1303fea77d26e64fc505a7180.png)
#### 手机站截图
1. 首页
![index](./assets/QQ%E5%9B%BE%E7%89%8720191018162208.jpg)
![index](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.0/assets/QQ%E5%9B%BE%E7%89%8720191018162208.jpg)
2. 小说详情页
![微信图片_20190904181558](./assets/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20190904181558.png)
![微信图片_20190904181558](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.0/assets/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20190904181558.png)
3. 目录页
![QQ图片20191018161901](./assets/QQ%E5%9B%BE%E7%89%8720191108022250.png)
![QQ图片20191018161901](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.0/assets/QQ%E5%9B%BE%E7%89%8720191108022250.png)
4. 小说阅读页
![QQ图片20191018161901](./assets/QQ%E5%9B%BE%E7%89%8720191018161901.png)
![QQ图片20191018161901](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.0/assets/QQ%E5%9B%BE%E7%89%8720191018161901.png)
#### 爬虫管理系统截图
![QQ图片20191018161901](./assets/crawl_index.png)
![img](https://gitee.com/xiongxyang/novel-plus/raw/release_v2.1.2/assets/crawl_index.png)
#### 后台管理系统截图

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 62 KiB

82
es/index_create.txt Normal file
View File

@ -0,0 +1,82 @@
PUT /novel
{
"mappings" : {
"book" : {
"properties" : {
"id" : {
"type" : "long"
},
"authorId" : {
"type" : "long"
},
"authorName" : {
"type" : "text",
"analyzer": "ik_smart",
"boost": 1.9
},
"bookName" : {
"type" : "text",
"analyzer": "ik_smart",
"boost": 2
},
"bookDesc" : {
"type" : "text",
"analyzer": "ik_smart",
"boost": 0.1
},
"bookStatus" : {
"type" : "short"
},
"catId" : {
"type" : "integer"
},
"catName" : {
"type" : "text",
"analyzer": "ik_smart",
"boost": 1.0
},
"lastIndexId" : {
"type" : "long"
},
"lastIndexName" : {
"type" : "text",
"analyzer": "ik_smart",
"boost": 0.1
},
"lastIndexUpdateTime" : {
"type": "keyword"
},
"picUrl" : {
"type" : "keyword"
},
"score" : {
"type" : "float"
},
"wordCount" : {
"type" : "integer"
},
"workDirection" : {
"type" : "short"
},
"visitCount" : {
"type": "long"
}
}
}
}
}

6
novel-admin/Dockerfile Normal file
View File

@ -0,0 +1,6 @@
FROM java:8
ADD novel-admin-1.0.0.jar /root
ENV dburl=""
ENV username=""
ENV password=""
ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -jar /root/novel-admin-1.0.0.jar"]

File diff suppressed because one or more lines are too long

View File

@ -5,7 +5,7 @@
<parent>
<artifactId>novel</artifactId>
<groupId>com.java2nb</groupId>
<version>2.0.2</version>
<version>2.1.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@ -31,11 +31,20 @@
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<!--ehcache-->
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache</artifactId>
</dependency>
<!--集成redis-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-redis</artifactId>
<version>${redis.version}</version>
</dependency>
<!--mysql驱动-->
<dependency>
<groupId>mysql</groupId>

View File

@ -46,4 +46,23 @@ public interface CacheKey {
* */
String RUNNING_CRAWL_THREAD_KEY_PREFIX = "runningCrawlTreadDataKeyPrefix";
/**
* 上一次搜索引擎更新的时间
* */
String ES_LAST_UPDATE_TIME = "esLastUpdateTime";
/**
* 搜索引擎转换锁
* */
String ES_TRANS_LOCK = "esTransLock";
/**
* 上一次搜索引擎是否更新过小说点击量
* */
String ES_IS_UPDATE_VISIT = "esIsUpdateVisit";
/**
* 累积的小说点击量
* */
String BOOK_ADD_VISIT_COUNT = "bookAddVisitCount";
}

View File

@ -51,9 +51,5 @@ public interface CacheService {
* */
void expire(String key, long timeout);
/**
* 刷新缓存
* */
void refresh(String key);
}

View File

@ -1,22 +1,23 @@
package com.java2nb.novel.core.cache.impl;
import com.java2nb.novel.core.cache.CacheService;
import lombok.RequiredArgsConstructor;
import net.sf.ehcache.Cache;
import net.sf.ehcache.CacheManager;
import net.sf.ehcache.Element;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;
/**
* @author xxy
*/
@ConditionalOnProperty(prefix = "cache", name = "type", havingValue = "ehcache")
@RequiredArgsConstructor
@Service
public class EhCacheServiceImpl implements CacheService {
@Autowired
private CacheManager cacheManager ;
private static final String CACHE_NAME = "utilCache";
private final CacheManager cacheManager ;
/**
@ -30,14 +31,6 @@ public class EhCacheServiceImpl implements CacheService {
}
public CacheManager getCacheManager() {
return cacheManager;
}
@Override
public String get(String key) {
Element element = getCache().get(key);
@ -125,20 +118,6 @@ public class EhCacheServiceImpl implements CacheService {
}
@Override
public void refresh(String key) {
Element element = getCache().get(key);
if (element != null) {
Object value = element.getValue();
int timeToLive = element.getTimeToLive();
element = new Element(key, value);
element.setTimeToLive(timeToLive);
Cache cache = getCache();
cache.put(element);
}
}
}

View File

@ -0,0 +1,73 @@
package com.java2nb.novel.core.cache.impl;
import com.java2nb.novel.core.cache.CacheService;
import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;
/**
* @author xxy
*/
@ConditionalOnProperty(prefix = "cache", name = "type", havingValue = "redis")
@RequiredArgsConstructor
@Service
public class RedisServiceImpl implements CacheService {
private final StringRedisTemplate stringRedisTemplate;
private final RedisTemplate<Object, Object> redisTemplate;
@Override
public String get(String key) {
return stringRedisTemplate.opsForValue().get(key);
}
@Override
public void set(String key, String value) {
stringRedisTemplate.opsForValue().set(key,value);
}
@Override
public void set(String key, String value, long timeout) {
stringRedisTemplate.opsForValue().set(key,value,timeout, TimeUnit.SECONDS);
}
@Override
public Object getObject(String key) {
return redisTemplate.opsForValue().get(key);
}
@Override
public void setObject(String key, Object value) {
redisTemplate.opsForValue().set(key,value);
}
@Override
public void setObject(String key, Object value, long timeout) {
redisTemplate.opsForValue().set(key,value,timeout, TimeUnit.SECONDS);
}
@Override
public void del(String key) {
redisTemplate.delete(key);
stringRedisTemplate.delete(key);
}
@Override
public boolean contains(String key) {
return redisTemplate.hasKey(key) || stringRedisTemplate.hasKey(key);
}
@Override
public void expire(String key, long timeout) {
redisTemplate.expire(key,timeout, TimeUnit.SECONDS);
stringRedisTemplate.expire(key,timeout, TimeUnit.SECONDS);
}
}

View File

@ -58,13 +58,16 @@ public enum ResponseStatus {
, BOOKNAME_EXISTS(4003,"已发布过同名小说!")
,
/**
* 搜索引擎相关错误
* */
ES_SEARCH_FAIL(9001,"密码错误!"),
/**
* 其他通用错误
* */
PASSWORD_ERROR(88001,"密码错误!");
private int code;
private String msg;

View File

@ -1,6 +1,7 @@
package com.java2nb.novel.core.utils;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.Charsets;
import org.apache.http.client.utils.DateUtils;
import org.springframework.core.io.Resource;
@ -9,6 +10,7 @@ import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import javax.imageio.ImageIO;
import java.io.*;
import java.util.Date;
import java.util.Objects;
@ -17,34 +19,65 @@ import java.util.Objects;
* 文件操作工具类
* @author 11797
*/
@Slf4j
public class FileUtil {
/**
* 网络图片转本地
* */
@SneakyThrows
public static String network2Local(String picSrc,String picSavePath,String visitPrefix) {
//本地图片保存
HttpHeaders headers = new HttpHeaders();
HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
ResponseEntity<Resource> resEntity = RestTemplateUtil.getInstance(Charsets.ISO_8859_1.name()).exchange(picSrc, HttpMethod.GET, requestEntity, Resource.class);
InputStream input = Objects.requireNonNull(resEntity.getBody()).getInputStream();
Date currentDate = new Date();
picSrc = visitPrefix + DateUtils.formatDate(currentDate, "yyyy") + "/" + DateUtils.formatDate(currentDate, "MM") + "/" + DateUtils.formatDate(currentDate, "dd") + "/"
+ UUIDUtil.getUUID32()
+ picSrc.substring(picSrc.lastIndexOf("."));
File picFile = new File(picSavePath + picSrc);
File parentFile = picFile.getParentFile();
if (!parentFile.exists()) {
parentFile.mkdirs();
InputStream input = null;
OutputStream out = null;
try {
//本地图片保存
HttpHeaders headers = new HttpHeaders();
HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
ResponseEntity<Resource> resEntity = RestTemplateUtil.getInstance(Charsets.ISO_8859_1.name()).exchange(picSrc, HttpMethod.GET, requestEntity, Resource.class);
input = Objects.requireNonNull(resEntity.getBody()).getInputStream();
Date currentDate = new Date();
picSrc = visitPrefix + DateUtils.formatDate(currentDate, "yyyy") + "/" + DateUtils.formatDate(currentDate, "MM") + "/" + DateUtils.formatDate(currentDate, "dd") + "/"
+ UUIDUtil.getUUID32()
+ picSrc.substring(picSrc.lastIndexOf("."));
File picFile = new File(picSavePath + picSrc);
File parentFile = picFile.getParentFile();
if (!parentFile.exists()) {
parentFile.mkdirs();
}
out = new FileOutputStream(picFile);
byte[] b = new byte[4096];
for (int n; (n = input.read(b)) != -1; ) {
out.write(b, 0, n);
}
out.flush();
if( ImageIO.read(picFile) == null){
picSrc = "/images/default.gif";
}
}catch (Exception e){
log.error(e.getMessage(),e);
picSrc = "/images/default.gif";
}finally {
if(input != null){
try {
input.close();
} catch (IOException e) {
log.error(e.getMessage(),e);
}finally {
if(out != null){
try {
out.close();
} catch (IOException e) {
log.error(e.getMessage(),e);
}
}
}
}
}
OutputStream out = new FileOutputStream(picFile);
byte[] b = new byte[4096];
for (int n; (n = input.read(b)) != -1; ) {
out.write(b, 0, n);
}
out.close();
input.close();
return picSrc;
}

View File

@ -0,0 +1,73 @@
package com.java2nb.novel.core.utils;
import org.apache.commons.lang3.StringUtils;
import java.util.Arrays;
/**
* @author xiongxiaoyang
*/
public class StringUtil {
/**
* 将驼峰式命名的字符串转换为下划线大写方式。如果转换前的驼峰式命名的字符串为空,则返回空字符串。</br>
* 例如HelloWorld->HELLO_WORLD
* @param name 转换前的驼峰式命名的字符串
* @return 转换后下划线大写方式命名的字符串
*/
public static String underscoreName(String name) {
StringBuilder result = new StringBuilder();
if (name != null && name.length() > 0) {
// 将第一个字符处理成大写
result.append(name.substring(0, 1).toUpperCase());
// 循环处理其余字符
for (int i = 1; i < name.length(); i++) {
String s = name.substring(i, i + 1);
// 在大写字母前添加下划线
if (s.equals(s.toUpperCase()) && !Character.isDigit(s.charAt(0))) {
result.append("_");
}
// 其他字符直接转成大写
result.append(s.toUpperCase());
}
}
return result.toString();
}
/**
* 将下划线大写方式命名的字符串转换为驼峰式。如果转换前的下划线大写方式命名的字符串为空,则返回空字符串。</br>
* 例如HELLO_WORLD->HelloWorld
* @param name 转换前的下划线大写方式命名的字符串
* @return 转换后的驼峰式命名的字符串
*/
public static String camelName(String name) {
StringBuilder result = new StringBuilder();
// 快速检查
if (name == null || name.isEmpty()) {
// 没必要转换
return "";
} else if (!name.contains("_")) {
// 不含下划线,仅将首字母小写
return name.substring(0, 1).toLowerCase() + name.substring(1);
}
// 用下划线将原始字符串分割
String camels[] = name.split("_");
for (String camel : camels) {
// 跳过原始字符串中开头、结尾的下换线或双重下划线
if (camel.isEmpty()) {
continue;
}
// 处理真正的驼峰片段
if (result.length() == 0) {
// 第一个驼峰片段,全部字母都小写
result.append(camel.toLowerCase());
} else {
// 其他的驼峰片段,首字母大写
result.append(camel.substring(0, 1).toUpperCase());
result.append(camel.substring(1).toLowerCase());
}
}
return result.toString();
}
}

View File

@ -4,8 +4,27 @@ spring:
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
password:
driver-class-name: com.mysql.cj.jdbc.Driver
#Redis服务器IP
redis:
host: 127.0.0.1
#Redis服务器连接端口
port: 6379
#Redis服务器连接密码
password: test
jedis:
pool:
#连接池最大连接数使用负值表示没有限制
max-active: 8
#连接池最大阻塞等待时间使用负值表示没有限制
max-wait: 1
#连接池最大阻塞等待时间使用负值表示没有限制
max-idle: 8
#连接池中的最小空闲连接
min-idle: 0
#连接超时时间毫秒
timeout: 30000

View File

@ -11,6 +11,9 @@ spring:
generator:
write-numbers-as-strings: true
#缓存类型ehcache(默认)redis
cache:
type: ehcache
mybatis:
configuration:

View File

@ -5,7 +5,7 @@
<parent>
<artifactId>novel</artifactId>
<groupId>com.java2nb</groupId>
<version>2.0.2</version>
<version>2.1.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@ -30,6 +30,8 @@ import static java.util.regex.Pattern.compile;
@Slf4j
public class CrawlParser {
private static IdWorker idWorker = new IdWorker();
public static final Integer BOOK_INDEX_LIST_KEY = 1;
public static final Integer BOOK_CONTENT_LIST_KEY = 2;
@ -94,6 +96,8 @@ public class CrawlParser {
String desc = bookDetailHtml.substring(bookDetailHtml.indexOf(ruleBean.getDescStart()) + ruleBean.getDescStart().length());
desc = desc.substring(0, desc.indexOf(ruleBean.getDescEnd()));
//过滤掉简介中的a标签
desc = desc.replaceAll("<a[^<]+</a>","");
//设置书籍简介
book.setBookDesc(desc);
if (StringUtils.isNotBlank(ruleBean.getStatusPatten())) {
@ -173,6 +177,7 @@ public class CrawlParser {
String lastIndexName = null;
while (isFindIndex) {
BookIndex hasIndex = hasIndexs.get(indexNum);
String indexName = indexNameMatch.group(1);
@ -199,7 +204,7 @@ public class CrawlParser {
if(hasIndexs.size() == 0){
//新书入库
//设置目录和章节内容
Long indexId = new IdWorker().nextId();
Long indexId = idWorker.nextId();
lastIndexId = indexId;
lastIndexName = indexName;
bookIndex.setId(indexId);

View File

@ -167,6 +167,11 @@ public class CrawlServiceImpl implements CrawlService {
boolean isFindBookId = bookIdMatcher.find();
while (isFindBookId) {
try {
if(Thread.currentThread().isInterrupted()){
return;
}
String bookId = bookIdMatcher.group(1);
Book book = CrawlParser.parseBook(ruleBean, bookId);
//这里只做新书入库,查询是否存在这本书

View File

@ -5,7 +5,7 @@
<parent>
<artifactId>novel</artifactId>
<groupId>com.java2nb</groupId>
<version>2.0.2</version>
<version>2.1.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@ -27,6 +27,31 @@
<version>${jjwt.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>io.searchbox</groupId>
<artifactId>jest</artifactId>
<version>${jest.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<!--引入redisson分布式锁-->
<!-- <dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson-spring-boot-starter</artifactId>
<version>${redisson.version}</version>
</dependency>-->
<dependency>
<groupId>com.alipay.sdk</groupId>

View File

@ -55,7 +55,7 @@ public class AuthorController extends BaseController{
* 发布小说
* */
@PostMapping("addBook")
public ResultBean addBook(Book book,HttpServletRequest request){
public ResultBean addBook(@RequestParam("bookDesc") String bookDesc,Book book,HttpServletRequest request){
//查询作家信息
Author author = authorService.queryAuthor(getUserDetails(request).getId());
@ -67,6 +67,10 @@ public class AuthorController extends BaseController{
}
//bookDesc不能使用book对象来接收否则会自动去掉前面的空格
book.setBookDesc(bookDesc
.replaceAll("\\n","<br>")
.replaceAll("\\s","&nbsp;"));
//发布小说
bookService.addBook(book,author.getId(),author.getPenName());
@ -109,6 +113,8 @@ public class AuthorController extends BaseController{
return ResultBean.fail(ResponseStatus.AUTHOR_STATUS_FORBIDDEN);
}
content = content.replaceAll("\\n","<br>")
.replaceAll("\\s","&nbsp;");
//发布章节内容
bookService.addBookContent(bookId,indexName,content,author.getId());

View File

@ -4,14 +4,14 @@ import com.github.pagehelper.PageInfo;
import com.java2nb.novel.core.bean.ResultBean;
import com.java2nb.novel.core.bean.UserDetails;
import com.java2nb.novel.core.enums.ResponseStatus;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.entity.BookComment;
import com.java2nb.novel.entity.BookIndex;
import com.java2nb.novel.search.BookSP;
import com.java2nb.novel.service.BookService;
import com.java2nb.novel.vo.BookVO;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
@ -19,7 +19,6 @@ import org.springframework.web.bind.annotation.RestController;
import javax.servlet.http.HttpServletRequest;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
@ -33,6 +32,11 @@ public class BookController extends BaseController{
private final BookService bookService;
private final RabbitTemplate rabbitTemplate;
@Value("${spring.rabbitmq.enable}")
private Integer enableMq;
/**
* 查询首页小说设置列表数据
@ -79,8 +83,8 @@ public class BookController extends BaseController{
* */
@PostMapping("searchByPage")
public ResultBean searchByPage(BookSP bookSP, @RequestParam(value = "curr", defaultValue = "1") int page, @RequestParam(value = "limit", defaultValue = "20") int pageSize){
List<BookVO> books = bookService.searchByPage(bookSP,page,pageSize);
return ResultBean.ok(new PageInfo<>(books));
PageInfo<BookVO> pageInfo = bookService.searchByPage(bookSP,page,pageSize);
return ResultBean.ok(pageInfo);
}
/**
@ -105,7 +109,11 @@ public class BookController extends BaseController{
* */
@PostMapping("addVisitCount")
public ResultBean addVisitCount(Long bookId){
bookService.addVisitCount(bookId);
if(enableMq == 1) {
rabbitTemplate.convertAndSend("ADD-BOOK-VISIT-EXCHANGE", null, bookId);
}else {
bookService.addVisitCount(bookId, 1);
}
return ResultBean.ok();
}

View File

@ -0,0 +1,59 @@
package com.java2nb.novel.core.config;
import org.springframework.amqp.core.*;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* @author 11797
*/
@Configuration
@ConditionalOnProperty(prefix = "spring.rabbitmq", name = "enable", havingValue = "1")
public class RabbitConfig {
/**
* 更新数据库队列
*/
@Bean
public Queue updateDbQueue() {
return new Queue("UPDATE-DB-QUEUE", true);
}
/**
* 更新数据库队列
*/
@Bean
public Queue updateEsQueue() {
return new Queue("UPDATE-ES-QUEUE", true);
}
/**
* 增加点击量交换机
*/
@Bean
public FanoutExchange addVisitExchange() {
return new FanoutExchange("ADD-BOOK-VISIT-EXCHANGE");
}
/**
* 更新搜索引擎队列绑定到增加点击量交换机中
*/
@Bean
public Binding updateEsBinding() {
return BindingBuilder.bind(updateEsQueue()).to(addVisitExchange());
}
/**
* 更新数据库绑定到增加点击量交换机中
*/
@Bean
public Binding updateDbBinding() {
return BindingBuilder.bind(updateDbQueue()).to(addVisitExchange());
}
}

View File

@ -0,0 +1,107 @@
package com.java2nb.novel.core.listener;
import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.utils.Constants;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.service.BookService;
import com.java2nb.novel.service.SearchService;
import com.java2nb.novel.vo.EsBookVO;
import com.rabbitmq.client.Channel;
import io.searchbox.client.JestClient;
import io.searchbox.core.Index;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.beans.BeanUtils;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Component;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import java.text.SimpleDateFormat;
/**
* @author 11797
*/
@Component
@Slf4j
@RequiredArgsConstructor
@ConditionalOnProperty(prefix = "spring.rabbitmq", name = "enable", havingValue = "1")
public class BookVisitAddListener {
private final BookService bookService;
private final CacheService cacheService;
private final SearchService searchService;
// private final RedissonClient redissonClient;
/**
* 更新数据库
* 流量削峰每本小说累积10个点击更新一次
*/
@SneakyThrows
@RabbitListener(queues = {"UPDATE-DB-QUEUE"})
public void updateDb(Long bookId, Channel channel, Message message) {
log.debug("收到更新数据库消息:" + bookId);
Thread.sleep(1000 * 2);
//TODO 操作共享资源visitCount集群环境下有线程安全问题引入Redisson框架实现分布式锁
//RLock lock = redissonClient.getLock("visitCount");
//lock.lock();
//目前visitCount不重要数据可丢失暂不实现分布式锁
Integer visitCount = (Integer) cacheService.getObject(CacheKey.BOOK_ADD_VISIT_COUNT+bookId);
if(visitCount == null){
visitCount = 0 ;
}
cacheService.setObject(CacheKey.BOOK_ADD_VISIT_COUNT+bookId,++visitCount);
if(visitCount >= Constants.ADD_MAX_VISIT_COUNT) {
bookService.addVisitCount(bookId,visitCount);
cacheService.del(CacheKey.BOOK_ADD_VISIT_COUNT+bookId);
}
//TODO 操作共享资源visitCount集群环境下有线程安全问题引入Redisson框架实现分布式锁
//lock.unlock();
}
/**
* 更新搜索引擎
* 流量削峰每本小说1个小时更新一次
*/
@RabbitListener(queues = {"UPDATE-ES-QUEUE"})
public void updateEs(Long bookId, Channel channel, Message message) {
log.debug("收到更新搜索引擎消息:" + bookId);
if (cacheService.get(CacheKey.ES_IS_UPDATE_VISIT + bookId) == null) {
cacheService.set(CacheKey.ES_IS_UPDATE_VISIT + bookId, "1", 60 * 60);
try {
Thread.sleep(1000 * 5);
Book book = bookService.queryBookDetail(bookId);
searchService.importToEs(book);
}catch (Exception e){
cacheService.del(CacheKey.ES_IS_UPDATE_VISIT + bookId);
log.error("更新搜索引擎失败"+bookId);
}
}
}
}

View File

@ -0,0 +1,87 @@
package com.java2nb.novel.core.schedule;
import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.utils.BeanUtil;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.service.BookService;
import com.java2nb.novel.service.SearchService;
import com.java2nb.novel.vo.EsBookVO;
import io.searchbox.client.JestClient;
import io.searchbox.core.DocumentResult;
import io.searchbox.core.Index;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
/**
* 小说导入搜索引擎
*
* @author Administrator
*/
@ConditionalOnProperty(prefix = "spring.elasticsearch", name = "enable", havingValue = "1")
@Service
@RequiredArgsConstructor
@Slf4j
public class BookToEsSchedule {
private final BookService bookService;
private final CacheService cacheService;
private final SearchService searchService;
/**
* 1分钟导入一次
*/
@Scheduled(fixedRate = 1000 * 60)
public void saveToEs() {
//TODO 引入Redisson框架实现分布式锁
//可以重复更新,只是效率可能略有降低,所以暂不实现分布式锁
if (cacheService.get(CacheKey.ES_TRANS_LOCK) == null) {
cacheService.set(CacheKey.ES_TRANS_LOCK, "1", 60 * 20);
try {
//查询需要更新的小说
Date lastDate = (Date) cacheService.getObject(CacheKey.ES_LAST_UPDATE_TIME);
if (lastDate == null) {
lastDate = new SimpleDateFormat("yyyy-MM-dd").parse("2020-01-01");
}
List<Book> books = bookService.queryBookByUpdateTimeByPage(lastDate, 100);
for (Book book : books) {
searchService.importToEs(book);
lastDate = book.getUpdateTime();
Thread.sleep(5000);
}
cacheService.setObject(CacheKey.ES_LAST_UPDATE_TIME, lastDate);
} catch (Exception e) {
log.error(e.getMessage(), e);
}
cacheService.del(CacheKey.ES_TRANS_LOCK);
}
}
}

View File

@ -3,6 +3,7 @@ package com.java2nb.novel.core.schedule;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.service.BookService;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
@ -13,9 +14,10 @@ import java.util.List;
/**
* 网络图片转存本地任务
*
* @author Administrator
*/
@ConditionalOnProperty(prefix = "pic.save",name = "type",havingValue = "2")
@ConditionalOnProperty(prefix = "pic.save", name = "type", havingValue = "2")
@Service
@RequiredArgsConstructor
@Slf4j
@ -33,20 +35,18 @@ public class Network2LocalPicSchedule {
* 10分钟转一次
*/
@Scheduled(fixedRate = 1000 * 60 * 10)
@SneakyThrows
public void trans() {
log.info("Network2LocalPicSchedule。。。。。。。。。。。。");
Integer offset = 0, limit = 100;
List<Book> networkPicBooks;
do {
networkPicBooks = bookService.queryNetworkPicBooks(limit, offset);
for (Book book : networkPicBooks) {
bookService.updateBookPicToLocal(book.getPicUrl(), book.getId());
}
offset += limit;
} while (networkPicBooks.size() > 0);
List<Book> networkPicBooks = bookService.queryNetworkPicBooks(100);
for (Book book : networkPicBooks) {
bookService.updateBookPicToLocal(book.getPicUrl(), book.getId());
//3秒钟转化一张图片10分钟转化200张
Thread.sleep(3000);
}
}

View File

@ -30,4 +30,9 @@ public class Constants {
* 首页设置的小说数量
* */
public static final int INDEX_BOOK_SETTING_NUM = 32;
/**
* 累积的最大点击量
* */
public static final Integer ADD_MAX_VISIT_COUNT = 10;
}

View File

@ -5,6 +5,7 @@ import com.java2nb.novel.search.BookSP;
import com.java2nb.novel.vo.BookVO;
import org.apache.ibatis.annotations.Param;
import java.util.Date;
import java.util.List;
/**
@ -15,13 +16,13 @@ public interface FrontBookMapper extends BookMapper {
List<BookVO> searchByPage(BookSP params);
void addVisitCount(@Param("bookId") Long bookId);
void addVisitCount(@Param("bookId") Long bookId, @Param("visitCount") Integer visitCount);
List<Book> listRecBookByCatId(@Param("catId") Integer catId);
void addCommentCount(@Param("bookId") Long bookId);
List<Book> queryNetworkPicBooks(@Param("limit") Integer limit,@Param("offset") Integer offset);
List<Book> queryNetworkPicBooks(@Param("limit") Integer limit);
/**
* 按评分随机查询小说集合

View File

@ -1,12 +1,14 @@
package com.java2nb.novel.service;
import com.github.pagehelper.PageInfo;
import com.java2nb.novel.search.BookSP;
import com.java2nb.novel.vo.BookCommentVO;
import com.java2nb.novel.vo.BookSettingVO;
import com.java2nb.novel.entity.*;
import com.java2nb.novel.vo.BookVO;
import java.util.Date;
import java.util.List;
import java.util.Map;
@ -44,9 +46,9 @@ public interface BookService {
* @param params 搜索参数
* @param page 页码
* @param pageSize 分页大小
* @return 小说集合
* @return 小说集合分页信息
* */
List<BookVO> searchByPage(BookSP params, int page, int pageSize);
PageInfo searchByPage(BookSP params, int page, int pageSize);
/**
* 查询小说分类列表
@ -110,8 +112,8 @@ public interface BookService {
/**
* 增加点击次数
* @param bookId 书籍ID
* */
void addVisitCount(Long bookId);
* @param visitCount*/
void addVisitCount(Long bookId, Integer visitCount);
/**
* 查询章节数
@ -179,10 +181,9 @@ public interface BookService {
/**
* 查询网络图片的小说
* @param limit 查询条数
* @param offset 开始行数
* @return 返回小说集合
* */
List<Book> queryNetworkPicBooks(Integer limit, Integer offset);
List<Book> queryNetworkPicBooks(Integer limit);
/**
@ -224,4 +225,13 @@ public interface BookService {
* @param authorId 作者ID
* */
void addBookContent(Long bookId, String indexName, String content, Long authorId);
/**
* 根据更新时间分页查询书籍列表
* @param startDate 开始时间,包括该时间
* @param limit 查询数量
* @return 书籍列表
* */
List<Book> queryBookByUpdateTimeByPage(Date startDate, int limit);
}

View File

@ -0,0 +1,27 @@
package com.java2nb.novel.service;
import com.github.pagehelper.PageInfo;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.search.BookSP;
/**
* @author 11797
*/
public interface SearchService {
/**
* 导入到es
* @param book 小说数据
*/
void importToEs(Book book);
/**
* 搜索
* @param params 搜索参数
* @param page 当前页码
* @param pageSize 每页大小
* @return 分页信息
*/
PageInfo searchBook(BookSP params, int page, int pageSize);
}

View File

@ -2,25 +2,25 @@ package com.java2nb.novel.service.impl;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.pagehelper.PageHelper;
import com.github.pagehelper.PageInfo;
import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.enums.ResponseStatus;
import com.java2nb.novel.core.exception.BusinessException;
import com.java2nb.novel.core.utils.BeanUtil;
import com.java2nb.novel.core.utils.Constants;
import com.java2nb.novel.core.utils.FileUtil;
import com.java2nb.novel.core.utils.IdWorker;
import com.java2nb.novel.core.utils.*;
import com.java2nb.novel.entity.*;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.mapper.*;
import com.java2nb.novel.search.BookSP;
import com.java2nb.novel.service.AuthorService;
import com.java2nb.novel.service.BookService;
import com.java2nb.novel.service.SearchService;
import com.java2nb.novel.vo.BookCommentVO;
import com.java2nb.novel.vo.BookSettingVO;
import com.java2nb.novel.vo.BookVO;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.mybatis.dynamic.sql.SortSpecification;
import org.mybatis.dynamic.sql.render.RenderingStrategies;
@ -50,14 +50,18 @@ import static org.mybatis.dynamic.sql.select.SelectDSL.select;
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class BookServiceImpl implements BookService {
/**
* 本地图片保存路径
* */
*/
@Value("${pic.save.path}")
private String picSavePath;
@Value("${spring.elasticsearch.enable}")
private Integer esEnable;
private final FrontBookSettingMapper bookSettingMapper;
private final FrontBookMapper bookMapper;
@ -76,6 +80,8 @@ public class BookServiceImpl implements BookService {
private final AuthorService authorService;
private final SearchService searchService;
@SneakyThrows
@Override
@ -83,24 +89,24 @@ public class BookServiceImpl implements BookService {
String result = cacheService.get(CacheKey.INDEX_BOOK_SETTINGS_KEY);
if (result == null || result.length() < Constants.OBJECT_JSON_CACHE_EXIST_LENGTH) {
List<BookSettingVO> list = bookSettingMapper.listVO();
if(list.size() == 0) {
if (list.size() == 0) {
//如果首页小说没有被设置,则初始化首页小说设置
list = initIndexBookSetting();
}
result = new ObjectMapper().writeValueAsString(list.stream().collect(Collectors.groupingBy(BookSettingVO::getType)));
cacheService.set(CacheKey.INDEX_BOOK_SETTINGS_KEY, result);
}
return new ObjectMapper().readValue(result,Map.class);
return new ObjectMapper().readValue(result, Map.class);
}
/**
* 初始化首页小说设置
* */
*/
private List<BookSettingVO> initIndexBookSetting() {
Date currentDate = new Date();
List<Book> books = bookMapper.selectIdsByScoreAndRandom(Constants.INDEX_BOOK_SETTING_NUM);
if(books.size() == Constants.INDEX_BOOK_SETTING_NUM) {
List<Book> books = bookMapper.selectIdsByScoreAndRandom(Constants.INDEX_BOOK_SETTING_NUM);
if (books.size() == Constants.INDEX_BOOK_SETTING_NUM) {
List<BookSetting> bookSettingList = new ArrayList<>(Constants.INDEX_BOOK_SETTING_NUM);
List<BookSettingVO> bookSettingVOList = new ArrayList<>(Constants.INDEX_BOOK_SETTING_NUM);
for (int i = 0; i < books.size(); i++) {
@ -114,7 +120,7 @@ public class BookServiceImpl implements BookService {
type = 2;
} else if (i < 26) {
type = 3;
}else{
} else {
type = 4;
}
BookSettingVO bookSettingVO = new BookSettingVO();
@ -126,8 +132,8 @@ public class BookServiceImpl implements BookService {
bookSetting.setUpdateTime(currentDate);
bookSettingList.add(bookSetting);
BeanUtils.copyProperties(book,bookSettingVO);
BeanUtils.copyProperties(bookSetting,bookSettingVO);
BeanUtils.copyProperties(book, bookSettingVO);
BeanUtils.copyProperties(bookSetting, bookSettingVO);
bookSettingVOList.add(bookSettingVO);
}
@ -164,25 +170,43 @@ public class BookServiceImpl implements BookService {
List<BookVO> result = (List<BookVO>) cacheService.getObject(CacheKey.INDEX_UPDATE_BOOK_KEY);
if (result == null || result.size() == 0) {
List<Book> bookPOList = listRank((byte) 2, 23);
result = BeanUtil.copyList(bookPOList,BookVO.class);
result = BeanUtil.copyList(bookPOList, BookVO.class);
cacheService.setObject(CacheKey.INDEX_UPDATE_BOOK_KEY, result, 60 * 10);
}
return result;
}
@Override
public List<BookVO> searchByPage(BookSP params, int page, int pageSize) {
PageHelper.startPage(page, pageSize);
public PageInfo searchByPage(BookSP params, int page, int pageSize) {
if (params.getUpdatePeriod() != null) {
long cur = System.currentTimeMillis();
long period = params.getUpdatePeriod() * 24 * 3600 * 1000;
long time = cur - period;
params.setUpdateTimeMin(new Date(time));
}
if (esEnable == 1) {
try {
return searchService.searchBook(params,page,pageSize);
}catch (Exception e){
log.error(e.getMessage(),e);
}
}
PageHelper.startPage(page, pageSize);
if (StringUtils.isNotBlank(params.getSort())) {
OrderByHelper.orderBy(params.getSort() + " desc");
}
return bookMapper.searchByPage(params);
return new PageInfo<>(bookMapper.searchByPage(params));
}
@Override
@ -197,7 +221,7 @@ public class BookServiceImpl implements BookService {
@Override
public Book queryBookDetail(Long bookId) {
SelectStatementProvider selectStatement = select(id, catName, catId, picUrl, bookName, authorId, authorName, bookDesc, bookStatus, visitCount, wordCount, lastIndexId, lastIndexName, lastIndexUpdateTime,score,status)
SelectStatementProvider selectStatement = select(book.allColumns())
.from(book)
.where(id, isEqualTo(bookId))
.build()
@ -206,15 +230,15 @@ public class BookServiceImpl implements BookService {
}
@Override
public List<BookIndex> queryIndexList(Long bookId,String orderBy, Integer limit) {
if(StringUtils.isNotBlank(orderBy)){
public List<BookIndex> queryIndexList(Long bookId, String orderBy, Integer limit) {
if (StringUtils.isNotBlank(orderBy)) {
OrderByHelper.orderBy(orderBy);
}
if(limit != null){
PageHelper.startPage(1,limit);
if (limit != null) {
PageHelper.startPage(1, limit);
}
SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.id, BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum, BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.updateTime,BookIndexDynamicSqlSupport.isVip)
SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.id, BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum, BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.updateTime, BookIndexDynamicSqlSupport.isVip)
.from(bookIndex)
.where(BookIndexDynamicSqlSupport.bookId, isEqualTo(bookId))
.build()
@ -225,7 +249,7 @@ public class BookServiceImpl implements BookService {
@Override
public BookIndex queryBookIndex(Long bookIndexId) {
SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.id, BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum, BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.wordCount, BookIndexDynamicSqlSupport.updateTime,BookIndexDynamicSqlSupport.isVip)
SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.id, BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum, BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.wordCount, BookIndexDynamicSqlSupport.updateTime, BookIndexDynamicSqlSupport.isVip)
.from(bookIndex)
.where(BookIndexDynamicSqlSupport.id, isEqualTo(bookIndexId))
.build()
@ -271,7 +295,7 @@ public class BookServiceImpl implements BookService {
@Override
public BookContent queryBookContent(Long bookIndexId) {
SelectStatementProvider selectStatement = select(BookContentDynamicSqlSupport.id,BookContentDynamicSqlSupport.content)
SelectStatementProvider selectStatement = select(BookContentDynamicSqlSupport.id, BookContentDynamicSqlSupport.content)
.from(bookContent)
.where(BookContentDynamicSqlSupport.indexId, isEqualTo(bookIndexId))
.limit(1)
@ -305,7 +329,7 @@ public class BookServiceImpl implements BookService {
}
SelectStatementProvider selectStatement = select(id, catId, catName, bookName, lastIndexId, lastIndexName, authorId, authorName, picUrl, bookDesc, wordCount, lastIndexUpdateTime)
.from(book)
.where(wordCount,isGreaterThan(0))
.where(wordCount, isGreaterThan(0))
.orderBy(sortSpecification)
.limit(limit)
.build()
@ -315,9 +339,8 @@ public class BookServiceImpl implements BookService {
}
@Override
public void addVisitCount(Long bookId) {
bookMapper.addVisitCount(bookId);
public void addVisitCount(Long bookId, Integer visitCount) {
bookMapper.addVisitCount(bookId,visitCount);
}
@Override
@ -349,10 +372,10 @@ public class BookServiceImpl implements BookService {
}
@Override
public List<BookCommentVO> listCommentByPage(Long userId,Long bookId, int page, int pageSize) {
public List<BookCommentVO> listCommentByPage(Long userId, Long bookId, int page, int pageSize) {
PageHelper.startPage(page, pageSize);
OrderByHelper.orderBy("t1.create_time desc");
return bookCommentMapper.listCommentByPage(userId,bookId);
return bookCommentMapper.listCommentByPage(userId, bookId);
}
@Transactional(rollbackFor = Exception.class)
@ -361,11 +384,11 @@ public class BookServiceImpl implements BookService {
//判断该用户是否已评论过该书籍
SelectStatementProvider selectStatement = select(count(BookCommentDynamicSqlSupport.id))
.from(bookComment)
.where(BookCommentDynamicSqlSupport.createUserId,isEqualTo(userId))
.and(BookCommentDynamicSqlSupport.bookId,isEqualTo(comment.getBookId()))
.where(BookCommentDynamicSqlSupport.createUserId, isEqualTo(userId))
.and(BookCommentDynamicSqlSupport.bookId, isEqualTo(comment.getBookId()))
.build()
.render(RenderingStrategies.MYBATIS3);
if(bookCommentMapper.count(selectStatement)>0){
if (bookCommentMapper.count(selectStatement) > 0) {
throw new BusinessException(ResponseStatus.HAS_COMMENTS);
}
//增加评论
@ -382,14 +405,14 @@ public class BookServiceImpl implements BookService {
Long authorId;
SelectStatementProvider selectStatement = select(BookAuthorDynamicSqlSupport.id)
.from(BookAuthorDynamicSqlSupport.bookAuthor)
.where(BookAuthorDynamicSqlSupport.penName,isEqualTo(authorName))
.where(BookAuthorDynamicSqlSupport.penName, isEqualTo(authorName))
.build()
.render(RenderingStrategies.MYBATIS3);
List<BookAuthor> bookAuthors = bookAuthorMapper.selectMany(selectStatement);
if(bookAuthors.size()>0){
if (bookAuthors.size() > 0) {
//作者存在
authorId = bookAuthors.get(0).getId();
}else{
} else {
//作者不存在,先创建作者
Date currentDate = new Date();
authorId = new IdWorker().nextId();
@ -409,18 +432,17 @@ public class BookServiceImpl implements BookService {
}
@Override
public Long queryIdByNameAndAuthor(String bookName, String author) {
//查询小说ID
SelectStatementProvider selectStatement = select(id)
.from(book)
.where(BookDynamicSqlSupport.bookName,isEqualTo(bookName))
.and(BookDynamicSqlSupport.authorName,isEqualTo(authorName))
.where(BookDynamicSqlSupport.bookName, isEqualTo(bookName))
.and(BookDynamicSqlSupport.authorName, isEqualTo(authorName))
.build()
.render(RenderingStrategies.MYBATIS3);
List<Book> books = bookMapper.selectMany(selectStatement);
if(books.size()>0){
if (books.size() > 0) {
return books.get(0).getId();
}
return null;
@ -430,7 +452,7 @@ public class BookServiceImpl implements BookService {
public List<Integer> queryIndexNumByBookId(Long bookId) {
SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.indexNum)
.from(BookIndexDynamicSqlSupport.bookIndex)
.where(BookIndexDynamicSqlSupport.bookId,isEqualTo(bookId))
.where(BookIndexDynamicSqlSupport.bookId, isEqualTo(bookId))
.build()
.render(RenderingStrategies.MYBATIS3);
@ -438,21 +460,21 @@ public class BookServiceImpl implements BookService {
}
@Override
public List<Book> queryNetworkPicBooks(Integer limit, Integer offset) {
return bookMapper.queryNetworkPicBooks(limit,offset);
public List<Book> queryNetworkPicBooks(Integer limit) {
return bookMapper.queryNetworkPicBooks(limit);
}
@Override
public void updateBookPicToLocal(String picUrl, Long bookId) {
picUrl = FileUtil.network2Local(picUrl,picSavePath, Constants.LOCAL_PIC_PREFIX);
picUrl = FileUtil.network2Local(picUrl, picSavePath, Constants.LOCAL_PIC_PREFIX);
bookMapper.update(update(book)
.set(BookDynamicSqlSupport.picUrl)
.equalTo(picUrl)
.set(updateTime)
.equalTo(new Date())
.where(id,isEqualTo(bookId))
.where(id, isEqualTo(bookId))
.build()
.render(RenderingStrategies.MYBATIS3));
@ -461,7 +483,7 @@ public class BookServiceImpl implements BookService {
@Override
public List<Book> listBookPageByUserId(Long userId, int page, int pageSize) {
PageHelper.startPage(page,pageSize);
PageHelper.startPage(page, pageSize);
SelectStatementProvider selectStatement = select(id, bookName, visitCount, lastIndexName, status)
.from(book)
@ -476,10 +498,11 @@ public class BookServiceImpl implements BookService {
@Override
public void addBook(Book book, Long authorId, String penName) {
//判断小说名是否存在
if(queryIdByNameAndAuthor(book.getBookName(),penName)!=null){
if (queryIdByNameAndAuthor(book.getBookName(), penName) != null) {
//该作者发布过此书名的小说
throw new BusinessException(ResponseStatus.BOOKNAME_EXISTS);
};
}
;
book.setAuthorName(penName);
book.setAuthorId(authorId);
book.setVisitCount(0L);
@ -497,18 +520,18 @@ public class BookServiceImpl implements BookService {
bookMapper.update(update(book)
.set(BookDynamicSqlSupport.status)
.equalTo(status)
.where(id,isEqualTo(bookId))
.and(BookDynamicSqlSupport.authorId,isEqualTo(authorId))
.where(id, isEqualTo(bookId))
.and(BookDynamicSqlSupport.authorId, isEqualTo(authorId))
.build()
.render(RenderingStrategies.MYBATIS3));
}
@Transactional
@Transactional(rollbackFor = Exception.class)
@Override
public void addBookContent(Long bookId, String indexName, String content, Long authorId) {
Book book = queryBookDetail(bookId);
if(!authorId.equals(book.getAuthorId())){
if (!authorId.equals(book.getAuthorId())) {
//并不是更新自己的小说
return;
}
@ -525,15 +548,15 @@ public class BookServiceImpl implements BookService {
.set(BookDynamicSqlSupport.lastIndexUpdateTime)
.equalTo(currentDate)
.set(BookDynamicSqlSupport.wordCount)
.equalTo(book.getWordCount()+wordCount)
.where(id,isEqualTo(bookId))
.and(BookDynamicSqlSupport.authorId,isEqualTo(authorId))
.equalTo(book.getWordCount() + wordCount)
.where(id, isEqualTo(bookId))
.and(BookDynamicSqlSupport.authorId, isEqualTo(authorId))
.build()
.render(RenderingStrategies.MYBATIS3));
//更新小说目录表
int indexNum = 0;
if(book.getLastIndexId() != null){
indexNum = queryBookIndex(book.getLastIndexId()).getIndexNum()+1;
if (book.getLastIndexId() != null) {
indexNum = queryBookIndex(book.getLastIndexId()).getIndexNum() + 1;
}
BookIndex lastBookIndex = new BookIndex();
lastBookIndex.setId(lastIndexId);
@ -553,7 +576,19 @@ public class BookServiceImpl implements BookService {
bookContentMapper.insertSelective(bookContent);
}
@Override
public List<Book> queryBookByUpdateTimeByPage(Date startDate, int limit) {
return bookMapper.selectMany(select(book.allColumns())
.from(book)
.where(updateTime, isGreaterThan(startDate))
.orderBy(updateTime)
.limit(limit)
.build()
.render(RenderingStrategies.MYBATIS3));
}

View File

@ -0,0 +1,199 @@
package com.java2nb.novel.service.impl;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.pagehelper.PageInfo;
import com.java2nb.novel.core.enums.ResponseStatus;
import com.java2nb.novel.core.exception.BusinessException;
import com.java2nb.novel.core.utils.StringUtil;
import com.java2nb.novel.entity.Book;
import com.java2nb.novel.search.BookSP;
import com.java2nb.novel.service.SearchService;
import com.java2nb.novel.vo.EsBookVO;
import io.searchbox.client.JestClient;
import io.searchbox.core.*;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.BeanUtils;
import org.springframework.stereotype.Service;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
/**
* @author 11797
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class SearchServiceImpl implements SearchService {
private final String INDEX = "novel";
private final String TYPE = "book";
private final JestClient jestClient;
@Override
@SneakyThrows
public void importToEs(Book book) {
//导入到ES
EsBookVO esBookVO = new EsBookVO();
BeanUtils.copyProperties(book, esBookVO, "lastIndexUpdateTime");
esBookVO.setLastIndexUpdateTime(new SimpleDateFormat("yyyy/MM/dd HH:mm").format(book.getLastIndexUpdateTime()));
Index action = new Index.Builder(esBookVO).index(INDEX).type(TYPE).id(book.getId().toString()).build();
jestClient.execute(action);
}
@SneakyThrows
@Override
public PageInfo searchBook(BookSP params, int page, int pageSize) {
List<EsBookVO> bookList = new ArrayList<>(0);
//使用搜索引擎搜索
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// 构造查询哪个字段
if (StringUtils.isNoneBlank(params.getKeyword())) {
boolQueryBuilder = boolQueryBuilder.must(QueryBuilders.queryStringQuery(params.getKeyword()));
}
// 作品方向
if (params.getWorkDirection() != null) {
boolQueryBuilder.filter(QueryBuilders.termQuery("workDirection", params.getWorkDirection()));
}
// 分类
if (params.getCatId() != null) {
boolQueryBuilder.filter(QueryBuilders.termQuery("catId", params.getCatId()));
}
if (params.getBookStatus() != null) {
boolQueryBuilder.filter(QueryBuilders.termQuery("bookStatus", params.getBookStatus()));
}
if (params.getWordCountMin() == null) {
params.setWordCountMin(0);
}
if (params.getWordCountMax() == null) {
params.setWordCountMax(Integer.MAX_VALUE);
}
boolQueryBuilder.filter(QueryBuilders.rangeQuery("wordCount").gte(params.getWordCountMin()).lte(params.getWordCountMax()));
if (params.getUpdateTimeMin() != null) {
boolQueryBuilder.filter(QueryBuilders.rangeQuery("lastIndexUpdateTime").gte(params.getUpdateTimeMin()));
}
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(boolQueryBuilder);
Count count = new Count.Builder().addIndex(INDEX).addType(TYPE)
.query(searchSourceBuilder.toString()).build();
CountResult results = jestClient.execute(count);
Double total = results.getCount();
// 高亮字段
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("authorName");
highlightBuilder.field("bookName");
highlightBuilder.field("bookDesc");
highlightBuilder.field("lastIndexName");
highlightBuilder.field("catName");
highlightBuilder.preTags("<span style='color:red'>").postTags("</span>");
highlightBuilder.fragmentSize(20000);
searchSourceBuilder.highlighter(highlightBuilder);
//设置排序
if (params.getSort() != null) {
searchSourceBuilder.sort(StringUtil.camelName(params.getSort()), SortOrder.DESC);
}
// 设置分页
searchSourceBuilder.from((page - 1) * pageSize);
searchSourceBuilder.size(pageSize);
// 构建Search对象
Search search = new Search.Builder(searchSourceBuilder.toString()).addIndex(INDEX).addType(TYPE).build();
log.debug(search.toString());
SearchResult result;
result = jestClient.execute(search);
if (result.isSucceeded()) {
log.debug(result.getJsonString());
Map resultMap = new ObjectMapper().readValue(result.getJsonString(), Map.class);
if (resultMap.get("hits") != null) {
Map hitsMap = (Map) resultMap.get("hits");
if (hitsMap.size() > 0 && hitsMap.get("hits") != null) {
List hitsList = (List) hitsMap.get("hits");
if (hitsList.size() > 0 && result.getSourceAsString() != null) {
JavaType jt = new ObjectMapper().getTypeFactory().constructParametricType(ArrayList.class, EsBookVO.class);
bookList = new ObjectMapper().readValue("[" + result.getSourceAsString() + "]", jt);
if (bookList != null) {
for (int i = 0; i < bookList.size(); i++) {
hitsMap = (Map) hitsList.get(i);
Map highlightMap = (Map) hitsMap.get("highlight");
if (highlightMap != null && highlightMap.size() > 0) {
List<String> authorNameList = (List<String>) highlightMap.get("authorName");
if (authorNameList != null && authorNameList.size() > 0) {
bookList.get(i).setAuthorName(authorNameList.get(0));
}
List<String> bookNameList = (List<String>) highlightMap.get("bookName");
if (bookNameList != null && bookNameList.size() > 0) {
bookList.get(i).setBookName(bookNameList.get(0));
}
List<String> bookDescList = (List<String>) highlightMap.get("bookDesc");
if (bookDescList != null && bookDescList.size() > 0) {
bookList.get(i).setBookDesc(bookDescList.get(0));
}
List<String> lastIndexNameList = (List<String>) highlightMap.get("lastIndexName");
if (lastIndexNameList != null && lastIndexNameList.size() > 0) {
bookList.get(i).setLastIndexName(lastIndexNameList.get(0));
}
List<String> catNameList = (List<String>) highlightMap.get("catName");
if (catNameList != null && catNameList.size() > 0) {
bookList.get(i).setCatName(catNameList.get(0));
}
}
}
}
}
}
}
PageInfo<EsBookVO> pageInfo = new PageInfo<>(bookList);
pageInfo.setTotal(total.longValue());
pageInfo.setPageNum(page);
pageInfo.setPageSize(pageSize);
return pageInfo;
}
throw new BusinessException(ResponseStatus.ES_SEARCH_FAIL);
}
}

View File

@ -0,0 +1,85 @@
package com.java2nb.novel.vo;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.java2nb.novel.entity.Book;
import lombok.Data;
import javax.annotation.Generated;
import java.io.Serializable;
import java.util.Date;
/**
* @author Administrator
*/
@Data
public class EsBookVO {
private Long id;
private Byte workDirection;
private Integer catId;
private String catName;
private String picUrl;
private String bookName;
private Long authorId;
private String authorName;
private String bookDesc;
private Float score;
private Byte bookStatus;
private Long visitCount;
private Integer wordCount;
private Integer commentCount;
private Long lastIndexId;
private String lastIndexName;
private String lastIndexUpdateTime;
private Byte isVip;
private Byte status;
private Integer crawlSourceId;
private String crawlBookId;
private Byte crawlIsStop;
}

View File

@ -6,6 +6,29 @@ spring:
active: dev
include: alipay
rabbitmq:
enable: 0
host: 127.0.0.1
username: guest
password: guest
virtual-host: /novel-plus
template:
# 缺省的交换机名称此处配置后发送消息如果不指定交换机就会使用这个
exchange: novel.exchange
publisher-confirms: false
elasticsearch:
#是否开启搜索引擎1开启0不开启
enable: 0
jest:
uris: http://127.0.0.1:9200
redisson:
singleServerConfig:
address: 127.0.0.1:6379
jwt:
secret: novel!#20191230
expiration: 604800

View File

@ -34,8 +34,9 @@
</select>
<update id="addVisitCount" parameterType="long">
update book set visit_count = visit_count + 1 where id = #{bookId}
<update id="addVisitCount" >
update book set visit_count = visit_count + ${visitCount}
where id = #{bookId}
</update>
<select id="listRecBookByCatId" parameterType="int" resultType="com.java2nb.novel.entity.Book">
@ -54,8 +55,8 @@
<select id="queryNetworkPicBooks" resultType="com.java2nb.novel.entity.Book">
select
id,pic_url from book
where pic_url like 'http://%' or pic_url like 'https://%'
limit #{offset},#{limit}
where pic_url like 'http%'
limit #{limit}
</select>
<select id="selectIdsByScoreAndRandom" parameterType="int" resultType="com.java2nb.novel.entity.Book">

View File

@ -130,33 +130,43 @@
<script src="/javascript/header.js" type="text/javascript"></script>
<script src="/javascript/user.js" type="text/javascript"></script>
<script language="javascript" type="text/javascript">
var lock = false;
function addBook() {
if(lock){
return;
}
lock = true;
var bookName = $("#bookName").val();
if(!bookName){
$("#LabErr").html("小说名不能为空");
lock = false;
return;
}
if(bookName.length > 20){
$("#LabErr").html("小说名太长");
lock = false;
return;
}
var picUrl = $("#picUrl").val();
if(!picUrl){
$("#LabErr").html("封面图片不能为空");
lock = false;
return;
}
var bookDesc = $("#bookDesc").val();
if(!bookDesc){
$("#LabErr").html("简介不能为空");
lock = false;
return;
}
$.ajax({
type: "POST",
url: "/author/addBook",
@ -171,11 +181,13 @@
} else {
lock = false;
$("#LabErr").html(data.msg);
}
},
error: function () {
lock = false;
layer.alert('网络异常');
}
})

View File

@ -58,10 +58,8 @@
<b>章节名:</b>
<li><input type="text" id="bookIndex" name="bookIndex" class="s_input" ></li>
<b>章节内容:</b>
<input type="hidden" id="bookContent" name="bookContent" >
<div style="width:600px" id="contentEditor">
</div>
<textarea name="bookContent" rows="30" cols="80" id="bookContent"
class="textarea"></textarea>
<li style="margin-top: 10px"><input type="button" onclick="addBookContent()" name="btnRegister" value="提交"
id="btnRegister" class="btn_red">
@ -114,43 +112,18 @@
<script src="/javascript/header.js" type="text/javascript"></script>
<script src="/javascript/user.js" type="text/javascript"></script>
<script src="/javascript/common.js" type="text/javascript"></script>
<script type="text/javascript" src="/wangEditor/release/wangEditor.js"></script>
<script language="javascript" type="text/javascript">
var E = window.wangEditor;
var editor2 = new E('#contentEditor');
// 自定义菜单配置
editor2.customConfig.menus = [
'head', // 标题
'bold', // 粗体
'fontSize', // 字号
'fontName', // 字体
'italic', // 斜体
'underline', // 下划线
'strikeThrough', // 删除线
'foreColor', // 文字颜色
//'backColor', // 背景颜色
//'link', // 插入链接
'list', // 列表
'justify', // 对齐方式
'quote', // 引用
'emoticon', // 表情
'image', // 插入图片
//'table', // 表格
//'video', // 插入视频
//'code', // 插入代码
'undo', // 撤销
'redo' // 重复
];
editor2.customConfig.onchange = function (html) {
// html 即变化之后的内容
$("#bookContent").val(html);
}
editor2.create();
var lock = false;
function addBookContent() {
if(lock){
return;
}
lock = true;
var bookId = getSearchString("bookId");
@ -158,12 +131,14 @@
var indexName = $("#bookIndex").val();
if(!indexName){
$("#LabErr").html("章节名不能为空");
lock = false;
return;
}
var content = $("#bookContent").val();
if(!content && content.length<=13){
if(!content){
$("#LabErr").html("章节内容不能为空");
lock = false;
return;
}
@ -182,11 +157,13 @@
} else {
lock = false;
$("#LabErr").html(data.msg);
}
},
error: function () {
lock = false;
layer.alert('网络异常');
}
})

View File

@ -179,9 +179,13 @@
for (var i = 0; i < bookList.length; i++) {
var book = bookList[i];
var end = book.bookDesc.indexOf("<");
/*var end = book.bookDesc.indexOf("<");
if(end != -1) {
book.bookDesc = book.bookDesc.substring(0,end);
}*/
if(book.bookDesc){
book.bookDesc = book.bookDesc.replace(/<[^>]+>/g,"").replace(/\s+/g,"").replace(/&nbsp;/g,"");
}
bookListHtml += ("<div class=\"layui-row\" style=\"margin-bottom:10px;padding:10px;background: #f2f2f2\">\n" +
@ -202,7 +206,7 @@
" </a>\n" +
" <div style=\"margin-top: 5px;color: #4c6978;\">类别:"+book.catName+"</div>\n" +
" <div style=\"margin-top: 5px;color: #4c6978;\">状态:"+(book.bookStatus==0?'连载':'完结')+"</div>\n" +
" <div style=\"margin-top: 5px;color: #4c6978;\">更新:<i>"+book.lastIndexUpdateTime+"</i>\n" +
" <div style=\"margin-top: 5px;color: #4c6978;\">更新:<i>"+book.lastIndexUpdateTime.substr(0,11)+"</i>\n" +
" </div>\n" +
" <div style=\"margin-top: 5px;color: #4c6978;\">简介:"+(book.bookDesc?(book.bookDesc.length>20?(book.bookDesc.substr(0,20)+"..."):book.bookDesc):book.bookDesc)+"</div>\n" +
"\n" +

View File

@ -276,6 +276,10 @@
for (var i = 0; i < 6; i++) {
var hotRecBook = hotRecBooks[i];
if(hotRecBook.bookDesc){
hotRecBook.bookDesc = hotRecBook.bookDesc.replace(/<[^>]+>/g,"").replace(/\s+/g,"");
}
hotRecBooksHtml += ("<div style=\"margin-bottom: 5px\" class=\"layui-col-xs12 layui-col-sm6 layui-col-md4 layui-col-lg4\">\n" +
" <a href=\"/book/"+hotRecBook.bookId+".html\">\n" +
" <div class=\"layui-col-xs5 layui-col-sm4 layui-col-md4 layui-col-lg4\" >\n" +
@ -323,9 +327,8 @@
for (var i = 0; i < 10; i++) {
var updateRankBook = updateRankBooks[i];
var end = updateRankBook.bookDesc.indexOf("<");
if(end != -1) {
updateRankBook.bookDesc = updateRankBook.bookDesc.substring(0,end);
if(updateRankBook.bookDesc){
updateRankBook.bookDesc = updateRankBook.bookDesc.replace(/<[^>]+>/g,"").replace(/\s+/g,"");
}
updateRankBookHtml += ("<div style=\"padding-bottom: 30px\"\n" +

View File

@ -5,7 +5,7 @@
<groupId>com.java2nb</groupId>
<artifactId>novel</artifactId>
<version>2.0.2</version>
<version>2.1.2</version>
<modules>
<module>novel-common</module>
<module>novel-front</module>
@ -36,6 +36,10 @@
<orderbyhelper.version>1.0.2</orderbyhelper.version>
<commons-lang3.version>3.4</commons-lang3.version>
<jjwt.version>0.9.0</jjwt.version>
<elasticsearch.version>6.2.2</elasticsearch.version>
<jest.version>6.3.1</jest.version>
<redis.version>1.4.1.RELEASE</redis.version>
<redisson.version>3.12.5</redisson.version>
</properties>
<dependencyManagement>