Update README.md

顶部推荐栏重复渲染同一本小说

README.md

@@ -1,50 +1,34 @@
-[![index]( https://z3.ax1x.com/2021/11/11/IwNJAg.jpg )]( https://cloud.tencent.com/act/cps/redirect?redirect=1052&cps_key=736e609d66e0ac4e57813316cec6fd0b&from=console )
+[![index]( https://youdoc.github.io/img/kuaidl4.png )]( https://www.kuaidaili.com/?ref=mdpz65syhqup )
 
-# 小说精品屋-plus
-
-[![Github stars](https://img.shields.io/github/stars/201206030/novel-plus?logo=github)](https://github.com/201206030/novel-plus)
-[![Github forks](https://img.shields.io/github/forks/201206030/novel-plus?logo=github)](https://github.com/201206030/novel-plus)
-[![Gitee star](https://gitee.com/novel_dev_team/novel-plus/badge/star.svg?theme=gitee)](https://gitee.com/novel_dev_team/novel-plus)
-[![Gitee fork](https://gitee.com/novel_dev_team/novel-plus/badge/fork.svg?theme=gitee)](https://gitee.com/novel_dev_team/novel-plus)
+<p align="center">
+    <a href='https://github.com/201206030/novel-plus'><img alt="Github stars" src="https://img.shields.io/github/stars/201206030/novel-plus?logo=github"></a>
+    <a href='https://github.com/201206030/novel-plus'><img alt="Github forks" src="https://img.shields.io/github/forks/201206030/novel-plus?logo=github"></a>
+    <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee stars" src="https://gitee.com/novel_dev_team/novel-plus/badge/star.svg?theme=gitee"></a>
+    <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee forks" src="https://gitee.com/novel_dev_team/novel-plus/badge/fork.svg?theme=gitee"></a>
+    <a href="https://github.com/201206030/novel-plus"><img src="https://visitor-badge.glitch.me/badge?page_id=201206030.novel-plus" alt="visitors"></a>
+</p>
 
 #### 官网
 
 https://201206030.github.io
 
-#### 新项目：小说精品屋-微服务版
+#### 学习版
 
-基于小说精品屋-plus构建的Spring Cloud 微服务小说门户平台。
+[基于 Spring Boot 3 + Vue 3 开发的前后端分离学习型小说项目](https://github.com/201206030/novel)
 
-GitHub仓库地址： https://github.com/201206030/novel-cloud
+#### 微服务版
 
-Gitee仓库地址： https://gitee.com/novel_dev_team/novel-cloud
+[Github](https://github.com/201206030/novel-cloud) | [码云](https://gitee.com/novel_dev_team/novel-cloud)
 
 #### 演示地址
 
 [点击前往](http://47.106.243.172:8888/)
 
-
 #### 项目介绍
 
-小说精品屋-plus是在[小说精品屋](https://github.com/201206030/fiction_house)的基础上，去除了漫画和弹幕模块，专注于小说，是一个多端（PC、WAP）阅读、功能完善的原创文学CMS系统，由前台门户系统、作家后台管理系统、平台后台管理系统、爬虫管理系统等多个子系统构成，支持会员充值、订阅模式、新闻发布和实时统计报表等功能。
-
-小说精品屋-plus重新进行了数据库设计、代码重构和功能增强，提升了程序整体的可读性和性能，增加了很多商用特性。主要升级如下：
-
-- [x] 数据库重新设计，结构调整。
-- [x] 服务端代码重构，MyBatis3升级为MyBatis3DynamicSql。
-- [x] 移动站与PC站站点分离，浏览器自动识别跳转。
-- [x] PC站UI更新。
-- [x] 支持前端模版自定义，内置多套模版。
-- [x] 可拓展的多种方式存储小说内容，内置数据库（分表）存储和TXT文本存储。
-- [x] 新闻模块。
-- [x] 排行榜。
-- [x] 小说评论模块。
-- [x] 阅读主题模块。
-- [x] 作家专区。
-- [x] 充值。
-- [x] 订阅。
-- [x] 后台管理系统。
-- [x] 爬虫管理系统。
+novel-plus 是一个多端（PC、WAP）阅读，功能完善的原创文学 CMS
+系统。由前台门户系统、作家后台管理系统、平台后台管理系统和爬虫管理系统等多个子系统构成，包括小说推荐、作品检索、小说排行、小说阅读、小说评论、会员中心、作家专区等功能，支持自定义多模版、可拓展的多种小说内容存储方式（内置数据库分表存储和
+TXT 文本存储）、阅读主题切换、多爬虫源自动采集和更新数据、会员充值、订阅模式、新闻发布和实时统计报表。
 
 #### 项目结构
 
@@ -59,42 +43,32 @@ novel-plus -- 父工程
 
 #### 技术选型
 
-| 技术                 | 说明                                                         
-| -------------------- | ---------------------------
-| SpringBoot           | Spring应用快速开发脚手架     
-| MyBatis              | 持久层ORM框架 
-| MyBatis Dynamic SQL  | Mybatis动态sql
-| PageHelper           | MyBatis分页插件
-| MyBatisGenerator     | 持久层代码生成插件
-| Sharding-Jdbc        | 代码层分库分表中间件
-| JJWT                 | JWT登录支持  
-| SpringSecurity       | 安全框架                           
-| Shiro                | 安全框架  
-| Ehcache              | Java进程内缓存框架(默认缓存)  
-| Redis                | 分布式缓存(缓存替换方案，默认关闭，一行配置开启)                               
-| ElasticSearch        | 搜索引擎(搜索增强方案，默认关闭，一行配置开启)                      
-| RabbitMq             | 消息队列(流量削峰，默认关闭，一行配置开启)  
-| OSS                  | 阿里云对象存储服务(图片存储方式之一，一行配置即可切换) 
-| FastDfs              |开源轻量级分布式文件系统(图片存储方式之一，一行配置即可切换)                      
-| Redisson             | 实现分布式锁                                       
-| Lombok               | 简化对象封装工具                                                                               
-| Docker               | 应用容器引擎   
-| Mysql                | 数据库服务   
-| Thymeleaf            | 模板引擎     
-| Layui                | 前端UI                    
-                 
-
-#### 开发工具
-感谢Jetbrains公司提供的免费License。
-
-[![index]( https://s3.ax1x.com/2021/01/14/sdHsJg.png )]( https://www.jetbrains.com/?from=小说精品屋)
-
-
-#### 接口文档
-
-[点击查看接口文档示例](doc/api/api.md)
+| 技术                  | 说明
+|---------------------| ---------------------------
+| Spring Boot         | Spring 应用快速开发脚手架
+| MyBatis             | 持久层 ORM 框架
+| MyBatis Dynamic SQL | Mybatis 动态 sql
+| PageHelper          | MyBatis 分页插件
+| MyBatisGenerator    | 持久层代码生成插件
+| Sharding-Jdbc       | 代码层分库分表中间件
+| JJWT                | JWT 登录支持
+| SpringSecurity      | 安全框架
+| Shiro               | 安全框架
+| Ehcache             | Java 进程内缓存框架(默认缓存)
+| Redis               | 分布式缓存(缓存替换方案，默认关闭，一行配置开启)
+| Elasticsearch       | 搜索引擎(搜索增强方案，默认关闭，一行配置开启)
+| RabbitMQ            | 消息队列(流量削峰，默认关闭，一行配置开启)
+| Aliyun OSS          | 阿里云对象存储服务(图片存储方式之一，一行配置即可切换)
+| FastDFS             | 开源轻量级分布式文件系统(图片存储方式之一，一行配置即可切换)
+| Redisson            | 实现分布式锁
+| Lombok              | 简化对象封装工具
+| Docker              | 应用容器引擎
+| MySQL               | 数据库服务
+| Thymeleaf           | 模板引擎
+| Layui               | 前端 UI 框架
 
 #### 橙色主题模版截图
+
 ##### PC站截图
 
 1. 首页
@@ -184,6 +158,7 @@ novel-plus -- 父工程
 ![img](https://oscimg.oschina.net/oscnet/up-faf5dda7320674c29a1772bc0c81d74762e.png)
 
 #### 深色主题模版截图
+
 ##### PC站截图
 
 1. 首页
@@ -191,6 +166,7 @@ novel-plus -- 父工程
    ![index](https://static.oschina.net/uploads/img/202006/24151811_wIus.png)
 
 ##### 手机站截图
+
 1. 首页
 
    ![index](https://static.oschina.net/uploads/img/202006/24151812_OOob.jpg)
@@ -209,91 +185,81 @@ novel-plus -- 父工程
 
 #### 蓝色主题模版截图（更新中）
 
-   ![QQ图片20191018161901](https://s3.ax1x.com/2020/12/27/r5Fe0A.png)
+![QQ图片20191018161901](https://s3.ax1x.com/2020/12/27/r5Fe0A.png)
 
-#### 安装步骤（源码，小白请看其他安装教程）
+#### 源码安装教程
 
-##### 数据库安装：
+- JDK 安装
 
-1. 安装MySQL软件。
-2. 修改MySQL`max_allowed_packet `配置（建议100M）。
-3. 新建数据库，设置编码为utf8mb4。
-4. 执行doc/sql/novel_plus.sql脚本文件。
+  建议[安装 JDK 8](https://docs.oracle.com/javase/8/docs/technotes/guides/install/linux_jdk.html)
 
-##### 爬虫管理系统安装：
+- MySQL 安装：
 
-1. 修改novel-common模块下application-common-dev.yml（dev环境，默认环境）或application-common-prod.yml（prod环境，需要在application.yml配置文件中切换）配置文件中的数据库配置。
-2. 修改novel-crawl模块下application.yml文件中的管理员账号密码。
-3. 启动程序，打开浏览器，默认8081端口访问。
-4. 选择已有或新增爬虫源（支持自定义爬虫规则），点击`开启`按钮，开始爬取小说数据。
+    1. [安装 MySQL 服务](https://dev.mysql.com/doc/refman/8.0/en/linux-installation.html)
+    2. 修改 MySQL`max_allowed_packet `配置（建议 100 M）
+    3. 新建数据库，设置编码为 utf8mb4
+    4. 执行 doc/sql/novel_plus.sql 脚本文件
 
-##### 前台小说门户安装（dev环境跳过3、4步骤）：
+- Maven 安装
 
-1. 修改novel-common模块下application-common-dev.yml（dev环境，默认环境）或application-common-prod.yml（prod环境，需要在application.yml配置文件中切换）配置文件中的数据库配置。
+  [安装 Apache Maven](https://maven.apache.org/install.html)
 
-2. 修改novel-front模块下application-website配置文件中的网站信息。
+- 源码打包
 
-  ```
-  #网站配置
-  website:
-    #网站名
-    name: 小说精品屋
-    #域名
-    domain: xiongxyang.gitee.io
-    #SEO关键词
-    keyword: ${website.name},小说,小说CMS,原创文学系统,开源小说系统,免费小说建站程序
-    #SEO描述
-    description: ${website.name}是一个多端（PC、WAP）阅读、功能完善的原创文学CMS系统，由前台门户系统、作家后台管理系统、平台后台管理系统、爬虫管理系统等多个子系统构成，支持会员充值、订阅模式、新闻发布和实时统计报表等功能，新书自动入库，老书自动更新。
-    #联系QQ
-    qq: 1179705413
-  ```
+  novel-plus 根目录下执行打包命令`mvn clean package -Dmaven.test.skip`
 
-3. prod环境下需要修改novel-front模块下application-prod.yml配置文件中的模版名为你需要使用的模版名（templates文件夹下的模版文件夹名，内置orange和dark两套模版，prod环境下才支持多模版）。
+- 爬虫安装
 
-  ```
-  #模版配置
-  templates:
-    #模版名
-    name: orange
-  ```
+    1. 上传 novel-crawl/target/build/novel-crawl.zip 压缩包到 Linux 服务器的 novel-crawl 目录
+    2. 使用`unzip novel-crawl.zip`命令解压 novel-crawl.zip
+    3. 修改 `config/application-common-prod.yml` 文件中的数据库配置
+    4. 修改 `config/application-common-prod.yml` 文件中的管理员账号密码
+    5. 根据需要[设置爬虫的代理IP](https://github.com/201206030/novel-plus/wiki/%E7%88%AC%E8%99%AB%E7%A8%8B%E5%BA%8F-HTTP-%E4%BB%A3%E7%90%86%E8%AE%BE%E7%BD%AE)
+    6. novel-crawl 目录下使用`bin/novel-crawl.sh start`命令启动爬虫程序
+    7. 打开浏览器，默认`8083`端口访问
+    8. 选择已有或新增爬虫源（支持自定义爬虫规则），点击`开启`按钮，开始采集小说数据
+    9. novel-crawl 目录下使用`bin/novel-crawl.sh stop`命令停止爬虫程序
+    10. novel-crawl 目录下使用`bin/novel-crawl.sh restart`命令重启爬虫程序
+    11. novel-crawl 目录下使用`bin/novel-crawl.sh status`命令查看爬虫程序的运行状态
 
-4. prod环境下的jar包形式部署时，需要复制templates文件夹到jar文件的相同目录下。   
+- 前台安装
 
-5. 启动程序，打开浏览器，默认8080端口访问。
+    1. 上传 novel-front/target/build/novel-front.zip 压缩包到 Linux 服务器的 novel-front 目录
+    2. 使用`unzip novel-front.zip`命令解压 novel-front.zip
+    3. 修改 `config/application-common-prod.yml` 文件中的数据库配置
+    4. 修改 `config/application-website.yml` 配置文件中的网站信息
+    5. novel-front 目录下使用`bin/novel-front.sh start`命令启动前台程序
+    6. 打开浏览器，默认`8085`端口访问
+    7. novel-front 目录下使用`bin/novel-front.sh stop`命令停止前台程序
+    8. novel-front 目录下使用`bin/novel-front.sh restart`命令重启前台程序
+    9. novel-front 目录下使用`bin/novel-front.sh status`命令查看前台程序的运行状态
 
-**喜欢此项目的可以给我的GitHub和Gitee加个Star支持一下 。**
+- 后台安装
 
-#### 其他安装教程（如果链接打不开，可关注公众号获取）
+    1. 上传 novel-admin/target/build/novel-admin.zip 压缩包到 Linux 服务器的 novel-admin 目录
+    2. 使用`unzip novel-admin.zip`命令解压 novel-admin.zip
+    3. 修改 `config/application-prod.yml` 文件中的数据库配置
+    4. novel-admin 目录下使用`bin/novel-admin.sh start`命令启动后台程序
+    5. 打开浏览器，默认`8088`端口访问
+    6. novel-admin 目录下使用`bin/novel-admin.sh stop`命令停止后台程序
+    7. novel-admin 目录下使用`bin/novel-admin.sh restart`命令重启后台程序
+    8. novel-admin 目录下使用`bin/novel-admin.sh status`命令查看后台程序的运行状态
 
-##### version>=3.5.0版本
-
-包安装及低版本升级教程：[点击前往](https://my.oschina.net/java2nb/blog/4914688)
-
-##### 3.3.0<=version<3.5.0版本
-
-包安装教程：[点击前往](https://my.oschina.net/java2nb/blog/4842472) 
-
-##### version<3.3.0版本
-
-包安装教程：[点击前往](https://my.oschina.net/java2nb/blog/4272630) 
-
-宝塔安装教程（非官方）：[点击前往](https://www.daniao.org/9166.html) 
-
-docker安装教程：[点击前往](https://my.oschina.net/java2nb/blog/4271989)
+**注：部分环境（如新版 Mac OS 系统）获取不到主机 IP，需要手动修改 hosts 文件，增加 IP-主机名（通过 hostname 命令查看主机名）的映射。**
 
 #### 代码仓库
 
- GitHub仓库地址： https://github.com/201206030/novel-plus 
+Github 仓库地址： https://github.com/201206030/novel-plus
 
- Gitee仓库地址： https://gitee.com/novel_dev_team/novel-plus
+Gitee 仓库地址： https://gitee.com/novel_dev_team/novel-plus
 
-#### QQ交流群
+#### QQ 交流群
 
 [点击前往官网查看](https://xiongxyang.gitee.io/service.htm)
 
 #### 微信交流群
 
-微信群二维码会过期，所以每周在公众号更新一次，请关注公众号“**IT进阶**”回复关键词“**微信群**”获取。
+微信群二维码会过期，所以每周在公众号更新一次，请关注公众号“**xxyopen**”回复关键词“**微信群**”获取。
 
 问问题的三要素
 
@@ -313,17 +279,10 @@ docker安装教程：[点击前往](https://my.oschina.net/java2nb/blog/4271989)
 
 - 服务器的费用也是一笔开销
 - 为用户提供更好的开发环境
-- 一杯咖啡 
+- 一杯咖啡
 
 ![mini-code](https://s1.ax1x.com/2020/10/31/BUQJwq.png)
 
 #### 免责声明
 
-本项目提供的爬虫工具仅用于采集项目初期的测试数据，请勿用于商业盈利。
-用户使用本系统从事任何违法违规的事情，一切后果由用户自行承担，作者不承担任何责任。
-
-#### 备注
-
-精品小说屋所有相关项目均已在开源中国公开，感兴趣的可进入[开源中国](https://www.oschina.net/p/fiction_house)按关键字`精品小说屋`搜索。
-
-[![index](https://s1.ax1x.com/2020/07/03/NOSuMF.jpg)](https://www.aliyun.com/minisite/goods?userCode=uf4nasee )
+本项目提供的爬虫工具仅用于采集项目初期的测试数据，请勿用于商业盈利。 用户使用本系统从事任何违法违规的事情，一切后果由用户自行承担，作者不承担任何责任。

doc/sql/readme.md

@@ -0,0 +1,3 @@
+1. novel_plus.sql 为全量 sql 文件，yyyyMMdd.sql 为增量 sql 文件
+2. 第一次安装只需要执行 novel_plus.sql 文件即可
+3. 后续版本升级需要根据上次代码版本的时间，执行该日期之后的增量 sql 文件（简单来说就是 sql 文件夹中相较于上次多出来的 sql 文件）

doc/sql/sql文件说明.txt

@@ -1,3 +0,0 @@
-novel_plus.sqlΪȫ<CEAA><C8AB>sql<71>ļ<EFBFBD><C4BC><EFBFBD>yyyyMMdd.sqlΪ<6C><CEAA><EFBFBD><EFBFBD>sql<71>ļ<EFBFBD><C4BC><EFBFBD>
-<EFBFBD><EFBFBD><EFBFBD>û<EFBFBD>ֻ<EFBFBD><EFBFBD>Ҫִ<EFBFBD><EFBFBD>novel_plus.sql<71>ļ<EFBFBD><C4BC><EFBFBD><EFBFBD>ɡ<EFBFBD>
-<EFBFBD><EFBFBD><EFBFBD>û<EFBFBD><EFBFBD><EFBFBD><EFBFBD>´<EFBFBD><EFBFBD><EFBFBD><EFBFBD>󣬸<EFBFBD><EFBFBD><EFBFBD><EFBFBD>ϴδ<EFBFBD><EFBFBD><EFBFBD><EFBFBD>ĸ<EFBFBD><EFBFBD><EFBFBD>ʱ<EFBFBD>䣬ִ<EFBFBD>и<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֮<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>sql<EFBFBD>ļ<EFBFBD><EFBFBD><EFBFBD>

novel-admin/pom.xml

@@ -5,7 +5,7 @@
 
     <groupId>com.java2nb</groupId>
     <artifactId>novel-admin</artifactId>
-    <version>3.6.1</version>
+    <version>3.6.2</version>
     <packaging>jar</packaging>
 
     <name>novel-admin</name>
@@ -106,7 +106,7 @@
         <dependency>
             <groupId>org.apache.shiro</groupId>
             <artifactId>shiro-spring</artifactId>
-            <version>1.3.2</version>
+            <version>1.7.0</version>
         </dependency>
         <!-- shiro ehcache -->
         <dependency>
@@ -224,67 +224,36 @@
         <!--<scope>provided</scope>-->
         <!--</dependency>-->
     </dependencies>
-  <!--  <build>
-        <plugins>
-            &lt;!&ndash;<plugin>
-                <groupId>org.springframework.boot</groupId>
-                <artifactId>spring-boot-maven-plugin</artifactId>
-                <configuration>
-                    <executable>true</executable>
-                </configuration>
-            </plugin>&ndash;&gt;
-            &lt;!&ndash;SpringBoot项目默认使用spring-boot-maven-plugin，要打成被其他项目引用的jar包，需要更换此插件&ndash;&gt;
-            &lt;!&ndash; <plugin>
-                 <groupId>org.apache.maven.plugins</groupId>
-                 <artifactId>maven-compiler-plugin</artifactId>
-                 <configuration>
-                     <source>1.8</source>
-                     <target>1.8</target>
-                     <encoding>UTF-8</encoding>
-                 </configuration>
-             </plugin>&ndash;&gt;
-            <plugin>
-                <groupId>org.springframework.boot</groupId>
-                <artifactId>spring-boot-maven-plugin</artifactId>
-            </plugin>
-        </plugins>
-    </build>-->
+    <!--  <build>
+          <plugins>
+              &lt;!&ndash;<plugin>
+                  <groupId>org.springframework.boot</groupId>
+                  <artifactId>spring-boot-maven-plugin</artifactId>
+                  <configuration>
+                      <executable>true</executable>
+                  </configuration>
+              </plugin>&ndash;&gt;
+              &lt;!&ndash;SpringBoot项目默认使用spring-boot-maven-plugin，要打成被其他项目引用的jar包，需要更换此插件&ndash;&gt;
+              &lt;!&ndash; <plugin>
+                   <groupId>org.apache.maven.plugins</groupId>
+                   <artifactId>maven-compiler-plugin</artifactId>
+                   <configuration>
+                       <source>1.8</source>
+                       <target>1.8</target>
+                       <encoding>UTF-8</encoding>
+                   </configuration>
+               </plugin>&ndash;&gt;
+              <plugin>
+                  <groupId>org.springframework.boot</groupId>
+                  <artifactId>spring-boot-maven-plugin</artifactId>
+              </plugin>
+          </plugins>
+      </build>-->
     <build>
         <plugins>
             <plugin>
-                <!--打包时去除第三方依赖-->
                 <groupId>org.springframework.boot</groupId>
                 <artifactId>spring-boot-maven-plugin</artifactId>
-                <configuration>
-                    <layout>ZIP</layout>
-                    <includes>
-                        <include>
-                            <groupId>non-exists</groupId>
-                            <artifactId>non-exists</artifactId>
-                        </include>
-                    </includes>
-                </configuration>
-            </plugin>
-            <!--拷贝第三方依赖文件到指定目录-->
-            <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-dependency-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <id>copy-dependencies</id>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>copy-dependencies</goal>
-                        </goals>
-                        <configuration>
-                            <!--target/lib是依赖jar包的输出目录，根据自己喜好配置-->
-                            <outputDirectory>target/lib</outputDirectory>
-                            <excludeTransitive>false</excludeTransitive>
-                            <stripVersion>false</stripVersion>
-                            <includeScope>runtime</includeScope>
-                        </configuration>
-                    </execution>
-                </executions>
             </plugin>
             <plugin>
                 <artifactId>maven-antrun-plugin</artifactId>
@@ -298,28 +267,24 @@
                         <configuration>
                             <tasks>
                                 <!-- 文件夹 -->
-                                <copy todir="${project.build.directory}/build/conf" overwrite="true">
-                                    <fileset dir="${basedir}/src/main/resources">
-                                        <include name="**/*.*"/>
-                                        <exclude name="mybatis/*/*.*"/>
+                                <copy todir="${project.build.directory}/build/config" overwrite="true">
+                                    <fileset dir="${basedir}/src/main/build/config">
+                                        <include name="*.*"/>
                                     </fileset>
                                 </copy>
-                                <move todir="${project.build.directory}/build/lib">
-                                    <fileset dir="target/lib"/>
-                                </move>
                                 <copy file="${project.build.directory}/${project.artifactId}-${project.version}.jar"
-                                      tofile="${project.build.directory}/build/${project.artifactId}.jar" />
+                                      tofile="${project.build.directory}/build/${project.artifactId}.jar"/>
 
                                 <fixcrlf srcdir="${basedir}/src/main/build/scripts" eol="unix"/>
                                 <copy todir="${project.build.directory}/build/bin">
                                     <fileset dir="${basedir}/src/main/build/scripts">
-                                        <include name="*.sh" />
-                                        <include name="*.txt" />
-                                        <include name="*.bat" />
+                                        <include name="*.sh"/>
+                                        <include name="*.txt"/>
+                                        <include name="*.bat"/>
                                     </fileset>
                                 </copy>
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
-                                    <zipfileset filemode="755" dir= '${project.build.directory}/build/' />
+                                    <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
                             </tasks>
                         </configuration>

novel-admin/src/main/build/config/application-prod.yml

@@ -0,0 +1,10 @@
+#端口号
+server:
+  port: 8088
+spring:
+  datasource:
+    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+
+  

novel-admin/src/main/build/scripts/novel-admin.sh

@@ -0,0 +1,94 @@
+#!/bin/sh
+APP_NAME=novel-admin
+JAR_NAME=$APP_NAME\.jar
+#PID  代表是PID文件
+PID=$APP_NAME\.pid
+
+
+#使用说明，用来提示输入参数
+usage() {
+    echo "Usage: ./novel-admin.sh [start|stop|restart|status]"
+    exit 1
+}
+
+#检查程序是否在运行
+is_exist(){
+  pid=`ps -ef|grep $JAR_NAME|grep -v grep|awk '{print $2}' `
+  #如果不存在返回1，存在返回0     
+  if [ -z "${pid}" ]; then
+   return 1
+  else
+    return 0
+  fi
+}
+
+#启动方法
+start(){
+  is_exist
+  if [ $? -eq "0" ]; then 
+    echo ">>> 小说精品屋后台正在运行 PID = ${pid} <<<" 
+  else 
+    echo ">>> 小说精品屋后台开始启动 <<<" 
+    nohup java -jar -Dspring.profiles.active=prod $JAR_NAME >/dev/null 2>&1 &
+    sleep 20
+    echo $! > $PID
+    echo ">>> 小说精品屋后台启动完成 PID = $! <<<" 
+    status
+   fi
+  }
+
+#停止方法
+stop(){
+  #is_exist
+  pidf=$(cat $PID)
+  #echo "$pidf"  
+  echo ">>> 小说精品屋后台 PID = $pidf 开始停止 <<<"
+  kill $pidf
+  rm -rf $PID
+  sleep 2
+  is_exist
+  if [ $? -eq "0" ]; then 
+    echo ">>> 小说精品屋后台 PID = $pid 开始强制停止 <<<"
+    kill -9  $pid
+    sleep 2
+    status 
+  else
+    status
+  fi  
+}
+
+#输出运行状态
+status(){
+  is_exist
+  if [ $? -eq "0" ]; then
+    echo ">>> 小说精品屋后台正在运行 PID = ${pid} <<<"
+  else
+    echo ">>> 小说精品屋后台没有运行 <<<"
+  fi
+}
+
+#重启
+restart(){
+  stop
+  start
+}
+
+#根据输入参数，选择执行对应方法，不输入则执行使用说明
+case "$1" in
+  "start")
+    start
+    ;;
+  "stop")
+    stop
+    ;;
+  "status")
+    status
+    ;;
+  "restart")
+    restart
+    ;;
+  *)
+    usage
+    ;;
+esac
+exit 0

novel-admin/src/main/build/scripts/readme.txt

@@ -1,8 +0,0 @@
-1：linux启动环境
-sh start.sh
-
-3：windows启动环境
-windows-start.bat
-
-3：linux停止应用
-sh stop.sh

novel-admin/src/main/build/scripts/start.sh

@@ -1,47 +0,0 @@
-#!/bin/bash
-
-ENGINE=novel-admin.jar
-cd ../
-
-#部署目路
-DEPLOY_DIR=`pwd`
-#获取到当前目录的名称
-SERVER_NAME=`basename $DEPLOY_DIR`
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$ENGINE" |awk '{print $2}'`
-#设置日志文件的输出目录
-LOGS_DIR=$DEPLOY_DIR/logs
-if [ ! -d $LOGS_DIR ]; then
-    mkdir $LOGS_DIR
-fi
-#日志
-STDOUT_FILE=$LOGS_DIR/stdout.log
-#JAVA 环境配置
-JAVA_OPTS=" -Djava.net.preferIPv4Stack=true -Dlog.home=$LOGS_DIR"
-
-JAVA_MEM_OPTS=" -server -Xms1024m -Xmx1024m -XX:MetaspaceSize=64m -XX:MaxMetaspaceSize=256m  -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=50 -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Xloggc:$LOGS_DIR/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof "
-#退出标志
-RETVAL="0"
-
-if [ -n "$PIDS" ]; then
-		echo "ERROR: The $SERVER_NAME already started!"
-		echo "PID: $PIDS"
-		exit $RETVAL
-fi
-
-nohup java -jar  $JAVA_OPTS $JAVA_MEM_OPTS  -Dloader.path=conf,lib $ENGINE > $STDOUT_FILE 2>&1 &
-COUNT=0
-	while [ $COUNT -lt 1 ]; do
-		echo -e ".\c"
-		sleep 1
-		COUNT=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}' | wc -l`
-		if [ $COUNT -gt 0 ]; then
-			break
-		fi
-	done
-
-	echo "OK!"
-	PIDS=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}'`
-	echo "PID: $PIDS"
-	echo "STDOUT: $STDOUT_FILE"

novel-admin/src/main/build/scripts/stop.sh

@@ -1,33 +0,0 @@
-#!/bin/bash
-SERVER_NAME=novel-admin.jar
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$SERVER_NAME" |awk '{print $2}'`
-if [ -z "$PIDS" ]; then
-    		echo "ERROR: The $SERVER_NAME does not started!"
-    	exit 1
-	fi
-
-	echo -e "Stopping the $SERVER_NAME ...\c"
-	for PID in $PIDS ; do
-    		kill $PID > /dev/null 2>&1
-	done
-
-	COUNT=0
-	while [ $COUNT -lt 1 ]; do
-    		echo -e ".\c"
-    		sleep 1
-    		COUNT=1
-    		for PID in $PIDS ; do
-        	PID_EXIST=`ps -f -p $PID | grep java`
-        	if [ -n "$PID_EXIST" ]; then
-            		COUNT=0
-            		break
-        	fi
-    		done
-	done
-
-	echo "OK!"
-	echo "PID: $PIDS"
-	PIDS=""
-

novel-admin/src/main/build/scripts/windows-start.bat

@@ -1,10 +0,0 @@
-@echo off
-setlocal enabledelayedexpansion
-set JAVA=java
-
-set OPTS=-XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -Xms1024m -Xmx1024m -Xmn256m -Xss256k -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC
-set ENGINE=novel-admin.jar
-cd ../
-java -jar %OPTS%   -Dloader.path=conf,lib %ENGINE%
-pause
-

novel-admin/src/main/java/com/java2nb/AdminApplication.java

@@ -1,12 +1,18 @@
 package com.java2nb;
 
+import lombok.extern.slf4j.Slf4j;
 import org.mybatis.spring.annotation.MapperScan;
+import org.springframework.boot.CommandLineRunner;
 import org.springframework.boot.SpringApplication;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 import org.springframework.boot.web.servlet.ServletComponentScan;
 import org.springframework.cache.annotation.EnableCaching;
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.annotation.Bean;
 import org.springframework.transaction.annotation.EnableTransactionManagement;
 
+import java.net.InetAddress;
+
 
 @EnableTransactionManagement
 @ServletComponentScan
@@ -15,9 +21,17 @@ import org.springframework.transaction.annotation.EnableTransactionManagement;
         org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration.class
 })
 @EnableCaching
+@Slf4j
 public class AdminApplication {
     public static void main(String[] args) {
         SpringApplication.run(AdminApplication.class, args);
     }
 
+    @Bean
+    public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
+        return args -> {
+            log.info("项目启动啦，访问路径：{}", "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment().getProperty("server.port"));
+        };
+    }
+
 }

novel-admin/src/main/resources/application.yml

@@ -36,3 +36,7 @@ mybatis:
   typeAliasesPackage: com.java2nb.**.domain
 #[弃用]配置缓存和session存储方式，默认ehcache,可选redis,[弃用]调整至 spring cache type【shiro.用户，权限，session，spring.cache通用】
 #[弃用]cacheType: ehcache
+
+
+logging:
+  config: classpath:logback-boot.xml

novel-admin/src/main/resources/logback-boot.xml

@@ -24,7 +24,7 @@
     <!-- RollingFileAppender：滚动记录文件，先将日志记录到指定文件，当符合某个条件时，将日志记录到其他文件 -->
     <!-- 以下的大概意思是：1.先按日期存日志，日期变了，将前一天的日志文件名重命名为XXX%日期%索引，新的日志仍然是demo.log -->
     <!-- 2.如果日期没有发生变化，但是当前日志的文件大小超过1KB时，对当前日志进行分割 重命名 -->
-    <appender name="debug" class="ch.qos.logback.core.rolling.RollingFileAppender">
+    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
 
         <File>logs/debug.log</File>
         <!-- rollingPolicy:当发生滚动时，决定 RollingFileAppender 的行为，涉及文件移动和重命名。 -->
@@ -52,11 +52,13 @@
     <!-- 控制台输出日志级别 -->
     <root level="INFO">
         <appender-ref ref="STDOUT" />
+        <appender-ref ref="FILE" />
     </root>
     <!-- 指定项目中某个包，当有日志操作行为时的日志记录级别 -->
     <!-- com.maijinjie.springboot 为根包，也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG -->
     <!-- 级别依次为【从高到低】：FATAL > ERROR > WARN > INFO > DEBUG > TRACE -->
     <logger name="com.java2nb" level="DEBUG" additivity="false">
-        <appender-ref ref="debug" />
+        <appender-ref ref="STDOUT" />
+        <appender-ref ref="FILE" />
     </logger>
 </configuration>

novel-common/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>3.6.1</version>
+        <version>3.6.2</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -142,7 +142,6 @@
         </dependency>
 
 
-
     </dependencies>
 
 

novel-common/src/main/java/com/java2nb/novel/core/cache/CacheKey.java

@@ -65,4 +65,8 @@ public interface CacheKey {
      * 累积的小说点击量
      * */
     String BOOK_ADD_VISIT_COUNT = "bookAddVisitCount";
-}
+    /**
+     * 测试爬虫规则缓存
+     */
+    String BOOK_TEST_PARSE = "testParse";
+}

novel-common/src/main/java/com/java2nb/novel/core/config/HttpProxyProperties.java

@@ -0,0 +1,22 @@
+package com.java2nb.novel.core.config;
+
+import lombok.Data;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.stereotype.Component;
+
+/**
+ * @author xiongxiaoyang
+ * @date 2022/7/14
+ */
+@Data
+@Component
+@ConfigurationProperties(prefix = "http.proxy")
+public class HttpProxyProperties {
+
+    private Boolean enabled;
+
+    private String ip;
+
+    private Integer port;
+
+}

novel-common/src/main/java/com/java2nb/novel/core/utils/RestTemplateUtil.java

@@ -1,6 +1,8 @@
 package com.java2nb.novel.core.utils;
 
+import com.java2nb.novel.core.config.HttpProxyProperties;
 import lombok.SneakyThrows;
+import org.apache.http.HttpHost;
 import org.apache.http.config.Registry;
 import org.apache.http.config.RegistryBuilder;
 import org.apache.http.conn.socket.ConnectionSocketFactory;
@@ -8,37 +10,46 @@ import org.apache.http.conn.socket.PlainConnectionSocketFactory;
 import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
 import org.apache.http.conn.ssl.TrustStrategy;
 import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
 import org.apache.http.impl.client.HttpClients;
 import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
 import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
 import org.springframework.http.converter.HttpMessageConverter;
 import org.springframework.http.converter.StringHttpMessageConverter;
+import org.springframework.stereotype.Component;
 import org.springframework.web.client.RestTemplate;
 
 import javax.net.ssl.SSLContext;
 import java.nio.charset.Charset;
 import java.security.cert.X509Certificate;
 import java.util.List;
+import java.util.Objects;
 
+@Component
 public class RestTemplateUtil {
 
+    private static HttpProxyProperties httpProxyProperties;
 
+    RestTemplateUtil(HttpProxyProperties properties) {
+        httpProxyProperties = properties;
+    }
 
     @SneakyThrows
     public static RestTemplate getInstance(String charset) {
+
         TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
 
         //忽略证书
         SSLContext sslContext = org.apache.http.ssl.SSLContexts.custom()
-                .loadTrustMaterial(null, acceptingTrustStrategy)
-                .build();
+            .loadTrustMaterial(null, acceptingTrustStrategy)
+            .build();
 
         SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(sslContext);
 
         Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
-                .register("http", PlainConnectionSocketFactory.getSocketFactory())
-                .register("https", csf)
-                .build();
+            .register("http", PlainConnectionSocketFactory.getSocketFactory())
+            .register("https", csf)
+            .build();
         PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
 
         //连接池的最大连接数，0代表不限；如果取0，需要考虑连接泄露导致系统崩溃的后果
@@ -46,22 +57,26 @@ public class RestTemplateUtil {
         //每个路由的最大连接数,如果只调用一个地址,可以将其设置为最大连接数
         connectionManager.setDefaultMaxPerRoute(300);
 
-        CloseableHttpClient httpClient = HttpClients.custom()
-                .setConnectionManager(connectionManager)
-                .build();
-
+        HttpClientBuilder clientBuilder = HttpClients.custom();
+        if (Objects.nonNull(httpProxyProperties) && Boolean.TRUE.equals(httpProxyProperties.getEnabled())) {
+            HttpHost proxy = new HttpHost(httpProxyProperties.getIp(), httpProxyProperties.getPort());
+            clientBuilder.setProxy(proxy);
+        }
+        CloseableHttpClient httpClient = clientBuilder.setConnectionManager(connectionManager)
+            .build();
 
         HttpComponentsClientHttpRequestFactory requestFactory =
-                new HttpComponentsClientHttpRequestFactory();
+            new HttpComponentsClientHttpRequestFactory();
 
         requestFactory.setHttpClient(httpClient);
         requestFactory.setConnectionRequestTimeout(3000);
         requestFactory.setConnectTimeout(3000);
         requestFactory.setReadTimeout(30000);
+
         RestTemplate restTemplate = new RestTemplate(requestFactory);
         List<HttpMessageConverter<?>> list = restTemplate.getMessageConverters();
         for (HttpMessageConverter<?> httpMessageConverter : list) {
-            if(httpMessageConverter instanceof StringHttpMessageConverter) {
+            if (httpMessageConverter instanceof StringHttpMessageConverter) {
                 ((StringHttpMessageConverter) httpMessageConverter).setDefaultCharset(Charset.forName(charset));
                 break;
             }

novel-common/src/main/resources/application-common-dev.yml

@@ -1,6 +1,6 @@
 spring:
   profiles:
-    include: [common]
+    include: [ common ]
   main:
     allow-bean-definition-overriding: true
   #Redis服务器IP
@@ -54,24 +54,30 @@ sharding:
         props:
           sql.show: true
         tables:
-          book_content:  #book_content表
+          book_content: #book_content表
             key-generator-column-name: id  #主键
             actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
             #            database-strategy:  #分库策略
             #              inline:
             #                sharding-column: book_id
             #                algorithm-expression: ds${book_id % 10}
-            table-strategy:  #分表策略
+            table-strategy: #分表策略
               inline:
                 shardingColumn: index_id
                 algorithm-expression: book_content${index_id % 10}
 
 
-
-
-
-
 content:
   save:
     storage: db #存储介质，db：数据库，file：txt文本
-    path: /Users/xiongxiaoyang/books  #txt小说文本保存路径
+    path: /Users/xiongxiaoyang/books  #txt小说文本保存路径
+
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: u493.kdltps.com
+    # 代理端口号
+    port: 15818

novel-common/src/main/resources/application-common-prod.yml

@@ -1,6 +1,6 @@
 spring:
   profiles:
-    include: [common]
+    include: [ common ]
   main:
     allow-bean-definition-overriding: true
   #Redis服务器IP
@@ -54,14 +54,14 @@ sharding:
         props:
           sql.show: true
         tables:
-          book_content:  #book_content表
+          book_content: #book_content表
             key-generator-column-name: id  #主键
             actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
             #            database-strategy:  #分库策略
             #              inline:
             #                sharding-column: book_id
             #                algorithm-expression: ds${book_id % 10}
-            table-strategy:  #分表策略
+            table-strategy: #分表策略
               inline:
                 shardingColumn: index_id
                 algorithm-expression: book_content${index_id % 10}
@@ -79,7 +79,15 @@ content:
     storage: db #存储介质，db：数据库，file：txt文本
     path: /Users/xiongxiaoyang/books  #txt小说文本保存路径
 
-
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: 40.83.102.86
+    # 代理端口号
+    port: 80
 
 
 

novel-crawl/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>3.6.1</version>
+        <version>3.6.2</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -31,64 +31,9 @@
 
     <build>
         <plugins>
-            <!-- 将相同groupId的依赖模块打包进来 -->
             <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-shade-plugin</artifactId>
-                <configuration>
-                    <createDependencyReducedPom>false</createDependencyReducedPom>
-                </configuration>
-                <executions>
-                    <execution>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>shade</goal>
-                        </goals>
-                        <configuration>
-                            <artifactSet>
-                                <includes>
-                                    <include>${project.groupId}:*:*</include>
-                                </includes>
-                            </artifactSet>
-                        </configuration>
-                    </execution>
-                </executions>
-            </plugin>
-            <plugin>
-                <!--打包时去除第三方依赖-->
                 <groupId>org.springframework.boot</groupId>
                 <artifactId>spring-boot-maven-plugin</artifactId>
-                <configuration>
-                    <layout>ZIP</layout>
-                    <includes>
-                        <include>
-                            <groupId>non-exists</groupId>
-                            <artifactId>non-exists</artifactId>
-                        </include>
-                    </includes>
-                </configuration>
-            </plugin>
-            <!--拷贝第三方依赖文件到指定目录-->
-            <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-dependency-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <id>copy-dependencies</id>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>copy-dependencies</goal>
-                        </goals>
-                        <configuration>
-                            <!--target/lib是依赖jar包的输出目录，根据自己喜好配置-->
-                            <outputDirectory>target/lib</outputDirectory>
-                            <excludeTransitive>false</excludeTransitive>
-                            <stripVersion>false</stripVersion>
-                            <includeScope>runtime</includeScope>
-                            <excludeGroupIds>${project.groupId}</excludeGroupIds>
-                        </configuration>
-                    </execution>
-                </executions>
             </plugin>
             <plugin>
                 <artifactId>maven-antrun-plugin</artifactId>
@@ -102,28 +47,24 @@
                         <configuration>
                             <tasks>
                                 <!-- 文件夹 -->
-                                <copy todir="${project.build.directory}/build/conf" overwrite="true">
-                                    <fileset dir="${basedir}/src/main/resources">
-                                        <include name="**/*.*"/>
-                                        <exclude name="mybatis/*/*.*"/>
+                                <copy todir="${project.build.directory}/build/config" overwrite="true">
+                                    <fileset dir="${basedir}/src/main/build/config">
+                                        <include name="*.*"/>
                                     </fileset>
                                 </copy>
-                                <move todir="${project.build.directory}/build/lib">
-                                <fileset dir="target/lib"/>
-                                </move>
                                 <copy file="${project.build.directory}/${project.artifactId}-${project.version}.jar"
-                                      tofile="${project.build.directory}/build/${project.artifactId}.jar" />
+                                      tofile="${project.build.directory}/build/${project.artifactId}.jar"/>
 
                                 <fixcrlf srcdir="${basedir}/src/main/build/scripts" eol="unix"/>
                                 <copy todir="${project.build.directory}/build/bin">
                                     <fileset dir="${basedir}/src/main/build/scripts">
-                                        <include name="*.sh" />
-                                        <include name="*.txt" />
-                                        <include name="*.bat" />
+                                        <include name="*.sh"/>
+                                        <include name="*.txt"/>
+                                        <include name="*.bat"/>
                                     </fileset>
                                 </copy>
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
-                                    <zipfileset filemode="755" dir= '${project.build.directory}/build/' />
+                                    <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
                             </tasks>
                         </configuration>

novel-crawl/src/main/build/config/application-common-prod.yml

@@ -0,0 +1,49 @@
+#端口号
+server:
+  port: 8083
+
+#不分表的数据库配置
+spring:
+  datasource:
+    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: root
+
+####使用shardingJdbc时，
+####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
+##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
+sharding:
+  jdbc:
+    datasource:
+      ds0:
+        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+        username: root
+        password: test123456
+
+#登录用户名密码
+admin:
+  username: admin
+  password: admin
+#
+##爬虫自动更新的线程数
+##建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
+##随着小说数量的增多可以逐渐增加，但建议不要超出CPU的线程数
+crawl:
+  update:
+    thread: 1
+
+#小说内容保存配置
+content:
+  save:
+    storage: db #存储介质，db：数据库，file：txt文本
+    path: /Users/xiongxiaoyang/books  #txt小说文本保存路径
+
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: u493.kdltps.com
+    # 代理端口号
+    port: 15818

novel-crawl/src/main/build/scripts/novel-crawl.sh

@@ -0,0 +1,94 @@
+#!/bin/sh
+APP_NAME=novel-crawl
+JAR_NAME=$APP_NAME\.jar
+#PID  代表是PID文件
+PID=$APP_NAME\.pid
+
+
+#使用说明，用来提示输入参数
+usage() {
+    echo "Usage: ./novel-crawl.sh [start|stop|restart|status]"
+    exit 1
+}
+
+#检查程序是否在运行
+is_exist(){
+  pid=`ps -ef|grep $JAR_NAME|grep -v grep|awk '{print $2}' `
+  #如果不存在返回1，存在返回0     
+  if [ -z "${pid}" ]; then
+   return 1
+  else
+    return 0
+  fi
+}
+
+#启动方法
+start(){
+  is_exist
+  if [ $? -eq "0" ]; then 
+    echo ">>> 小说精品屋爬虫正在运行 PID = ${pid} <<<" 
+  else 
+    echo ">>> 小说精品屋爬虫开始启动 <<<" 
+    nohup java -jar -Dspring.profiles.active=prod $JAR_NAME >/dev/null 2>&1 &
+    sleep 10
+    echo $! > $PID
+    echo ">>> 小说精品屋爬虫启动完成 PID = $! <<<" 
+    status
+   fi
+  }
+
+#停止方法
+stop(){
+  #is_exist
+  pidf=$(cat $PID)
+  #echo "$pidf"  
+  echo ">>> 小说精品屋爬虫 PID = $pidf 开始停止 <<<"
+  kill $pidf
+  rm -rf $PID
+  sleep 2
+  is_exist
+  if [ $? -eq "0" ]; then 
+    echo ">>> 小说精品屋爬虫 PID = $pid 开始强制停止 <<<"
+    kill -9  $pid
+    sleep 2
+    status 
+  else
+    status
+  fi  
+}
+
+#输出运行状态
+status(){
+  is_exist
+  if [ $? -eq "0" ]; then
+    echo ">>> 小说精品屋爬虫正在运行 PID = ${pid} <<<"
+  else
+    echo ">>> 小说精品屋爬虫没有运行 <<<"
+  fi
+}
+
+#重启
+restart(){
+  stop
+  start
+}
+
+#根据输入参数，选择执行对应方法，不输入则执行使用说明
+case "$1" in
+  "start")
+    start
+    ;;
+  "stop")
+    stop
+    ;;
+  "status")
+    status
+    ;;
+  "restart")
+    restart
+    ;;
+  *)
+    usage
+    ;;
+esac
+exit 0

novel-crawl/src/main/build/scripts/readme.txt

@@ -1,8 +0,0 @@
-1：linux启动环境
-sh start.sh
-
-3：windows启动环境
-windows-start.bat
-
-3：linux停止应用
-sh stop.sh

novel-crawl/src/main/build/scripts/start.sh

@@ -1,47 +0,0 @@
-#!/bin/bash
-
-ENGINE=novel-crawl.jar
-cd ../
-
-#部署目路
-DEPLOY_DIR=`pwd`
-#获取到当前目录的名称
-SERVER_NAME=`basename $DEPLOY_DIR`
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$ENGINE" |awk '{print $2}'`
-#设置日志文件的输出目录
-LOGS_DIR=$DEPLOY_DIR/logs
-if [ ! -d $LOGS_DIR ]; then
-    mkdir $LOGS_DIR
-fi
-#日志
-STDOUT_FILE=$LOGS_DIR/stdout.log
-#JAVA 环境配置
-JAVA_OPTS=" -Djava.net.preferIPv4Stack=true -Dlog.home=$LOGS_DIR"
-
-JAVA_MEM_OPTS=" -server -Xms1024m -Xmx1024m -XX:MetaspaceSize=64m -XX:MaxMetaspaceSize=256m  -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=50 -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Xloggc:$LOGS_DIR/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof "
-#退出标志
-RETVAL="0"
-
-if [ -n "$PIDS" ]; then
-		echo "ERROR: The $SERVER_NAME already started!"
-		echo "PID: $PIDS"
-		exit $RETVAL
-fi
-
-nohup java -jar  $JAVA_OPTS $JAVA_MEM_OPTS  -Dloader.path=conf,lib $ENGINE > $STDOUT_FILE 2>&1 &
-COUNT=0
-	while [ $COUNT -lt 1 ]; do
-		echo -e ".\c"
-		sleep 1
-		COUNT=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}' | wc -l`
-		if [ $COUNT -gt 0 ]; then
-			break
-		fi
-	done
-
-	echo "OK!"
-	PIDS=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}'`
-	echo "PID: $PIDS"
-	echo "STDOUT: $STDOUT_FILE"

novel-crawl/src/main/build/scripts/stop.sh

@@ -1,33 +0,0 @@
-#!/bin/bash
-SERVER_NAME=novel-crawl.jar
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$SERVER_NAME" |awk '{print $2}'`
-if [ -z "$PIDS" ]; then
-    		echo "ERROR: The $SERVER_NAME does not started!"
-    	exit 1
-	fi
-
-	echo -e "Stopping the $SERVER_NAME ...\c"
-	for PID in $PIDS ; do
-    		kill $PID > /dev/null 2>&1
-	done
-
-	COUNT=0
-	while [ $COUNT -lt 1 ]; do
-    		echo -e ".\c"
-    		sleep 1
-    		COUNT=1
-    		for PID in $PIDS ; do
-        	PID_EXIST=`ps -f -p $PID | grep java`
-        	if [ -n "$PID_EXIST" ]; then
-            		COUNT=0
-            		break
-        	fi
-    		done
-	done
-
-	echo "OK!"
-	echo "PID: $PIDS"
-	PIDS=""
-

novel-crawl/src/main/build/scripts/windows-start.bat

@@ -1,10 +0,0 @@
-@echo off
-setlocal enabledelayedexpansion
-set JAVA=java
-
-set OPTS=-XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -Xms1024m -Xmx1024m -Xmn256m -Xss256k -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC
-set ENGINE=novel-crawl.jar
-cd ../
-java -jar %OPTS%   -Dloader.path=conf,lib %ENGINE%
-pause
-

novel-crawl/src/main/java/com/java2nb/novel/CrawlNovelApplication.java

@@ -1,12 +1,18 @@
 package com.java2nb.novel;
 
+import lombok.extern.slf4j.Slf4j;
 import org.mybatis.spring.annotation.MapperScan;
+import org.springframework.boot.CommandLineRunner;
 import org.springframework.boot.SpringApplication;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 import org.springframework.boot.web.servlet.ServletComponentScan;
 import org.springframework.cache.annotation.EnableCaching;
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.annotation.Bean;
 import org.springframework.scheduling.annotation.EnableScheduling;
 
+import java.net.InetAddress;
+
 /**
  * @author Administrator
  */
@@ -15,11 +21,19 @@ import org.springframework.scheduling.annotation.EnableScheduling;
 @EnableScheduling
 @ServletComponentScan
 @MapperScan(basePackages = {"com.java2nb.novel.mapper"})
+@Slf4j
 public class CrawlNovelApplication {
 
     public static void main(String[] args) {
         SpringApplication.run(CrawlNovelApplication.class);
     }
 
+    @Bean
+    public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
+        return args -> {
+            log.info("项目启动啦，访问路径：{}", "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment().getProperty("server.port"));
+        };
+    }
+
 
 }

novel-crawl/src/main/java/com/java2nb/novel/controller/CrawlController.java

@@ -1,5 +1,8 @@
 package com.java2nb.novel.controller;
 
+import com.java2nb.novel.core.cache.CacheKey;
+import com.java2nb.novel.core.cache.CacheService;
+import com.java2nb.novel.core.utils.HttpUtil;
 import io.github.xxyopen.model.page.PageBean;
 
 import com.java2nb.novel.entity.CrawlSingleTask;
@@ -9,6 +12,11 @@ import io.github.xxyopen.model.resp.RestResult;
 import lombok.RequiredArgsConstructor;
 import org.springframework.web.bind.annotation.*;
 
+import java.util.HashMap;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
 /**
  * @author Administrator
  */
@@ -19,7 +27,7 @@ public class CrawlController {
 
     private final CrawlService crawlService;
 
-
+    private final CacheService cacheService;
     /**
      * 新增爬虫源
      * */
@@ -39,7 +47,70 @@ public class CrawlController {
 
         return RestResult.ok(crawlService.listCrawlByPage(page,pageSize));
     }
+    /**
+     * 获取爬虫源
+     * */
+    @GetMapping("getCrawlSource/{id}")
+    public RestResult<CrawlSource> getCrawlSource(@PathVariable("id") Integer id){
+        CrawlSource crawlSource=  crawlService.getCrawlSource(id);
+        return RestResult.ok(crawlSource);
 
+    }
+
+    /**
+     * 测试规则
+     * @param rule
+     * @param url
+     * @param isRefresh
+     * @return
+     */
+    @PostMapping("testParse")
+    public RestResult<Object> testParse(String rule,String url,String isRefresh){
+
+        Map<String,Object> resultMap=new HashMap<>();
+        String html =null;
+        if(url.startsWith("https://")||url.startsWith("http://")){
+            String refreshCache="1";
+            if(!refreshCache.equals(isRefresh)) {
+                Object cache = cacheService.getObject(CacheKey.BOOK_TEST_PARSE + url);
+                if (cache == null) {
+                    isRefresh="1";
+                }else {
+                    html = (String) cache;
+                }
+            }
+            if(refreshCache.equals(isRefresh)){
+                html = HttpUtil.getByHttpClientWithChrome(url);
+                if (html != null) {
+                    cacheService.setObject(CacheKey.BOOK_TEST_PARSE + url, html, 60 * 10);
+                }else{
+                    resultMap.put("msg","html is null");
+                    return RestResult.ok(resultMap);
+                }
+            }
+        }else{
+            resultMap.put("html","url is null");
+            return RestResult.ok(resultMap);
+        }
+        Pattern pattern = Pattern.compile(rule);
+        Matcher matcher = pattern.matcher(html);
+        boolean isFind = matcher.find();
+        resultMap.put("是否匹配",isFind);
+        if(isFind){
+            resultMap.put("匹配结果",matcher.group(1));
+        }
+       // resultMap.put("url",url);
+        return RestResult.ok(resultMap);
+    }
+    /**
+     * 修改爬虫源
+     * */
+    @PostMapping("updateCrawlSource")
+    public RestResult<Void> updateCrawlSource(CrawlSource source) {
+        crawlService.updateCrawlSource(source);
+        return RestResult.ok();
+
+    }
     /**
      * 开启或停止爬虫
      * */

novel-crawl/src/main/java/com/java2nb/novel/core/crawl/CrawlParser.java

@@ -1,6 +1,9 @@
 package com.java2nb.novel.core.crawl;
 
-import com.java2nb.novel.core.utils.*;
+import com.java2nb.novel.core.utils.HttpUtil;
+import com.java2nb.novel.core.utils.RandomBookInfoUtil;
+import com.java2nb.novel.core.utils.RestTemplateUtil;
+import com.java2nb.novel.core.utils.StringUtil;
 import com.java2nb.novel.entity.Book;
 import com.java2nb.novel.entity.BookContent;
 import com.java2nb.novel.entity.BookIndex;
@@ -9,7 +12,8 @@ import io.github.xxyopen.util.IdWorker;
 import lombok.SneakyThrows;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
-import org.springframework.http.*;
+import org.springframework.http.HttpStatus;
+import org.springframework.http.ResponseEntity;
 import org.springframework.web.client.RestTemplate;
 
 import java.text.SimpleDateFormat;
@@ -17,8 +21,6 @@ import java.util.*;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
-import static java.util.regex.Pattern.compile;
-
 /**
  * 爬虫解析器
  *
@@ -39,14 +41,14 @@ public class CrawlParser {
         String bookDetailUrl = ruleBean.getBookDetailUrl().replace("{bookId}", bookId);
         String bookDetailHtml = getByHttpClientWithChrome(bookDetailUrl);
         if (bookDetailHtml != null) {
-            Pattern bookNamePatten = compile(ruleBean.getBookNamePatten());
+            Pattern bookNamePatten = PatternFactory.getPattern(ruleBean.getBookNamePatten());
             Matcher bookNameMatch = bookNamePatten.matcher(bookDetailHtml);
             boolean isFindBookName = bookNameMatch.find();
             if (isFindBookName) {
                 String bookName = bookNameMatch.group(1);
                 //设置小说名
                 book.setBookName(bookName);
-                Pattern authorNamePatten = compile(ruleBean.getAuthorNamePatten());
+                Pattern authorNamePatten = PatternFactory.getPattern(ruleBean.getAuthorNamePatten());
                 Matcher authorNameMatch = authorNamePatten.matcher(bookDetailHtml);
                 boolean isFindAuthorName = authorNameMatch.find();
                 if (isFindAuthorName) {
@@ -54,7 +56,7 @@ public class CrawlParser {
                     //设置作者名
                     book.setAuthorName(authorName);
                     if (StringUtils.isNotBlank(ruleBean.getPicUrlPatten())) {
-                        Pattern picUrlPatten = compile(ruleBean.getPicUrlPatten());
+                        Pattern picUrlPatten = PatternFactory.getPattern(ruleBean.getPicUrlPatten());
                         Matcher picUrlMatch = picUrlPatten.matcher(bookDetailHtml);
                         boolean isFindPicUrl = picUrlMatch.find();
                         if (isFindPicUrl) {
@@ -67,7 +69,7 @@ public class CrawlParser {
                         }
                     }
                     if (StringUtils.isNotBlank(ruleBean.getScorePatten())) {
-                        Pattern scorePatten = compile(ruleBean.getScorePatten());
+                        Pattern scorePatten = PatternFactory.getPattern(ruleBean.getScorePatten());
                         Matcher scoreMatch = scorePatten.matcher(bookDetailHtml);
                         boolean isFindScore = scoreMatch.find();
                         if (isFindScore) {
@@ -77,7 +79,7 @@ public class CrawlParser {
                         }
                     }
                     if (StringUtils.isNotBlank(ruleBean.getVisitCountPatten())) {
-                        Pattern visitCountPatten = compile(ruleBean.getVisitCountPatten());
+                        Pattern visitCountPatten = PatternFactory.getPattern(ruleBean.getVisitCountPatten());
                         Matcher visitCountMatch = visitCountPatten.matcher(bookDetailHtml);
                         boolean isFindVisitCount = visitCountMatch.find();
                         if (isFindVisitCount) {
@@ -98,7 +100,7 @@ public class CrawlParser {
                     //设置书籍简介
                     book.setBookDesc(desc);
                     if (StringUtils.isNotBlank(ruleBean.getStatusPatten())) {
-                        Pattern bookStatusPatten = compile(ruleBean.getStatusPatten());
+                        Pattern bookStatusPatten = PatternFactory.getPattern(ruleBean.getStatusPatten());
                         Matcher bookStatusMatch = bookStatusPatten.matcher(bookDetailHtml);
                         boolean isFindBookStatus = bookStatusMatch.find();
                         if (isFindBookStatus) {
@@ -111,7 +113,7 @@ public class CrawlParser {
                     }
 
                     if (StringUtils.isNotBlank(ruleBean.getUpadateTimePatten()) && StringUtils.isNotBlank(ruleBean.getUpadateTimeFormatPatten())) {
-                        Pattern updateTimePatten = compile(ruleBean.getUpadateTimePatten());
+                        Pattern updateTimePatten = PatternFactory.getPattern(ruleBean.getUpadateTimePatten());
                         Matcher updateTimeMatch = updateTimePatten.matcher(bookDetailHtml);
                         boolean isFindUpdateTime = updateTimeMatch.find();
                         if (isFindUpdateTime) {
@@ -154,10 +156,10 @@ public class CrawlParser {
                 indexListHtml = indexListHtml.substring(indexListHtml.indexOf(ruleBean.getBookIndexStart()) + ruleBean.getBookIndexStart().length());
             }
 
-            Pattern indexIdPatten = compile(ruleBean.getIndexIdPatten());
+            Pattern indexIdPatten = PatternFactory.getPattern(ruleBean.getIndexIdPatten());
             Matcher indexIdMatch = indexIdPatten.matcher(indexListHtml);
 
-            Pattern indexNamePatten = compile(ruleBean.getIndexNamePatten());
+            Pattern indexNamePatten = PatternFactory.getPattern(ruleBean.getIndexNamePatten());
             Matcher indexNameMatch = indexNamePatten.matcher(indexListHtml);
 
             boolean isFindIndex = indexIdMatch.find() & indexNameMatch.find();

novel-crawl/src/main/java/com/java2nb/novel/core/crawl/PatternFactory.java

@@ -0,0 +1,30 @@
+package com.java2nb.novel.core.crawl;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+/**
+ * @author xiongxiaoyang
+ */
+public class PatternFactory {
+
+    private static final Map<String, Pattern> PATTERN_CACHED_MAP = new HashMap<>();
+
+    /**
+     * 根据正则表达式获取一个预编译的Pattern对象
+     */
+    public static Pattern getPattern(String regex) {
+        Pattern pattern = PATTERN_CACHED_MAP.get(regex);
+        if (Objects.isNull(pattern)) {
+            pattern = Pattern.compile(regex);
+            PATTERN_CACHED_MAP.put(regex, pattern);
+        }
+        return pattern;
+    }
+
+}
+
+
+

novel-crawl/src/main/java/com/java2nb/novel/service/CrawlService.java

@@ -18,7 +18,11 @@ public interface CrawlService {
      * */
     void addCrawlSource(CrawlSource source);
 
-
+    /**
+     * 修改爬虫源
+     * @param source
+     */
+    void updateCrawlSource(CrawlSource source);
     /**
      * 爬虫源分页列表
      * @param page 当前页码
@@ -106,4 +110,11 @@ public interface CrawlService {
      * @param status 采集状态
      * */
     void updateCrawlSingleTask(CrawlSingleTask task, Byte status);
+
+    /**
+     * 获取采集规则详细
+     * @param id
+     * @return
+     */
+    CrawlSource getCrawlSource(Integer id);
 }

novel-crawl/src/main/java/com/java2nb/novel/service/impl/CrawlServiceImpl.java

@@ -58,9 +58,10 @@ public class CrawlServiceImpl implements CrawlService {
 
     private final BookService bookService;
 
-
     private final CacheService cacheService;
 
+    private final IdWorker idWorker = IdWorker.INSTANCE;
+
 
     @Override
     public void addCrawlSource(CrawlSource source) {
@@ -70,7 +71,24 @@ public class CrawlServiceImpl implements CrawlService {
         crawlSourceMapper.insertSelective(source);
 
     }
-
+    @Override
+    public void updateCrawlSource(CrawlSource source) {
+        if(source.getId()!=null){
+            Optional<CrawlSource> opt=crawlSourceMapper.selectByPrimaryKey(source.getId());
+            if(opt.isPresent()) {
+                CrawlSource crawlSource =opt.get();
+                if (crawlSource.getSourceStatus() == (byte) 1) {
+                    //关闭
+                    openOrCloseCrawl(crawlSource.getId(),(byte)0);
+                }
+                Date currentDate = new Date();
+                crawlSource.setUpdateTime(currentDate);
+                crawlSource.setCrawlRule(source.getCrawlRule());
+                crawlSource.setSourceName(source.getSourceName());
+                crawlSourceMapper.updateByPrimaryKey(crawlSource);
+            }
+        }
+    }
     @Override
     public PageBean<CrawlSource> listCrawlByPage(int page, int pageSize) {
         PageHelper.startPage(page, pageSize);
@@ -206,6 +224,16 @@ public class CrawlServiceImpl implements CrawlService {
 
     }
 
+    @Override
+    public CrawlSource getCrawlSource(Integer id) {
+            Optional<CrawlSource> opt=crawlSourceMapper.selectByPrimaryKey(id);
+            if(opt.isPresent()) {
+                CrawlSource crawlSource =opt.get();
+                return crawlSource;
+            }
+            return null;
+    }
+
     /**
      * 解析分类列表
      */
@@ -300,7 +328,7 @@ public class CrawlServiceImpl implements CrawlService {
                 book.setCrawlBookId(bookId);
                 book.setCrawlSourceId(sourceId);
                 book.setCrawlLastTime(new Date());
-                book.setId(IdWorker.INSTANCE.nextId());
+                book.setId(idWorker.nextId());
                 //解析章节目录
                 boolean parseIndexContentResult = CrawlParser.parseBookIndexAndContent(bookId, book, ruleBean, new HashMap<>(0), chapter -> {
                     bookService.saveBookAndIndexAndContent(book, chapter.getBookIndexList(), chapter.getBookContentList());

novel-crawl/src/main/resources/templates/crawl/crawlSingleTask_add.html

@@ -30,6 +30,7 @@
             <ul class="log_list">
                 <li><a class="link_1" href="/">爬虫源管理</a></li>
                 <li><a class="link_1 on" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
                 <!--<li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
                 <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
             </ul>

novel-crawl/src/main/resources/templates/crawl/crawlSingleTask_list.html

@@ -30,6 +30,7 @@
             <ul class="log_list">
                 <li><a class="link_1" href="/">爬虫源管理</a></li>
                 <li><a class="link_1 on" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
                 <!-- <li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
                  <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
             </ul>

novel-crawl/src/main/resources/templates/crawl/crawlSource_add.html

@@ -30,6 +30,7 @@
             <ul class="log_list">
                 <li><a class="link_1 on" href="/">爬虫源管理</a></li>
                 <li><a class="link_1" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
                 <!--<li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
                 <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
             </ul>

novel-crawl/src/main/resources/templates/crawl/crawlSource_list.html

@@ -29,6 +29,7 @@
             <ul class="log_list">
                 <li><a class="link_1 on" href="/">爬虫源管理</a></li>
                 <li><a class="link_1" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
                <!-- <li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
                 <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
             </ul>
@@ -38,7 +39,7 @@
             <div class="my_bookshelf">
                 <div class="title cf">
                     <h2 class="fl">爬虫源列表</h2>
-                    <div class="fr"><a href="/crawl/crawlSource_add.html" class="btn_red">增加爬虫源</a></div>
+                    <div class="fr"><a href="/crawl/crawlSource_add.html" class="btn_red">增加爬虫源</a>
                 </div>
 
                 <div id="divData" class="updateTable">
@@ -119,6 +120,7 @@
 <script language="javascript" type="text/javascript">
     search(1, 10);
 
+    var pageCrawlSourceList=null;
     function search(curr, limit) {
 
         $.ajax({
@@ -129,6 +131,7 @@
             success: function (data) {
                 if (data.code == 200) {
                     var crawlSourceList = data.data.list;
+                    pageCrawlSourceList=data.data.list;
                     if (crawlSourceList.length > 0) {
                         var crawlSourceListHtml = "";
                         for(var i=0;i<crawlSourceList.length;i++){
@@ -147,7 +150,9 @@
                                 "                            <td class=\"goread\" id='sourceStatus"+crawlSource.id+"'>"+(crawlSource.sourceStatus==0?'停止运行':'正在运行')+
                                 "                            </td>\n" +
 
-                                " <td class=\"goread\" id='opt"+crawlSource.id+"'><a href='javascript:openOrStopCrawl("+crawlSource.id+","+crawlSource.sourceStatus+")'>"+(crawlSource.sourceStatus==0?'开启':'关闭')+"  </a></td>                     </tr>");
+                                " <td class=\"goread\" id='opt"+crawlSource.id+"'><a href='javascript:openOrStopCrawl("+crawlSource.id+","+crawlSource.sourceStatus+")'>"+(crawlSource.sourceStatus==0?'开启':'关闭')+"  </a>" +
+                                "<a href='javascript:updateCrawlSource("+crawlSource.id+")'>修改  </a>" +
+                                "</td>                     </tr>");
                         }
                         $("#crawlSourceList").html(crawlSourceListHtml);
 
@@ -196,7 +201,12 @@
         })
 
     }
+    function updateCrawlSource(crawlSourceId){
 
+        localStorage.setItem("crawlSourceId",crawlSourceId);
+        window.location.href="/crawl/crawlSource_update.html";
+
+    }
 
     function openOrStopCrawl(sourceId,status) {
 

novel-crawl/src/main/resources/templates/crawl/crawlSource_test.html

@@ -0,0 +1,171 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+    <head>
+        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
+        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/>
+        <title>爬虫管理系统-小说精品屋</title>
+        <link rel="stylesheet" href="/css/base.css?v=1"/>
+        <link rel="stylesheet" href="/css/user.css"/>
+    </head>
+</head>
+<body class="">
+
+<div class="header">
+    <div class="mainNav" id="mainNav">
+        <div class="box_center cf"
+             style="text-align: center;height: 44px;line-height: 48px;color: #fff;font-size: 16px;">
+
+            小说精品屋爬虫管理
+
+        </div>
+    </div>
+</div>
+
+<div class="main box_center cf">
+    <div class="userBox cf">
+        <div class="my_l">
+
+            <ul class="log_list">
+                <li><a class="link_1 on" href="/">爬虫源管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
+                <!--<li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
+                <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
+            </ul>
+
+        </div>
+        <div class="my_r">
+            <div class="my_bookshelf">
+
+                <div class="userBox cf">
+                    <form method="post" action="./register.html" id="form2">
+                        <div class="aspNetHidden">
+                            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
+                                   value="/wEPDwUKLTIzNjMxNDQxNw9kFgJmD2QWAmYPFgIeBFRleHQFqAE8YSBocmVmPSIvc2VhcmNoLmFzcHg/c2VhcmNoS2V5PeWWu+Wuiembr++8jOeLhOazve+8jOeBteW8gu+8jOWJjeS4luS7iueUn++8jOWGpeeOi+msvOWkqyIgdGFyZ2V0PSJfYmxhbmsiPuWWu+Wuiembr++8jOeLhOazve+8jOeBteW8gu+8jOWJjeS4luS7iueUn++8jOWGpeeOi+msvOWkqzwvYT5kZOquoASBvnvPbc/TYIQiLhSPJ8GKnYQrmk7jGhb5AC5Q">
+                        </div>
+
+                        <div class="aspNetHidden">
+
+                            <input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="23AA6834">
+                            <input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION"
+                                   value="/wEdAAVece19BIZ9HiByRfHz3pfnqKSXUE1UN51mNFrIuw38c3Y2+Mc6SrnAqio3oCKbxYZZ1lS+gZUZKpbsAea8j7ASAv40DHFcQ/NE7tJUnABeyQ3d9sFDIcFCYNqlVtprfLoh4JFy0U+R/CcMuyAiWTz7">
+                        </div>
+                        <div class="user_l">
+                            <div></div>
+                            <h3>爬虫源信息填写（示例均为顶点小说网：dingdiann.com）</h3>
+                            <ul class="log_list">
+                                <li><span id="LabErr"></span></li>
+                                示例：<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID，{page}代表分页页码)
+                                <li><input type="text" id="url" class="s_input icon_key"
+                                           placeholder="url"></li>
+
+                                示例：<b>value=\"(\\d+)/\\d+\"</b>
+                                <li><input type="text" id="rule" class="s_input icon_name" placeholder="规则"></li>
+                                示例：<b>1强制刷新 空或0使用缓存</b>
+                                <li><input type="text" id="isRefresh" class="s_input icon_name" placeholder="是否强制刷新"></li>
+
+                                <li><textarea rows="20" cols="100" id="resultMap"></textarea></li>
+
+
+
+                                <li><input type="button" onclick="testCrawlSource()" name="btnRegister" value="测试"
+                                           id="btnRegister" class="btn_red"></li>
+                            </ul>
+                        </div>
+                    </form>
+                </div>
+                <!--<div id="divData" class="updateTable">
+                    <table cellpadding="0" cellspacing="0">
+                        <thead>
+                        <tr>
+
+                            <th class="name">
+                                爬虫源（已开启的爬虫源）
+                            </th>
+                            <th class="chapter">
+                                成功爬取数量（websocket实现）
+                            </th>
+                            <th class="time">
+                            目标爬取数量
+                            </th>
+                            <th class="goread">
+                                状态（正在运行，已停止）（一次只能运行一个爬虫源）
+                            </th>
+                            <th class="goread">
+                                操作（启动，停止）
+                            </th>
+                        </tr>
+                        </thead>
+                        <tbody id="bookShelfList">
+
+
+
+                        </tbody>
+                    </table>
+                    <div class="pageBox cf" id="shellPage">
+                    </div>
+                </div>-->
+            </div>
+        </div>
+    </div>
+</div>
+
+
+</body>
+<script src="/javascript/jquery-1.8.0.min.js" type="text/javascript"></script>
+<script src="/layui/layui.all.js" type="text/javascript"></script>
+<script src="/javascript/header.js" type="text/javascript"></script>
+<script src="/javascript/user.js" type="text/javascript"></script>
+<script language="javascript" type="text/javascript">
+    function load() {
+       var testParseUrl= localStorage.getItem("testParseUrl");
+       $("#url").val(testParseUrl);
+        var testParseRule=localStorage.getItem("testParseRule");
+        $("#rule").val(testParseRule);
+    }
+    function testCrawlSource() {
+
+        var data = {};
+
+
+        var isRefresh = $("#isRefresh").val();
+        data.isRefresh = isRefresh;
+        var rule = $("#rule").val();
+        if (rule.length == 0) {
+            layer.alert("正则必填");
+            return false;
+        }
+        data.rule = rule;
+        var url = $("#url").val();
+        if (url.length == 0) {
+            layer.alert("url必填");
+            return false;
+        }
+        data.url = url;
+
+        localStorage.setItem("testParseUrl",url);
+        localStorage.setItem("testParseRule",rule);
+        $.ajax({
+            type: "POST",
+            url: "/crawl/testParse",
+            data: data,
+            dataType: "json",
+            success: function (data) {
+                if (data.code == 200) {
+                    $("#resultMap").val(JSON.stringify(data.data));
+                } else {
+                    layer.alert(data.msg);
+                }
+
+            },
+            error: function () {
+                layer.alert('网络异常');
+            }
+        })
+
+    }
+
+</script>
+</html>

novel-crawl/src/main/resources/templates/crawl/crawlSource_update.html

@@ -0,0 +1,522 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+    <head>
+        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
+        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/>
+        <title>爬虫管理系统-小说精品屋</title>
+        <link rel="stylesheet" href="/css/base.css?v=1"/>
+        <link rel="stylesheet" href="/css/user.css"/>
+    </head>
+</head>
+<body class="">
+
+<div class="header">
+    <div class="mainNav" id="mainNav">
+        <div class="box_center cf"
+             style="text-align: center;height: 44px;line-height: 48px;color: #fff;font-size: 16px;">
+
+            小说精品屋爬虫管理
+
+        </div>
+    </div>
+</div>
+
+<div class="main box_center cf">
+    <div class="userBox cf">
+        <div class="my_l">
+
+            <ul class="log_list">
+                <li><a class="link_1 on" href="/">爬虫源管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSingleTask_list.html">单本采集管理</a></li>
+                <li><a class="link_1" href="/crawl/crawlSource_test.html" target="_blank" >规则测试</a></li>
+                <!--<li><a class="link_1 " href="/user/userinfo.html">批量小说爬取</a></li>
+                <li><a class="link_4 " href="/user/favorites.html">单本小说爬取</a></li>-->
+            </ul>
+
+        </div>
+        <div class="my_r">
+            <div class="my_bookshelf">
+
+                <div class="userBox cf">
+                    <form method="post" action="./register.html" id="form2">
+                        <input type="hidden" name="id" id="sourceId"/>
+                        <div class="aspNetHidden">
+                            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
+                                   value="/wEPDwUKLTIzNjMxNDQxNw9kFgJmD2QWAmYPFgIeBFRleHQFqAE8YSBocmVmPSIvc2VhcmNoLmFzcHg/c2VhcmNoS2V5PeWWu+Wuiembr++8jOeLhOazve+8jOeBteW8gu+8jOWJjeS4luS7iueUn++8jOWGpeeOi+msvOWkqyIgdGFyZ2V0PSJfYmxhbmsiPuWWu+Wuiembr++8jOeLhOazve+8jOeBteW8gu+8jOWJjeS4luS7iueUn++8jOWGpeeOi+msvOWkqzwvYT5kZOquoASBvnvPbc/TYIQiLhSPJ8GKnYQrmk7jGhb5AC5Q">
+                        </div>
+
+                        <div class="aspNetHidden">
+
+                            <input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="23AA6834">
+                            <input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION"
+                                   value="/wEdAAVece19BIZ9HiByRfHz3pfnqKSXUE1UN51mNFrIuw38c3Y2+Mc6SrnAqio3oCKbxYZZ1lS+gZUZKpbsAea8j7ASAv40DHFcQ/NE7tJUnABeyQ3d9sFDIcFCYNqlVtprfLoh4JFy0U+R/CcMuyAiWTz7">
+                        </div>
+                        <div class="user_l">
+                            <div></div>
+                            <h3>爬虫源信息填写（示例均为顶点小说网：dingdiann.com）</h3>
+                            <ul class="log_list">
+                                <li><span id="LabErr"></span></li>
+                                示例：<b>新顶点小说网</b>
+                                <li><input type="text" id="sourceName" class="s_input icon_name" placeholder="源站名"></li>
+                                <!--示例：<b>https://m.xdingdiann.com/sort/0/1.html</b>
+                                <li><input type="text" id="updateBookListUrl" class="s_input icon_key"
+                                           placeholder="小说更新列表url"></li>-->
+                                示例：<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID，{page}代表分页页码)
+                                <li><input type="text" id="bookListUrl" class="s_input icon_key"
+                                           placeholder="分类列表页URL规则"></li>
+                                示例：<b>1</b>
+                                <li><input type="text" id="catId1" class="s_input icon_key" placeholder="玄幻奇幻分类ID"></li>
+                                示例：<b>2</b>
+                                <li><input type="text" id="catId2" class="s_input icon_key" placeholder="武侠仙侠分类ID"></li>
+                                示例：<b>3</b>
+                                <li><input type="text" id="catId3" class="s_input icon_key" placeholder="都市言情分类ID"></li>
+                                示例：<b>4</b>
+                                <li><input type="text" id="catId4" class="s_input icon_key" placeholder="历史军事分类ID"></li>
+                                示例：<b>5</b>
+                                <li><input type="text" id="catId5" class="s_input icon_key" placeholder="科幻灵异分类ID"></li>
+                                示例：<b>6</b>
+                                <li><input type="text" id="catId6" class="s_input icon_key" placeholder="网游竞技分类ID"></li>
+                                示例：<b>7</b>
+                                <li><input type="text" id="catId7" class="s_input icon_key" placeholder="女生频道分类ID"></li>
+                                示例：<b>href="/ddk(\d+)/"</b>
+                                <li><input type="text" id="bookIdPatten" class="s_input icon_key"
+                                           placeholder="列表页小说ID正则表达式："></li>
+                                <b>value="(\d+)/\d+"</b>
+                                <li><input type="text" id="pagePatten" class="s_input icon_key"
+                                           placeholder="列表页当前分页页码正则表达式："></li>
+                                <b>value="\d+/(\d+)"</b>
+                                <li><input type="text" id="totalPagePatten" class="s_input icon_key"
+                                           placeholder="列表页分页总页数正则表达式："></li>
+                                <b>http://m.xdingdiann.com/ddk{bookId}</b> (bookId代表小说ID)
+                                <li><input type="text" id="bookDetailUrl" class="s_input icon_key"
+                                           placeholder="详情页URL规则："></li>
+                                示例：<b>&lt;p class="title"&gt;([^/]+)&lt;/p&gt;</b>
+                                <li><input type="text" id="bookNamePatten" class="s_input icon_key"
+                                           placeholder="小说名的正则表达式："></li>
+                                示例：<b>作者：([^/]+)<</b>
+                                <li><input type="text" id="authorNamePatten" class="s_input icon_key"
+                                           placeholder="小说作者的正则表达式："></li>
+                                示例：<b>&lt;img src="([^>]+)"\s+onerror="this.src=</b>
+                                <li><input type="text" id="picUrlPatten" class="s_input icon_key"
+                                           placeholder="小说图片路径的正则表达式："></li>
+                                <b>可空，适用于图片路径为相对路径的源站，加上小说图片路径，则为完整的可访问的图片路径</b>
+                                <li><input type="text" id="picUrlPrefix" class="s_input icon_key"
+                                           placeholder="小说图片访问路径前缀："></li>
+                                示例：<b>状态：([^/]+)&lt;/li&gt;</b>
+                                <li><input type="text" id="statusPatten" class="s_input icon_key"
+                                           placeholder="小说状态的正则表达式："></li>
+                                示例：<b>连载</b>
+                                <li><input type="text" id="bookStatus0" class="s_input icon_key"
+                                           placeholder="连载中的小说在此网站的具体表现值："></li>
+                                示例：<b>完结</b>
+                                <li><input type="text" id="bookStatus1" class="s_input icon_key"
+                                           placeholder="全本小说在此网站的具体表现值："></li>
+                                示例：<b>&lt;div\s+class="score"&gt;(\d+\.\d+)分&lt;/div&gt;</b>
+                                <li><input type="text" id="scorePatten" class="s_input icon_key"
+                                           placeholder="小说评分的正则表达式："></li>
+                                示例：<b></b>
+                                <li><input type="text" id="visitCountPatten" class="s_input icon_key"
+                                           placeholder="小说点击量的正则表达式："></li>
+                                示例：<b>&lt;p class="review"&gt;</b>
+                                <li><input type="text" id="descStart" class="s_input icon_key"
+                                           placeholder="小说简介开始截取字符串："></li>
+                                示例：<b>&lt;/p&gt;</b>
+                                <li><input type="text" id="descEnd" class="s_input icon_key" placeholder="小说简介结束截取字符串：">
+                                </li>
+                                示例：<b>更新：(\d+-\d+-\d+\s\d+:\d+:\d+)&lt;/a&gt;</b>
+                                <li><input type="text" id="upadateTimePatten" class="s_input icon_key"
+                                           placeholder="小说更新时间的正则表达式："></li>
+                                示例：<b>yyyy-MM-dd HH:mm:ss</b>
+                                <li><input type="text" id="upadateTimeFormatPatten" class="s_input icon_key"
+                                           placeholder="小说更新时间在此网站的显示模式："></li>
+                                示例：<b>http://m.xdingdiann.com/ddk{bookId}/all.html</b> (bookId代表小说ID)
+                                <li><input type="text" id="bookIndexUrl" class="s_input icon_key"
+                                           placeholder="小说目录页的URL规则："></li>
+                                <b>可空，适用于最新章节列表和全部章节列表在同一个页面的源站</b>
+                                <li><input type="text" id="bookIndexStart" class="s_input icon_key"
+                                           placeholder="小说目录页内容开始截取字符串："></li>
+                                示例：<b>&lt;a\s+style=""\s+href="/ddk\d+/(\d+)\.html"&gt;[^/]+&lt;/a&gt;</b>
+                                <li><input type="text" id="indexIdPatten" class="s_input icon_key"
+                                           placeholder="目录页目录ID正则表达式："></li>
+                                示例：<b>&lt;a\s+style=""\s+href="/ddk\d+/\d+\.html"&gt;([^/]+)&lt;/a&gt;</b>
+                                <li><input type="text" id="indexNamePatten" class="s_input icon_key"
+                                           placeholder="目录页目录名的正则表达式："></li>
+                                示例：<b>http://m.xdingdiann.com/ddk{bookId}/{indexId}.html</b>
+                                (bookId代表小说ID,{indexId}代表目录ID)
+                                <li><input type="text" id="bookContentUrl" class="s_input icon_key"
+                                           placeholder="小说内容页的URL规则："></li>
+                                示例：<b>id="content"></b>
+                                <li><input type="text" id="contentStart" class="s_input icon_key"
+                                           placeholder="小说内容开始截取字符串："></li>
+                                示例：<b>&lt;script&gt;</b>
+                                <li><input type="text" id="contentEnd" class="s_input icon_key"
+                                           placeholder="小说内容结束截取字符串："></li>
+
+                                <li><input type="button" onclick="updateCrawlSource()" name="btnRegister" value="提交"
+                                           id="btnRegister" class="btn_red"></li>
+                            </ul>
+                        </div>
+                    </form>
+                </div>
+                <!--<div id="divData" class="updateTable">
+                    <table cellpadding="0" cellspacing="0">
+                        <thead>
+                        <tr>
+
+                            <th class="name">
+                                爬虫源（已开启的爬虫源）
+                            </th>
+                            <th class="chapter">
+                                成功爬取数量（websocket实现）
+                            </th>
+                            <th class="time">
+                            目标爬取数量
+                            </th>
+                            <th class="goread">
+                                状态（正在运行，已停止）（一次只能运行一个爬虫源）
+                            </th>
+                            <th class="goread">
+                                操作（启动，停止）
+                            </th>
+                        </tr>
+                        </thead>
+                        <tbody id="bookShelfList">
+
+
+
+                        </tbody>
+                    </table>
+                    <div class="pageBox cf" id="shellPage">
+                    </div>
+                </div>-->
+            </div>
+        </div>
+    </div>
+</div>
+
+
+</body>
+<script src="/javascript/jquery-1.8.0.min.js" type="text/javascript"></script>
+<script src="/layui/layui.all.js" type="text/javascript"></script>
+<script src="/javascript/header.js" type="text/javascript"></script>
+<script src="/javascript/user.js" type="text/javascript"></script>
+<script language="javascript" type="text/javascript">
+
+    function load(){
+        var crawlSourceId = localStorage.getItem("crawlSourceId")
+        if(crawlSourceId!=null){
+            $.ajax({
+                type: "GET",
+                url: "/crawl/getCrawlSource/"+crawlSourceId,
+                dataType: "json",
+                success: function (data) {
+                    if (data.code == 200) {
+                        loadPage(data.data);
+                    } else if (data.code == 1001) {
+                        //未登录
+                        location.href = '/user/login.html?originUrl=' + decodeURIComponent(location.href);
+
+                    }else {
+                        layer.alert(data.msg);
+                    }
+
+                },
+                error: function () {
+                    layer.alert('网络异常');
+                }
+            })
+        }
+    }
+    function loadPage(data){
+        $("#sourceId").val(data.id);
+        $("#sourceName").val(data.sourceName);
+        if(data.crawlRule){
+
+
+            var crawlRule= JSON.parse(data.crawlRule);
+            $("#bookListUrl").val(crawlRule.bookListUrl);
+            var catIdRule = crawlRule.catIdRule;
+            try{
+                for (var i = 1; i <= 7; i++) {
+                    $("#catId" + i).val(catIdRule["catId" + i]);
+                }
+            }catch(e){
+            }
+            $("#bookIdPatten").val(crawlRule.bookIdPatten);
+            $("#pagePatten").val(crawlRule.pagePatten);
+            $("#totalPagePatten").val(crawlRule.totalPagePatten);
+            $("#bookDetailUrl").val(crawlRule.bookDetailUrl);
+            $("#bookNamePatten").val(crawlRule.bookNamePatten);
+            $("#authorNamePatten").val(crawlRule.authorNamePatten);
+            $("#picUrlPatten").val(crawlRule.picUrlPatten);
+            $("#picUrlPrefix").val(crawlRule.picUrlPrefix);
+            $("#statusPatten").val(crawlRule.statusPatten);
+            try{
+                var bookStatusRule = crawlRule.bookStatusRule;
+                var i=0;
+                for(var key in bookStatusRule){
+                    $("#bookStatus" + i).val(key);
+                    i++;
+                }
+            }catch (e) {
+
+            }
+            $("#scorePatten").val(crawlRule.scorePatten);
+            $("#visitCountPatten").val(crawlRule.visitCountPatten);
+            $("#descStart").val(crawlRule.descStart);
+            $("#descEnd").val(crawlRule.descEnd);
+            $("#upadateTimePatten").val(crawlRule.upadateTimePatten);
+            $("#upadateTimeFormatPatten").val(crawlRule.upadateTimeFormatPatten);
+            $("#bookIndexUrl").val(crawlRule.bookIndexUrl);
+            $("#bookIndexStart").val(crawlRule.bookIndexStart);
+            $("#indexIdPatten").val(crawlRule.indexIdPatten);
+            $("#indexNamePatten").val(crawlRule.indexNamePatten);
+            $("#bookContentUrl").val(crawlRule.bookContentUrl);
+            $("#contentStart").val(crawlRule.contentStart);
+            $("#contentEnd").val(crawlRule.contentEnd);
+
+        }
+    }
+    load();
+
+    function updateCrawlSource() {
+
+        var crawlRule = {};
+        var sourceId =$("#sourceId").val();
+
+        var sourceName = $("#sourceName").val();
+        if (sourceName.length == 0) {
+            layer.alert("源站名必填");
+            return false;
+        }
+
+        var bookListUrl = $("#bookListUrl").val();
+        if (bookListUrl.length == 0) {
+            layer.alert("分类列表页URL规则必填");
+            return false;
+        }
+        crawlRule.bookListUrl = bookListUrl;
+
+        var catIdRule = {};
+
+        for (var i = 1; i <= 7; i++) {
+            var catId = $("#catId" + i).val();
+            if (catId.length > 0) {
+                catIdRule["catId" + i] = catId;
+            }
+        }
+
+        if (Object.keys(catIdRule).length == 0) {
+            layer.alert("分类ID至少要填一项");
+            return false;
+        }
+        crawlRule.catIdRule = catIdRule;
+
+
+        var bookIdPatten = $("#bookIdPatten").val();
+
+
+        if (bookIdPatten.length == 0) {
+            layer.alert("列表页小说ID正则表达式必填");
+            return false;
+        }
+
+        crawlRule.bookIdPatten = bookIdPatten;
+
+        var pagePatten = $("#pagePatten").val();
+
+        if (pagePatten.length > 0) {
+            crawlRule.pagePatten = pagePatten;
+        }
+
+        var totalPagePatten = $("#totalPagePatten").val();
+
+        if (totalPagePatten.length > 0) {
+            crawlRule.totalPagePatten = totalPagePatten;
+        }
+
+        var bookDetailUrl = $("#bookDetailUrl").val();
+
+        if (bookDetailUrl.length == 0) {
+            layer.alert("详情页URL规则必填");
+            return false;
+        }
+
+        crawlRule.bookDetailUrl = bookDetailUrl;
+
+        var bookNamePatten = $("#bookNamePatten").val();
+
+        if (bookNamePatten.length == 0) {
+            layer.alert("小说名的正则表达式必填");
+            return false;
+        }
+
+        crawlRule.bookNamePatten = bookNamePatten;
+
+        var authorNamePatten = $("#authorNamePatten").val();
+
+        if (authorNamePatten.length == 0) {
+            layer.alert("小说作者的正则表达式必填");
+            return false;
+        }
+
+        crawlRule.authorNamePatten = authorNamePatten;
+
+        var picUrlPatten = $("#picUrlPatten").val();
+
+        if (picUrlPatten.length > 0) {
+            crawlRule.picUrlPatten = picUrlPatten;
+        }
+
+        var picUrlPrefix = $("#picUrlPrefix").val();
+
+        if (picUrlPrefix.length > 0) {
+            crawlRule.picUrlPrefix = picUrlPrefix;
+        }
+
+        var statusPatten = $("#statusPatten").val();
+        if (statusPatten.length > 0) {
+            crawlRule.statusPatten = statusPatten;
+        }
+
+        var bookStatusRule = {};
+
+        for (var i = 0; i <= 1; i++) {
+            var bookStatus = $("#bookStatus" + i).val();
+            if (bookStatus.length > 0) {
+                bookStatusRule[bookStatus] = i;
+            }
+        }
+
+        crawlRule.bookStatusRule = bookStatusRule;
+
+        var scorePatten = $("#scorePatten").val();
+
+        if (scorePatten.length > 0) {
+            crawlRule.scorePatten = scorePatten;
+        }
+
+        var visitCountPatten = $("#visitCountPatten").val();
+
+        if (visitCountPatten.length > 0) {
+            crawlRule.visitCountPatten = visitCountPatten;
+        }
+
+        var descStart = $("#descStart").val();
+
+        if (descStart.length == 0) {
+            layer.alert("小说简介开始截取字符串必填");
+            return false;
+        }
+
+        crawlRule.descStart = descStart;
+
+        var descEnd = $("#descEnd").val();
+
+        if (descEnd.length == 0) {
+            layer.alert("小说简介结束截取字符串必填");
+            return false;
+        }
+
+        crawlRule.descEnd = descEnd;
+
+        var upadateTimePatten = $("#upadateTimePatten").val();
+
+        if (upadateTimePatten.length > 0) {
+            crawlRule.upadateTimePatten = upadateTimePatten;
+        }
+
+        var upadateTimeFormatPatten = $("#upadateTimeFormatPatten").val();
+
+        if (upadateTimeFormatPatten.length > 0) {
+            crawlRule.upadateTimeFormatPatten = upadateTimeFormatPatten;
+        }
+
+        var bookIndexUrl = $("#bookIndexUrl").val();
+
+        if (bookIndexUrl.length == 0) {
+            layer.alert("小说目录页的URL规则必填");
+            return false;
+        }
+
+        crawlRule.bookIndexUrl = bookIndexUrl;
+
+
+        var bookIndexStart = $("#bookIndexStart").val();
+
+        if (bookIndexStart.length > 0) {
+            crawlRule.bookIndexStart = bookIndexStart;
+        }
+
+        var indexIdPatten = $("#indexIdPatten").val();
+
+        if (indexIdPatten.length == 0) {
+            layer.alert("小说目录页的目录ID正则表达式必填");
+            return false;
+        }
+
+        crawlRule.indexIdPatten = indexIdPatten;
+
+        var indexNamePatten = $("#indexNamePatten").val();
+
+        if (indexNamePatten.length == 0) {
+            layer.alert("小说目录页的目录名正则表达式必填");
+            return false;
+        }
+
+        crawlRule.indexNamePatten = indexNamePatten;
+
+        var bookContentUrl = $("#bookContentUrl").val();
+
+        if (bookContentUrl.length == 0) {
+            layer.alert("小说内容页的URL规则必填");
+            return false;
+        }
+
+        crawlRule.bookContentUrl = bookContentUrl;
+
+        var contentStart = $("#contentStart").val();
+
+        if (contentStart.length == 0) {
+            layer.alert("小说内容开始截取字符串必填");
+            return false;
+        }
+        crawlRule.contentStart = contentStart;
+
+        var contentEnd = $("#contentEnd").val();
+
+        if (contentEnd.length == 0) {
+            layer.alert("小说内容结束截取字符串必填");
+            return false;
+        }
+
+        crawlRule.contentEnd = contentEnd;
+
+
+        $.ajax({
+            type: "POST",
+            url: "/crawl/updateCrawlSource",
+            data: {'id':sourceId,'sourceName': sourceName, 'crawlRule': JSON.stringify(crawlRule)},
+            dataType: "json",
+            success: function (data) {
+                if (data.code == 200) {
+
+                    window.location.href = '/crawl/crawlSource_list.html';
+
+
+                } else {
+                    layer.alert(data.msg);
+                }
+
+            },
+            error: function () {
+                layer.alert('网络异常');
+            }
+        })
+
+    }
+
+</script>
+</html>

novel-front/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>3.6.1</version>
+        <version>3.6.2</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -70,14 +70,12 @@
         </dependency>
 
 
-
-
         <!--引入redisson分布式锁-->
-       <!-- <dependency>
-            <groupId>org.redisson</groupId>
-            <artifactId>redisson-spring-boot-starter</artifactId>
-            <version>${redisson.version}</version>
-        </dependency>-->
+        <!-- <dependency>
+             <groupId>org.redisson</groupId>
+             <artifactId>redisson-spring-boot-starter</artifactId>
+             <version>${redisson.version}</version>
+         </dependency>-->
 
 
         <dependency>
@@ -87,74 +85,11 @@
         </dependency>
     </dependencies>
 
-<!--    <build>-->
-<!--        <plugins>-->
-<!--            <plugin>-->
-<!--                <groupId>org.springframework.boot</groupId>-->
-<!--                <artifactId>spring-boot-maven-plugin</artifactId>-->
-<!--            </plugin>-->
-<!--        </plugins>-->
-<!--    </build>-->
     <build>
         <plugins>
-            <!-- 将相同groupId的依赖模块打包进来 -->
             <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-shade-plugin</artifactId>
-                <configuration>
-                    <createDependencyReducedPom>false</createDependencyReducedPom>
-                </configuration>
-                <executions>
-                    <execution>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>shade</goal>
-                        </goals>
-                        <configuration>
-                            <artifactSet>
-                                <includes>
-                                    <include>${project.groupId}:*:*</include>
-                                </includes>
-                            </artifactSet>
-                        </configuration>
-                    </execution>
-                </executions>
-            </plugin>
-            <plugin>
-                <!--打包时去除第三方依赖-->
                 <groupId>org.springframework.boot</groupId>
                 <artifactId>spring-boot-maven-plugin</artifactId>
-                <configuration>
-                    <layout>ZIP</layout>
-                    <includes>
-                        <include>
-                            <groupId>non-exists</groupId>
-                            <artifactId>non-exists</artifactId>
-                        </include>
-                    </includes>
-                </configuration>
-            </plugin>
-            <!--拷贝第三方依赖文件到指定目录-->
-            <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-dependency-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <id>copy-dependencies</id>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>copy-dependencies</goal>
-                        </goals>
-                        <configuration>
-                            <!--target/lib是依赖jar包的输出目录，根据自己喜好配置-->
-                            <outputDirectory>target/lib</outputDirectory>
-                            <excludeTransitive>false</excludeTransitive>
-                            <stripVersion>false</stripVersion>
-                            <includeScope>runtime</includeScope>
-                            <excludeGroupIds>${project.groupId}</excludeGroupIds>
-                        </configuration>
-                    </execution>
-                </executions>
             </plugin>
             <plugin>
                 <artifactId>maven-antrun-plugin</artifactId>
@@ -168,28 +103,33 @@
                         <configuration>
                             <tasks>
                                 <!-- 文件夹 -->
-                                <copy todir="${project.build.directory}/build/conf" overwrite="true">
+                                <copy todir="${project.build.directory}/build/config" overwrite="true">
+                                    <fileset dir="${basedir}/src/main/build/config">
+                                        <include name="*.*"/>
+                                    </fileset>
                                     <fileset dir="${basedir}/src/main/resources">
-                                        <include name="**/*.*"/>
-                                        <exclude name="mybatis/*/*.*"/>
+                                        <include name="application-alipay.yml"/>
+                                        <include name="application-website.yml"/>
+                                    </fileset>
+                                </copy>
+                                <copy todir="${project.build.directory}/build/templates" overwrite="true">
+                                    <fileset dir="${basedir}/../templates">
+                                        <include name="**/*.*"/>
                                     </fileset>
                                 </copy>
-                                <move todir="${project.build.directory}/build/lib">
-                                    <fileset dir="target/lib"/>
-                                </move>
                                 <copy file="${project.build.directory}/${project.artifactId}-${project.version}.jar"
-                                      tofile="${project.build.directory}/build/${project.artifactId}.jar" />
+                                      tofile="${project.build.directory}/build/${project.artifactId}.jar"/>
 
                                 <fixcrlf srcdir="${basedir}/src/main/build/scripts" eol="unix"/>
                                 <copy todir="${project.build.directory}/build/bin">
                                     <fileset dir="${basedir}/src/main/build/scripts">
-                                        <include name="*.sh" />
-                                        <include name="*.txt" />
-                                        <include name="*.bat" />
+                                        <include name="*.sh"/>
+                                        <include name="*.txt"/>
+                                        <include name="*.bat"/>
                                     </fileset>
                                 </copy>
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
-                                    <zipfileset filemode="755" dir= '${project.build.directory}/build/' />
+                                    <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
                             </tasks>
                         </configuration>
@@ -198,5 +138,112 @@
             </plugin>
         </plugins>
     </build>
+    <!--    <build>-->
+    <!--        <plugins>-->
+    <!--            &lt;!&ndash; 将相同groupId的依赖模块打包进来 &ndash;&gt;-->
+    <!--            <plugin>-->
+    <!--                <groupId>org.apache.maven.plugins</groupId>-->
+    <!--                <artifactId>maven-shade-plugin</artifactId>-->
+    <!--                <configuration>-->
+    <!--                    <createDependencyReducedPom>false</createDependencyReducedPom>-->
+    <!--                </configuration>-->
+    <!--                <executions>-->
+    <!--                    <execution>-->
+    <!--                        <phase>package</phase>-->
+    <!--                        <goals>-->
+    <!--                            <goal>shade</goal>-->
+    <!--                        </goals>-->
+    <!--                        <configuration>-->
+    <!--                            <artifactSet>-->
+    <!--                                <includes>-->
+    <!--                                    <include>${project.groupId}:*:*</include>-->
+    <!--                                </includes>-->
+    <!--                            </artifactSet>-->
+    <!--                        </configuration>-->
+    <!--                    </execution>-->
+    <!--                </executions>-->
+    <!--            </plugin>-->
+    <!--            <plugin>-->
+    <!--                &lt;!&ndash;打包时去除第三方依赖&ndash;&gt;-->
+    <!--                <groupId>org.springframework.boot</groupId>-->
+    <!--                <artifactId>spring-boot-maven-plugin</artifactId>-->
+    <!--                <configuration>-->
+    <!--                    <layout>ZIP</layout>-->
+    <!--                    <includes>-->
+    <!--                        <include>-->
+    <!--                            <groupId>non-exists</groupId>-->
+    <!--                            <artifactId>non-exists</artifactId>-->
+    <!--                        </include>-->
+    <!--                    </includes>-->
+    <!--                </configuration>-->
+    <!--            </plugin>-->
+    <!--            &lt;!&ndash;拷贝第三方依赖文件到指定目录&ndash;&gt;-->
+    <!--            <plugin>-->
+    <!--                <groupId>org.apache.maven.plugins</groupId>-->
+    <!--                <artifactId>maven-dependency-plugin</artifactId>-->
+    <!--                <executions>-->
+    <!--                    <execution>-->
+    <!--                        <id>copy-dependencies</id>-->
+    <!--                        <phase>package</phase>-->
+    <!--                        <goals>-->
+    <!--                            <goal>copy-dependencies</goal>-->
+    <!--                        </goals>-->
+    <!--                        <configuration>-->
+    <!--                            &lt;!&ndash;target/lib是依赖jar包的输出目录，根据自己喜好配置&ndash;&gt;-->
+    <!--                            <outputDirectory>target/lib</outputDirectory>-->
+    <!--                            <excludeTransitive>false</excludeTransitive>-->
+    <!--                            <stripVersion>false</stripVersion>-->
+    <!--                            <includeScope>runtime</includeScope>-->
+    <!--                            <excludeGroupIds>${project.groupId}</excludeGroupIds>-->
+    <!--                        </configuration>-->
+    <!--                    </execution>-->
+    <!--                </executions>-->
+    <!--            </plugin>-->
+    <!--            <plugin>-->
+    <!--                <artifactId>maven-antrun-plugin</artifactId>-->
+    <!--                <version>1.8</version>-->
+    <!--                <executions>-->
+    <!--                    <execution>-->
+    <!--                        <phase>package</phase>-->
+    <!--                        <goals>-->
+    <!--                            <goal>run</goal>-->
+    <!--                        </goals>-->
+    <!--                        <configuration>-->
+    <!--                            <tasks>-->
+    <!--                                &lt;!&ndash; 文件夹 &ndash;&gt;-->
+    <!--                                <copy todir="${project.build.directory}/build/conf" overwrite="true">-->
+    <!--                                    <fileset dir="${basedir}/src/main/resources">-->
+    <!--                                        <include name="**/*.*"/>-->
+    <!--                                        <exclude name="mybatis/*/*.*"/>-->
+    <!--                                    </fileset>-->
+    <!--                                    <fileset dir="${basedir}/../novel-common/src/main/resources">-->
+    <!--                                        <include name="**/*.*"/>-->
+    <!--                                        <exclude name="mybatis/**"/>-->
+    <!--                                    </fileset>-->
+    <!--                                </copy>-->
+    <!--                                <move todir="${project.build.directory}/build/lib">-->
+    <!--                                    <fileset dir="target/lib"/>-->
+    <!--                                </move>-->
+    <!--                                <copy file="${project.build.directory}/${project.artifactId}-${project.version}.jar"-->
+    <!--                                      tofile="${project.build.directory}/build/${project.artifactId}.jar" />-->
+
+    <!--                                <fixcrlf srcdir="${basedir}/src/main/build/scripts" eol="unix"/>-->
+    <!--                                <copy todir="${project.build.directory}/build/bin">-->
+    <!--                                    <fileset dir="${basedir}/src/main/build/scripts">-->
+    <!--                                        <include name="*.sh" />-->
+    <!--                                        <include name="*.txt" />-->
+    <!--                                        <include name="*.bat" />-->
+    <!--                                    </fileset>-->
+    <!--                                </copy>-->
+    <!--                                <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>-->
+    <!--                                    <zipfileset filemode="755" dir= '${project.build.directory}/build/' />-->
+    <!--                                </zip>-->
+    <!--                            </tasks>-->
+    <!--                        </configuration>-->
+    <!--                    </execution>-->
+    <!--                </executions>-->
+    <!--            </plugin>-->
+    <!--        </plugins>-->
+    <!--    </build>-->
 
 </project>

novel-front/src/main/build/config/application-common-prod.yml

@@ -0,0 +1,53 @@
+#端口号
+server:
+  port: 8085
+
+#不分表的数据库配置
+spring:
+  datasource:
+    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: root
+
+
+####使用shardingJdbc时，
+####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
+##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
+sharding:
+  jdbc:
+    datasource:
+      ds0:
+        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+        username: root
+        password: test123456
+
+pic:
+  save:
+    #图片保存方式, 1不保存，使用网络图片 ，2本地保存
+    type: 2
+    #图片保存路径
+    path: /var/pic
+
+
+#模版配置
+templates:
+  name: orange
+
+#小说内容保存配置
+content:
+  save:
+    storage: db #存储介质，db：数据库，file：txt文本
+    path: /Users/xiongxiaoyang/books  #txt小说文本保存路径
+
+
+
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: u493.kdltps.com
+    # 代理端口号
+    port: 15818
+

novel-front/src/main/build/scripts/novel-front.sh

@@ -0,0 +1,94 @@
+#!/bin/sh
+APP_NAME=novel-front
+JAR_NAME=$APP_NAME\.jar
+#PID  代表是PID文件
+PID=$APP_NAME\.pid
+
+
+#使用说明，用来提示输入参数
+usage() {
+    echo "Usage: ./novel-front.sh [start|stop|restart|status]"
+    exit 1
+}
+
+#检查程序是否在运行
+is_exist(){
+  pid=`ps -ef|grep $JAR_NAME|grep -v grep|awk '{print $2}' `
+  #如果不存在返回1，存在返回0
+  if [ -z "${pid}" ]; then
+   return 1
+  else
+    return 0
+  fi
+}
+
+#启动方法
+start(){
+  is_exist
+  if [ $? -eq "0" ]; then
+    echo ">>> 小说精品屋前台正在运行 PID = ${pid} <<<"
+  else
+    echo ">>> 小说精品屋前台开始启动 <<<"
+    nohup java -jar -Dspring.profiles.active=prod $JAR_NAME >/dev/null 2>&1 &
+    sleep 20
+    echo $! > $PID
+    echo ">>> 小说精品屋前台启动完成 PID = $! <<<"
+    status
+   fi
+  }
+
+#停止方法
+stop(){
+  #is_exist
+  pidf=$(cat $PID)
+  #echo "$pidf"
+  echo ">>> 小说精品屋前台 PID = $pidf 开始停止 <<<"
+  kill $pidf
+  rm -rf $PID
+  sleep 2
+  is_exist
+  if [ $? -eq "0" ]; then
+    echo ">>> 小说精品屋前台 PID = $pid 开始强制停止 <<<"
+    kill -9  $pid
+    sleep 2
+    status
+  else
+    status
+  fi
+}
+
+#输出运行状态
+status(){
+  is_exist
+  if [ $? -eq "0" ]; then
+    echo ">>> 小说精品屋前台正在运行 PID = ${pid} <<<"
+  else
+    echo ">>> 小说精品屋前台没有运行 <<<"
+  fi
+}
+
+#重启
+restart(){
+  stop
+  start
+}
+
+#根据输入参数，选择执行对应方法，不输入则执行使用说明
+case "$1" in
+  "start")
+    start
+    ;;
+  "stop")
+    stop
+    ;;
+  "status")
+    status
+    ;;
+  "restart")
+    restart
+    ;;
+  *)
+    usage
+    ;;
+esac
+exit 0

novel-front/src/main/build/scripts/readme.txt

@@ -1,8 +0,0 @@
-1：linux启动环境
-sh start.sh
-
-3：windows启动环境
-windows-start.bat
-
-3：linux停止应用
-sh stop.sh

novel-front/src/main/build/scripts/start.sh

@@ -1,47 +0,0 @@
-#!/bin/bash
-
-ENGINE=novel-front.jar
-cd ../
-
-#部署目路
-DEPLOY_DIR=`pwd`
-#获取到当前目录的名称
-SERVER_NAME=`basename $DEPLOY_DIR`
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$ENGINE" |awk '{print $2}'`
-#设置日志文件的输出目录
-LOGS_DIR=$DEPLOY_DIR/logs
-if [ ! -d $LOGS_DIR ]; then
-    mkdir $LOGS_DIR
-fi
-#日志
-STDOUT_FILE=$LOGS_DIR/stdout.log
-#JAVA 环境配置
-JAVA_OPTS=" -Djava.net.preferIPv4Stack=true -Dlog.home=$LOGS_DIR"
-
-JAVA_MEM_OPTS=" -server -Xms1024m -Xmx1024m -XX:MetaspaceSize=64m -XX:MaxMetaspaceSize=256m  -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=50 -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Xloggc:$LOGS_DIR/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof "
-#退出标志
-RETVAL="0"
-
-if [ -n "$PIDS" ]; then
-		echo "ERROR: The $SERVER_NAME already started!"
-		echo "PID: $PIDS"
-		exit $RETVAL
-fi
-
-nohup java -jar  $JAVA_OPTS $JAVA_MEM_OPTS  -Dloader.path=conf,lib $ENGINE > $STDOUT_FILE 2>&1 &
-COUNT=0
-	while [ $COUNT -lt 1 ]; do
-		echo -e ".\c"
-		sleep 1
-		COUNT=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}' | wc -l`
-		if [ $COUNT -gt 0 ]; then
-			break
-		fi
-	done
-
-	echo "OK!"
-	PIDS=`ps -f | grep java | grep "$DEPLOY_DIR" | awk '{print $2}'`
-	echo "PID: $PIDS"
-	echo "STDOUT: $STDOUT_FILE"

novel-front/src/main/build/scripts/stop.sh

@@ -1,33 +0,0 @@
-#!/bin/bash
-SERVER_NAME=novel-front.jar
-
-#应用进程
-PIDS=`ps -ef | grep java | grep "$SERVER_NAME" |awk '{print $2}'`
-if [ -z "$PIDS" ]; then
-    		echo "ERROR: The $SERVER_NAME does not started!"
-    	exit 1
-	fi
-
-	echo -e "Stopping the $SERVER_NAME ...\c"
-	for PID in $PIDS ; do
-    		kill $PID > /dev/null 2>&1
-	done
-
-	COUNT=0
-	while [ $COUNT -lt 1 ]; do
-    		echo -e ".\c"
-    		sleep 1
-    		COUNT=1
-    		for PID in $PIDS ; do
-        	PID_EXIST=`ps -f -p $PID | grep java`
-        	if [ -n "$PID_EXIST" ]; then
-            		COUNT=0
-            		break
-        	fi
-    		done
-	done
-
-	echo "OK!"
-	echo "PID: $PIDS"
-	PIDS=""
-

novel-front/src/main/build/scripts/windows-start.bat

@@ -1,10 +0,0 @@
-@echo off
-setlocal enabledelayedexpansion
-set JAVA=java
-
-set OPTS=-XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -Xms1024m -Xmx1024m -Xmn256m -Xss256k -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC
-set ENGINE=novel-front.jar
-cd ../
-java -jar %OPTS%   -Dloader.path=conf,lib %ENGINE%
-pause
-

novel-front/src/main/java/com/java2nb/novel/FrontNovelApplication.java

@@ -1,11 +1,14 @@
 package com.java2nb.novel;
 
 import com.github.tobato.fastdfs.FdfsClientConfig;
+import lombok.extern.slf4j.Slf4j;
 import org.mybatis.spring.annotation.MapperScan;
+import org.springframework.boot.CommandLineRunner;
 import org.springframework.boot.SpringApplication;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 import org.springframework.boot.web.servlet.ServletComponentScan;
 import org.springframework.cache.annotation.EnableCaching;
+import org.springframework.context.ApplicationContext;
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Import;
 import org.springframework.scheduling.TaskScheduler;
@@ -13,6 +16,8 @@ import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
 import org.springframework.transaction.annotation.EnableTransactionManagement;
 
+import java.net.InetAddress;
+
 /**
  * @author Administrator
  */
@@ -23,15 +28,23 @@ import org.springframework.transaction.annotation.EnableTransactionManagement;
 @ServletComponentScan
 @MapperScan(basePackages = {"com.java2nb.novel.mapper"})
 @Import(FdfsClientConfig.class)
+@Slf4j
 public class FrontNovelApplication {
 
     public static void main(String[] args) {
         SpringApplication.run(FrontNovelApplication.class);
     }
 
+    @Bean
+    public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
+        return args -> {
+            log.info("项目启动啦，访问路径：{}", "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment().getProperty("server.port"));
+        };
+    }
+
     /**
      * 解决同一时间只能一个定时任务执行的问题
-     * */
+     */
     @Bean
     public TaskScheduler taskScheduler() {
         ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();

novel-front/src/main/java/com/java2nb/novel/service/impl/BookServiceImpl.java

@@ -95,6 +95,8 @@ public class BookServiceImpl implements BookService {
 
     private final BookPriceProperties bookPriceConfig;
 
+    private final IdWorker idWorker = IdWorker.INSTANCE;
+
 
     @SneakyThrows
     @Override
@@ -428,7 +430,7 @@ public class BookServiceImpl implements BookService {
         } else {
             //作者不存在，先创建作者
             Date currentDate = new Date();
-            authorId = IdWorker.INSTANCE.nextId();
+            authorId = idWorker.nextId();
             BookAuthor bookAuthor = new BookAuthor();
             bookAuthor.setId(authorId);
             bookAuthor.setPenName(authorName);
@@ -548,7 +550,7 @@ public class BookServiceImpl implements BookService {
             //并不是更新自己的小说
             return;
         }
-        Long lastIndexId = IdWorker.INSTANCE.nextId();
+        Long lastIndexId = idWorker.nextId();
         Date currentDate = new Date();
         int wordCount = StringUtil.getStrValidWordCount(content);
 

novel-front/src/main/java/com/java2nb/novel/service/impl/UserServiceImpl.java

@@ -56,6 +56,7 @@ public class UserServiceImpl implements UserService {
 
     private final UserBuyRecordMapper userBuyRecordMapper;
 
+    private final IdWorker idWorker = IdWorker.INSTANCE;
 
 
     @Override
@@ -74,7 +75,7 @@ public class UserServiceImpl implements UserService {
         User entity = new User();
         BeanUtils.copyProperties(user,entity);
         //数据库生成注册记录
-        Long id = IdWorker.INSTANCE.nextId();
+        Long id = idWorker.nextId();
         entity.setId(id);
         entity.setNickName(entity.getUsername());
         Date currentDate = new Date();

novel-front/src/main/resources/templates/index.html

@@ -35,7 +35,7 @@
             <div class="hot_articles">
                 <dl class="hot_recommend" id="topBooks1" th:if="${bookMap['1']} and ${#lists.size(bookMap['1']) > 0}">
                     <dt><a th:href="'/book/'+${bookMap['1'][0].bookId}+'.html'" th:text="${bookMap['1'][0].bookName}"></a></dt>
-                    <dd th:if="${#lists.size(bookMap['1']) > 1}"><a th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 2}" th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a></dd>
+                    <dd th:if="${#lists.size(bookMap['1']) > 1}"><a th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 2}" th:href="'/book/'+${bookMap['1'][2].bookId}+'.html'" th:text="${bookMap['1'][2].bookName}"></a></dd>
                     <dd th:if="${#lists.size(bookMap['1']) > 3}"><a th:href="'/book/'+${bookMap['1'][3].bookId}+'.html'" th:text="${bookMap['1'][3].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 4}" th:href="'/book/'+${bookMap['1'][4].bookId}+'.html'" th:text="${bookMap['1'][4].bookName}"></a></dd>
                 </dl>
                 <dl class="hot_recommend" id="topBooks2" th:if="${bookMap['1']} and ${#lists.size(bookMap['1']) > 5}">

pom.xml

@@ -5,7 +5,7 @@
 
     <groupId>com.java2nb</groupId>
     <artifactId>novel</artifactId>
-    <version>3.6.1</version>
+    <version>3.6.2</version>
     <modules>
         <module>novel-common</module>
         <module>novel-front</module>

templates/orange/html/index.html

@@ -35,7 +35,7 @@
             <div class="hot_articles">
                 <dl class="hot_recommend" id="topBooks1" th:if="${bookMap['1']} and ${#lists.size(bookMap['1']) > 0}">
                     <dt><a th:href="'/book/'+${bookMap['1'][0].bookId}+'.html'" th:text="${bookMap['1'][0].bookName}"></a></dt>
-                    <dd th:if="${#lists.size(bookMap['1']) > 1}"><a th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 2}" th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a></dd>
+                    <dd th:if="${#lists.size(bookMap['1']) > 1}"><a th:href="'/book/'+${bookMap['1'][1].bookId}+'.html'" th:text="${bookMap['1'][1].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 2}" th:href="'/book/'+${bookMap['1'][2].bookId}+'.html'" th:text="${bookMap['1'][2].bookName}"></a></dd>
                     <dd th:if="${#lists.size(bookMap['1']) > 3}"><a th:href="'/book/'+${bookMap['1'][3].bookId}+'.html'" th:text="${bookMap['1'][3].bookName}"></a><a th:if="${#lists.size(bookMap['1']) > 4}" th:href="'/book/'+${bookMap['1'][4].bookId}+'.html'" th:text="${bookMap['1'][4].bookName}"></a></dd>
                 </dl>
                 <dl class="hot_recommend" id="topBooks2" th:if="${bookMap['1']} and ${#lists.size(bookMap['1']) > 5}">

templates/orange/html/mobile/book/book_detail.html

@@ -147,10 +147,10 @@
 
 
 <div style="text-align: center;height: 45px;line-height: 45px">
-    <a th:href="'/book/'+${book.id}+'/'+${firstBookIndexId}+'.html'"  type="button" class="layui-btn layui-btn-sm layui-btn-radius">开始阅读</a>
-    <button type="button" id="cFavs" onclick="addInShell()"  class="layui-btn layui-btn-sm layui-btn-radius layui-btn-warm">加入书架</button>
+    <a th:href="'/book/'+${book.id}+'/'+${firstBookIndexId}+'.html'"  type="button" style="background-color:#ff8900!important" class="layui-btn layui-btn-sm layui-btn-radius">开始阅读</a>
+    <button type="button" id="cFavs" onclick="addInShell()" style="background-color:#ffA640!important" class="layui-btn layui-btn-sm layui-btn-radius ">加入书架</button>
 
-    <button type="button" onclick="location.href='/user/favorites.html'" class="layui-btn layui-btn-sm layui-btn-radius layui-bg-cyan">我的书架</button>
+    <button type="button" onclick="location.href='/user/favorites.html'" style="background-color:#ffBE73!important" class="layui-btn layui-btn-sm layui-btn-radius ">我的书架</button>
 <!--
     <button type="button" onclick="downloadFile()" class="layui-btn layui-btn-sm layui-btn-radius layui-bg-normal">下载TXT</button>
 -->