v5.2.1 发布

MySQL 8.0+ 默认使用 caching_sha2_password 认证插件，这种认证方式有两种工作模式：
- 如果使用SSL连接，直接通过安全通道传输密码
- 如果不使用SSL连接，客户端需要从服务器获取RSA公钥来加密密码
当设置 useSSL=false 但未明确允许公钥检索时，JDBC驱动出于安全考虑会阻止这种操作（报错：Public Key Retrieval is not allowed）。
生产环境中，应优先考虑：
1. 启用 SSL/TLS 加密连接
2. 如需禁用 SSL，改用 mysql_native_password 认证
3. 仅在受控环境（如开发环境）中使用 allowPublicKeyRetrieval=true

.github/workflows/release.yml

@@ -0,0 +1,79 @@
+name: Create novel-plus Maven Release with ZIPs
+
+on:
+  push:
+    # 匹配所有以'v'开头的标签
+    tags:
+      - 'v*'
+
+jobs:
+  build-and-release:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v3
+        with:
+          java-version: '21'
+          # 可选，默认是 temurin，也可以选择其他发行版
+          distribution: 'temurin'
+
+      - name: Build project with Maven
+        run: mvn clean install -DskipTests=true -Pcentral-repo
+
+      - name: Create Release
+        id: create_release
+        uses: actions/create-release@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          # 使用 tag_name 而不是 github.ref
+          tag_name: ${{ github.ref_name }}
+          release_name: novel-plus ${{ github.ref_name }}
+          draft: false
+          prerelease: false
+
+
+      # 使用 action 来替代直接 curl 进行上传
+      - name: Upload sql.zip
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ${{ github.workspace }}/novel-common/target/build/sql.zip
+          asset_name: sql.zip
+          asset_content_type: application/zip
+
+      - name: Upload novel-crawl.zip
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ${{ github.workspace }}/novel-crawl/target/build/novel-crawl.zip
+          asset_name: novel-crawl.zip
+          asset_content_type: application/zip
+
+      - name: Upload novel-front.zip
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ${{ github.workspace }}/novel-front/target/build/novel-front.zip
+          asset_name: novel-front.zip
+          asset_content_type: application/zip
+
+      - name: Upload novel-admin.zip
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ${{ github.workspace }}/novel-admin/target/build/novel-admin.zip
+          asset_name: novel-admin.zip
+          asset_content_type: application/zip

README.md

@@ -1,28 +1,28 @@
-[![index]( https://youdoc.github.io/img/tencent.jpg )]( https://cloud.tencent.com/act/cps/redirect?redirect=2446&cps_key=736e609d66e0ac4e57813316cec6fd0b&from=console )
-
+<p align="center">
+    <a href="https://cloud.tencent.com/act/cps/redirect?redirect=2446&cps_key=736e609d66e0ac4e57813316cec6fd0b&from=console"><img src="https://youdoc.github.io/img/tencent.jpg" alt="AD" ></a>
+</p>
 <p align="center">
     <a href='https://github.com/201206030/novel-plus'><img alt="Github stars" src="https://img.shields.io/github/stars/201206030/novel-plus?logo=github"></a>
     <a href='https://github.com/201206030/novel-plus'><img alt="Github forks" src="https://img.shields.io/github/forks/201206030/novel-plus?logo=github"></a>
     <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee stars" src="https://gitee.com/novel_dev_team/novel-plus/badge/star.svg?theme=gitee"></a>
     <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee forks" src="https://gitee.com/novel_dev_team/novel-plus/badge/fork.svg?theme=gitee"></a>
-    <a href="https://github.com/201206030/novel-plus"><img src="https://visitor-badge.glitch.me/badge?page_id=201206030.novel-plus" alt="visitors"></a>
 </p>
 
 <p align="center">
-      👉 <a href='https://novel.xxyopen.com'>官网</a>  |  👉 <a href='https://www.bilibili.com/video/BV1Zo4y187Mi'>项目演示</a>  |  👉 <a href='https://docs.xxyopen.com/course/novelplus/1.html'>安装教程</a>
+      👉 <a href='https://novel.xxyopen.com'>官网</a>  |  👉 <a href='http://117.72.165.13:8888'>演示站点</a>  |  👉 <a href='https://docs.xxyopen.com/course/novelplus/1.html'>安装教程</a>
 </p> 
 
 ## 项目介绍
 
 novel-plus 是一个多端（PC、WAP）阅读，功能完善的原创文学 CMS
 系统。由前台门户系统、作家后台管理系统、平台后台管理系统和爬虫管理系统等多个子系统构成，包括小说推荐、作品检索、小说排行、小说阅读、小说评论、会员中心、作家专区等功能，支持自定义多模版、可拓展的多种小说内容存储方式（内置数据库分表存储和
-TXT 文本存储）、阅读主题切换、多爬虫源自动采集和更新数据、会员充值、订阅模式、新闻发布和实时统计报表。
+TXT 文本存储）、阅读主题切换、多爬虫源自动采集和更新数据、AI写作、会员充值、订阅模式、新闻发布和实时统计报表。
 
 ## 项目地址
 
 - 学习版：[GitHub](https://github.com/201206030/novel) ｜ [码云](https://gitee.com/novel_dev_team/novel)
   ｜ [保姆级教程](https://docs.xxyopen.com) 
-- **应用版**：[GitHub](https://github.com/201206030/novel-plus) ｜ [码云](https://gitee.com/novel_dev_team/novel-plus)
+- **应用版**：[GitHub](https://github.com/201206030/novel-plus) ｜ [码云](https://gitee.com/novel_dev_team/novel-plus) ｜ [演示站点](http://117.72.165.13:8888)
 - 微服务版：[GitHub](https://github.com/201206030/novel-cloud) ｜ [码云](https://gitee.com/novel_dev_team/novel-cloud)
 
 ## 项目结构
@@ -39,8 +39,9 @@ novel-plus -- 父工程
 ## 技术选型
 
 | 技术                  | 说明                  
-|---------------------| ---------------------------
+|---------------------|---------------------
 | Spring Boot         | Spring 应用快速开发脚手架    
+| Spring AI           | Spring 官方 AI 框架     
 | MyBatis             | 持久层 ORM 框架          
 | MyBatis Dynamic SQL | Mybatis 动态 sql      
 | PageHelper          | MyBatis 分页插件        
@@ -61,13 +62,46 @@ novel-plus -- 父工程
 
 ### 绿色主题模版
 
-[![点击查看大图](https://youdoc.gitee.io/resource/images/os/novel-plus/green.png)](https://youdoc.gitee.io/resource/images/os/novel-plus/green.png)
-[![点击查看大图](https://youdoc.gitee.io/resource/images/os/novel-plus/green3.png)](https://youdoc.gitee.io/resource/images/os/novel-plus/green3.png)
-[![点击查看大图](https://youdoc.gitee.io/resource/images/os/novel-plus/green2.png)](https://youdoc.gitee.io/resource/images/os/novel-plus/green2.png)
+[![点击查看大图](https://www.xxyopen.com/images/green_novel.png)](https://www.xxyopen.com/images/green_novel.png)
+[![点击查看大图](https://www.xxyopen.com/images/resource/os/novel-plus/green3.png)](https://www.xxyopen.com/images/resource/os/novel-plus/green3.png)
+[![点击查看大图](https://www.xxyopen.com/images/resource/os/novel-plus/green2.png)](https://www.xxyopen.com/images/resource/os/novel-plus/green2.png)
 
 ## 演示视频
 
-https://www.bilibili.com/video/BV1Zo4y187Mi
+https://www.bilibili.com/video/BV18e41197xs
+
+## AI 功能
+
+novel-plus 5.x 已集成 Spring 官方最新发布的 Spring AI 框架，并推出多项 AI 功能：
+
+1. v5.0.0 版本在小说章节发布页面的文本编辑器中集成了多项智能编辑功能，包括 AI 扩写、缩写、续写及文本润色等。这些功能的设计灵感来源于百家号文章编辑器中的 AI 助手。
+2. v5.1.0 版本在小说发布页面，新增 AI 生成封面图功能。若作家未上传自定义封面图，系统将根据小说信息自动生成封面图。
+
+目前，AI 功能仍处于实验阶段，仅实现了基础的核心功能。我们非常重视用户的实际使用体验和反馈，未来将根据用户需求和使用情况，持续优化和调整该功能。如果用户反馈积极，我们计划进一步开发更高级的 AI 功能，例如自动生成有声小说、智能情节推荐等，以全面提升 novel-plus 的创作能力和用户体验。
+
+我们将持续关注 AI 技术的发展，并致力于将其与小说创作场景深度融合，为用户带来更智能、更便捷的创作工具。
+
+novel-plus 项目默认使用的是第三方大模型服务平台[硅基流动](https://cloud.siliconflow.cn/i/DOgMRH9S)提供的 API（兼容 OpenAI 的相关接口，可直接通过 Spring AI 框架调用），采用的 AI 模型有对话模型`deepseek-ai/DeepSeek-R1-Distill-Llama-8B`（DeepSeek-R1 的蒸馏版本，免费使用）和生图模型`Kwai-Kolors/Kolors`（快手 Kolors 团队开发的文本到图像生成模型，免费使用）。只需注册一个硅基流动账号，创建一个 API 密钥，并将其添加到 novel-plus 项目 novel-front 模块的 yaml 配置文件中，即可体验 novel-plus 项目的 AI 写作功能。
+
+```yaml
+spring:
+  ai:
+    openai:
+      image:
+        enabled: true
+        base-url: https://api.siliconflow.cn
+        api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
+        options:
+          model: Kwai-Kolors/Kolors
+          response_format: URL
+      api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
+      base-url: https://api.siliconflow.cn
+      chat:
+        options:
+          model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
+```
+
+⚠️ novel-plus 项目默认使用的都是免费 AI 模型，生成效果有限。如果对生成内容有更高的要求，建议选用付费的 AI 模型。
 
 ## 增值服务
 
@@ -93,3 +127,5 @@ https://www.bilibili.com/video/BV1Zo4y187Mi
 ## 免责声明
 
 本项目提供的爬虫工具仅用于采集项目初期的测试数据，请勿用于商业盈利。 用户使用本系统从事任何违法违规的事情，一切后果由用户自行承担，作者不承担任何责任。
+
+

config/shardingsphere-jdbc.yml

@@ -0,0 +1,53 @@
+mode:
+  # 单机模式
+  type: Standalone
+  # 元数据持久化
+  repository:
+    # 数据库持久化
+    type: JDBC
+
+# 数据源配置
+dataSources:
+  ds_1:
+    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
+    driverClassName: com.mysql.cj.jdbc.Driver
+    jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+  ds_2:
+    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
+    driverClassName: com.mysql.cj.jdbc.Driver
+    url: jdbc:mysql://localhost:3306/information_schema?allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+# 规则配置
+rules:
+  - !SINGLE
+    tables:
+      - "*.*"
+  - !SHARDING
+    tables: # 数据分片规则配置
+      book_content:
+        # 分库策略，缺省表示使用默认分库策略
+        actualDataNodes: ds_${1}.book_content${0..9}
+        # 分表策略
+        tableStrategy:
+          standard:
+            # 分片列名称
+            shardingColumn: index_id
+            # 分片算法名称
+            shardingAlgorithmName: bookContentSharding
+
+    shardingAlgorithms:
+      bookContentSharding:
+        # 行表达式分片算法，使用 Groovy 的表达式，提供对 SQL 语句中的 = 和 IN 的分片操作支持
+        type: INLINE
+        props:
+          # 分片算法的行表达式
+          algorithm-expression: book_content${index_id % 10}
+
+
+
+props:
+  # 是否在日志中打印 SQL
+  sql-show: true

doc/sql/20240512.sql

@@ -0,0 +1,3 @@
+update crawl_source
+set crawl_rule = replace(crawl_rule, 'ibiquge.net', 'ibiquzw.org')
+where id = 16;

doc/sql/20250317.sql

@@ -0,0 +1,44 @@
+INSERT INTO crawl_source (source_name, crawl_rule, source_status, create_time, update_time)
+VALUES ('香书小说网', '{
+  "bookListUrl": "http://www.xbiqugu.la/fenlei/{catId}_{page}.html",
+  "catIdRule": {
+    "catId1": "1",
+    "catId2": "2",
+    "catId3": "3",
+    "catId4": "4",
+    "catId5": "6",
+    "catId6": "5"
+  },
+  "bookIdPatten": "<a\\\\s+href=\\"http://www.xbiqugu.la/(\\\\d+/\\\\d+)/\\"\\\\s+target=\\"_blank\\">",
+  "pagePatten": "<em\\\\s+id=\\"pagestats\\">(\\\\d+)/\\\\d+</em>",
+  "totalPagePatten": "<em\\\\s+id=\\"pagestats\\">\\\\d+/(\\\\d+)</em>",
+  "bookDetailUrl": "http://www.xbiqugu.la/{bookId}/",
+  "bookNamePatten": "<h1>([^/]+)</h1>",
+  "authorNamePatten": "者：([^/]+)</p>",
+  "picUrlPatten": "src=\\"(http://www.xbiqugu.la/files/article/image/\\\\d+/\\\\d+/\\\\d+s\\\\.jpg)\\"",
+  "bookStatusRule": {},
+  "descStart": "<div id=\\"intro\\">",
+  "descEnd": "</div>",
+  "upadateTimePatten": "<p>最后更新：(\\\\d+-\\\\d+-\\\\d+\\\\s\\\\d+:\\\\d+:\\\\d+)</p>",
+  "upadateTimeFormatPatten": "yyyy-MM-dd HH:mm:ss",
+  "bookIndexUrl": "http://www.xbiqugu.la/{bookId}/",
+  "indexIdPatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/(\\\\d+)\\\\.html''\\\\s+>[^/]+</a>",
+  "indexNamePatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/\\\\d+\\\\.html''\\\\s+>([^/]+)</a>",
+  "bookContentUrl": "http://www.xbiqugu.la/{bookId}/{indexId}.html",
+  "contentStart": "<div id=\\"content\\">",
+  "contentEnd": "<p>",
+  "filterContent":"<div\\\\s+id=\\"content_tip\\">\\\\s*<b>([^/]+)</b>\\\\s*</div>"
+}', 0, '2024-06-01 10:11:39', '2024-06-01 10:11:39');
+
+
+update crawl_source
+set crawl_rule = replace(crawl_rule, 'ibiquzw.org', 'biquxs.info')
+where id = 16;
+
+delete
+from sys_menu
+where menu_id = 104;
+
+delete
+from sys_menu
+where menu_id = 57;

doc/sql/20250630.sql

@@ -0,0 +1,3 @@
+alter table book_comment add column location varchar(50) DEFAULT NULL COMMENT '地理位置' after comment_content ;
+
+

doc/sql/20250711.sql

@@ -0,0 +1,3 @@
+alter table crawl_single_task add column crawl_chapters int DEFAULT 0 COMMENT '采集章节数量' after exc_count ;
+
+

doc/sql/20250712.sql

@@ -0,0 +1,13 @@
+DROP TABLE IF EXISTS `book_comment_reply`;
+CREATE TABLE `book_comment_reply`
+(
+    `id`             bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主键',
+    `comment_id`     bigint(20)   DEFAULT NULL COMMENT '评论ID',
+    `reply_content`  varchar(512) DEFAULT NULL COMMENT '回复内容',
+    `location` varchar(50) DEFAULT NULL COMMENT '地理位置',
+    `audit_status`   tinyint(1)   DEFAULT '0' COMMENT '审核状态，0：待审核，1：审核通过，2：审核不通过',
+    `create_time`    datetime     DEFAULT NULL COMMENT '回复用户ID',
+    `create_user_id` bigint(20)   DEFAULT NULL COMMENT '回复时间',
+    PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说评论回复表';

doc/sql/Dockerfile

@@ -0,0 +1,2 @@
+FROM mysql:8.0
+COPY novel_plus.sql /docker-entrypoint-initdb.d/init.sql

doc/sql/novel_plus.sql

@@ -1,20 +1,7 @@
-/*
-Navicat MySQL Data Transfer
+CREATE database if NOT EXISTS `novel_plus` default character set utf8mb4 collate utf8mb4_unicode_ci;
+use `novel_plus`;
 
-Source Server         : localhost
-Source Server Version : 50725
-Source Host           : localhost:3306
-Source Database       : novel_plus
-
-Target Server Type    : MYSQL
-Target Server Version : 50725
-File Encoding         : 65001
-
-Date: 2020-05-18 13:59:04
-*/
-
-SET
-FOREIGN_KEY_CHECKS=0;
+SET NAMES utf8mb4;
 
 -- ----------------------------
 -- Table structure for author
@@ -33,7 +20,9 @@ CREATE TABLE `author`
     `status`         tinyint(4)  DEFAULT '0' COMMENT '0：正常，1：封禁',
     `create_time`    datetime    DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COMMENT='作者表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 3
+  DEFAULT CHARSET = utf8mb4 COMMENT ='作者表';
 
 -- ----------------------------
 -- Records of author
@@ -58,7 +47,9 @@ CREATE TABLE `author_code`
     `create_user_id` bigint(20)   DEFAULT NULL COMMENT '创建人ID',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_code` (`invite_code`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COMMENT='作家邀请码表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 6
+  DEFAULT CHARSET = utf8mb4 COMMENT ='作家邀请码表';
 
 -- ----------------------------
 -- Records of author_code
@@ -105,7 +96,9 @@ CREATE TABLE `book`
     UNIQUE KEY `key_uq_bookName_authorName` (`book_name`, `author_name`) USING BTREE,
     KEY `key_lastIndexUpdateTime` (`last_index_update_time`) USING BTREE,
     KEY `key_createTime` (`create_time`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1262260513468559361 DEFAULT CHARSET=utf8mb4 COMMENT='小说表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1262260513468559361
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说表';
 
 -- ----------------------------
 -- Records of book
@@ -130,7 +123,9 @@ CREATE TABLE `book_author`
     `update_time`    datetime    DEFAULT NULL COMMENT '更新时间',
     `update_user_id` bigint(20)  DEFAULT NULL COMMENT '更新人ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=1254957873655066625 DEFAULT CHARSET=utf8mb4 COMMENT='作者表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1254957873655066625
+  DEFAULT CHARSET = utf8mb4 COMMENT ='作者表';
 
 -- ----------------------------
 -- Records of book_author
@@ -151,7 +146,9 @@ CREATE TABLE `book_category`
     `update_user_id` bigint(20)           DEFAULT NULL,
     `update_time`    datetime             DEFAULT NULL,
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8mb4 COMMENT='小说类别表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 8
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说类别表';
 
 -- ----------------------------
 -- Records of book_category
@@ -186,7 +183,9 @@ CREATE TABLE `book_comment`
     `create_user_id`  bigint(20)   DEFAULT NULL COMMENT '评价人',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_bookid_userid` (`book_id`, `create_user_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=13 DEFAULT CHARSET=utf8mb4 COMMENT='小说评论表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 13
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说评论表';
 
 -- ----------------------------
 -- Records of book_comment
@@ -209,7 +208,8 @@ CREATE TABLE `book_comment_reply`
     `create_time`    datetime     DEFAULT NULL COMMENT '回复用户ID',
     `create_user_id` bigint(20)   DEFAULT NULL COMMENT '回复时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='小说评论回复表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说评论回复表';
 
 -- ----------------------------
 -- Records of book_comment_reply
@@ -226,7 +226,9 @@ CREATE TABLE `book_content`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=3347665 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 3347665
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Records of book_content
@@ -250,7 +252,9 @@ CREATE TABLE `book_index`
     UNIQUE KEY `key_uq_bookId_indexNum` (`book_id`, `index_num`) USING BTREE,
     KEY `key_bookId` (`book_id`) USING BTREE,
     KEY `key_indexNum` (`index_num`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1262260612777095169 DEFAULT CHARSET=utf8mb4 COMMENT='小说目录表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1262260612777095169
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说目录表';
 
 -- ----------------------------
 -- Records of book_index
@@ -268,7 +272,9 @@ CREATE TABLE `book_screen_bullet`
     `create_time`   datetime     NOT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`),
     KEY `key_contentId` (`content_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=79 DEFAULT CHARSET=utf8mb4 COMMENT='小说弹幕表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 79
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说弹幕表';
 
 -- ----------------------------
 -- Records of book_screen_bullet
@@ -289,7 +295,9 @@ CREATE TABLE `book_setting`
     `update_time`    datetime   DEFAULT NULL COMMENT '更新时间',
     `update_user_id` bigint(20) DEFAULT NULL COMMENT '更新人ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=64 DEFAULT CHARSET=utf8mb4 COMMENT='首页小说设置表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 64
+  DEFAULT CHARSET = utf8mb4 COMMENT ='首页小说设置表';
 
 -- ----------------------------
 -- Records of book_setting
@@ -373,7 +381,8 @@ CREATE TABLE `crawl_batch_task`
     `start_time`          datetime   DEFAULT NULL COMMENT '任务开始时间',
     `end_time`            datetime   DEFAULT NULL COMMENT '任务结束时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='批量抓取任务表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='批量抓取任务表';
 
 -- ----------------------------
 -- Records of crawl_batch_task
@@ -392,7 +401,8 @@ CREATE TABLE `crawl_single_task`
     `exc_count`      tinyint(2)   DEFAULT '0' COMMENT '已经执行次数，最多执行5次',
     `create_time`    datetime     DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='抓取单本小说任务表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='抓取单本小说任务表';
 
 -- ----------------------------
 -- Records of crawl_single_task
@@ -411,7 +421,9 @@ CREATE TABLE `crawl_source`
     `create_time`   datetime    DEFAULT NULL COMMENT '创建时间',
     `update_time`   datetime    DEFAULT NULL COMMENT '更新时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COMMENT='爬虫源表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 5
+  DEFAULT CHARSET = utf8mb4 COMMENT ='爬虫源表';
 
 -- ----------------------------
 -- Records of crawl_source
@@ -449,7 +461,9 @@ CREATE TABLE `friend_link`
     `update_user_id` bigint(20)            DEFAULT NULL COMMENT '更新者用户id',
     `update_time`    datetime              DEFAULT NULL COMMENT '更新时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4;
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 6
+  DEFAULT CHARSET = utf8mb4;
 
 -- ----------------------------
 -- Records of friend_link
@@ -474,7 +488,9 @@ CREATE TABLE `news`
     `update_time`    datetime     DEFAULT NULL COMMENT '更新时间',
     `update_user_id` bigint(20)   DEFAULT NULL COMMENT '更新人ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COMMENT='新闻表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 3
+  DEFAULT CHARSET = utf8mb4 COMMENT ='新闻表';
 
 -- ----------------------------
 -- Records of news
@@ -502,7 +518,9 @@ CREATE TABLE `news_category`
     `update_user_id` bigint(20)           DEFAULT NULL,
     `update_time`    datetime             DEFAULT NULL,
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8mb4 COMMENT='新闻类别表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 4
+  DEFAULT CHARSET = utf8mb4 COMMENT ='新闻类别表';
 
 -- ----------------------------
 -- Records of news_category
@@ -528,7 +546,9 @@ CREATE TABLE `order_pay`
     `create_time`  datetime            DEFAULT NULL COMMENT '创建时间',
     `update_time`  datetime            DEFAULT NULL COMMENT '更新时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=15 DEFAULT CHARSET=utf8mb4 COMMENT='充值订单';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 15
+  DEFAULT CHARSET = utf8mb4 COMMENT ='充值订单';
 
 -- ----------------------------
 -- Records of order_pay
@@ -551,7 +571,9 @@ CREATE TABLE `sys_data_perm`
     `gmt_create`      datetime    DEFAULT NULL COMMENT '创建时间',
     `gmt_modified`    datetime    DEFAULT NULL COMMENT '修改时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=216 DEFAULT CHARSET=utf8 COMMENT='数据权限管理';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 216
+  DEFAULT CHARSET = utf8 COMMENT ='数据权限管理';
 
 -- ----------------------------
 -- Records of sys_data_perm
@@ -581,7 +603,9 @@ CREATE TABLE `sys_dept`
     `order_num` int(11)     DEFAULT NULL COMMENT '排序',
     `del_flag`  tinyint(4)  DEFAULT '0' COMMENT '是否删除  -1：已删除  0：正常',
     PRIMARY KEY (`dept_id`)
-) ENGINE=InnoDB AUTO_INCREMENT=17 DEFAULT CHARSET=utf8 COMMENT='部门管理';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 17
+  DEFAULT CHARSET = utf8 COMMENT ='部门管理';
 
 -- ----------------------------
 -- Records of sys_dept
@@ -618,7 +642,10 @@ CREATE TABLE `sys_dict`
     KEY `sys_dict_value` (`value`),
     KEY `sys_dict_label` (`name`),
     KEY `sys_dict_del_flag` (`del_flag`)
-) ENGINE=InnoDB AUTO_INCREMENT=142 DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='字典表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 142
+  DEFAULT CHARSET = utf8
+  COLLATE = utf8_bin COMMENT ='字典表';
 
 -- ----------------------------
 -- Records of sys_dict
@@ -895,7 +922,9 @@ CREATE TABLE `sys_file`
     `url`         varchar(200) DEFAULT NULL COMMENT 'URL地址',
     `create_date` datetime     DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=183 DEFAULT CHARSET=utf8 COMMENT='文件上传';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 183
+  DEFAULT CHARSET = utf8 COMMENT ='文件上传';
 
 -- ----------------------------
 -- Records of sys_file
@@ -1001,7 +1030,9 @@ CREATE TABLE `sys_gen_columns`
     `is_required`    tinyint(1)                       DEFAULT NULL COMMENT '是否必填',
     `dict_type`      varchar(100) CHARACTER SET utf8  DEFAULT '' COMMENT '页面显示为下拉时使用，字典类型从字典表中取出',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=815 DEFAULT CHARSET=utf8mb4;
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 815
+  DEFAULT CHARSET = utf8mb4;
 
 -- ----------------------------
 -- Records of sys_gen_columns
@@ -1189,7 +1220,8 @@ CREATE TABLE `sys_gen_table`
     `update_date`          datetime     NOT NULL COMMENT '更新时间',
     `remarks`              varchar(500)          DEFAULT NULL COMMENT '备注信息',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='代码生成表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8 COMMENT ='代码生成表';
 
 -- ----------------------------
 -- Records of sys_gen_table
@@ -1225,7 +1257,8 @@ CREATE TABLE `sys_gen_table_column`
     `options`      varchar(1000)  DEFAULT NULL COMMENT '其它生成选项',
     PRIMARY KEY (`id`),
     KEY `idx_gen_table_column_tn` (`table_id`) USING BTREE
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='代码生成表列';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8 COMMENT ='代码生成表列';
 
 -- ----------------------------
 -- Records of sys_gen_table_column
@@ -1247,7 +1280,9 @@ CREATE TABLE `sys_log`
     `ip`         varchar(64)   DEFAULT NULL COMMENT 'IP地址',
     `gmt_create` datetime      DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=1412 DEFAULT CHARSET=utf8 COMMENT='系统日志';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1412
+  DEFAULT CHARSET = utf8 COMMENT ='系统日志';
 
 
 -- ----------------------------
@@ -1267,7 +1302,9 @@ CREATE TABLE `sys_menu`
     `gmt_create`   datetime     DEFAULT NULL COMMENT '创建时间',
     `gmt_modified` datetime     DEFAULT NULL COMMENT '修改时间',
     PRIMARY KEY (`menu_id`)
-) ENGINE=InnoDB AUTO_INCREMENT=234 DEFAULT CHARSET=utf8 COMMENT='菜单管理';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 234
+  DEFAULT CHARSET = utf8 COMMENT ='菜单管理';
 
 -- ----------------------------
 -- Records of sys_menu
@@ -1417,7 +1454,9 @@ CREATE TABLE `sys_role`
     `gmt_create`     datetime     DEFAULT NULL COMMENT '创建时间',
     `gmt_modified`   datetime     DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`role_id`)
-) ENGINE=InnoDB AUTO_INCREMENT=62 DEFAULT CHARSET=utf8 COMMENT='角色';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 62
+  DEFAULT CHARSET = utf8 COMMENT ='角色';
 
 -- ----------------------------
 -- Records of sys_role
@@ -1435,7 +1474,9 @@ CREATE TABLE `sys_role_data_perm`
     `role_id` bigint(20) DEFAULT NULL COMMENT '角色ID',
     `perm_id` bigint(20) DEFAULT NULL COMMENT '权限ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=79 DEFAULT CHARSET=utf8 COMMENT='角色与数据权限对应关系';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 79
+  DEFAULT CHARSET = utf8 COMMENT ='角色与数据权限对应关系';
 
 -- ----------------------------
 -- Records of sys_role_data_perm
@@ -1473,7 +1514,9 @@ CREATE TABLE `sys_role_menu`
     `role_id` bigint(20) DEFAULT NULL COMMENT '角色ID',
     `menu_id` bigint(20) DEFAULT NULL COMMENT '菜单ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=4830 DEFAULT CHARSET=utf8 COMMENT='角色与菜单对应关系';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 4830
+  DEFAULT CHARSET = utf8 COMMENT ='角色与菜单对应关系';
 
 -- ----------------------------
 -- Records of sys_role_menu
@@ -2255,7 +2298,9 @@ CREATE TABLE `sys_user`
     `city`           varchar(255) DEFAULT NULL COMMENT '所在城市',
     `district`       varchar(255) DEFAULT NULL COMMENT '所在地区',
     PRIMARY KEY (`user_id`)
-) ENGINE=InnoDB AUTO_INCREMENT=139 DEFAULT CHARSET=utf8;
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 139
+  DEFAULT CHARSET = utf8;
 
 -- ----------------------------
 -- Records of sys_user
@@ -2275,7 +2320,9 @@ CREATE TABLE `sys_user_role`
     `user_id` bigint(20) DEFAULT NULL COMMENT '用户ID',
     `role_id` bigint(20) DEFAULT NULL COMMENT '角色ID',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=137 DEFAULT CHARSET=utf8 COMMENT='用户与角色对应关系';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 137
+  DEFAULT CHARSET = utf8 COMMENT ='用户与角色对应关系';
 
 -- ----------------------------
 -- Records of sys_user_role
@@ -2367,7 +2414,9 @@ CREATE TABLE `user`
     `update_time`     datetime     NOT NULL COMMENT '更新时间',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_username` (`username`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1255664783722586113 DEFAULT CHARSET=utf8mb4;
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1255664783722586113
+  DEFAULT CHARSET = utf8mb4;
 
 -- ----------------------------
 -- Records of user
@@ -2408,7 +2457,9 @@ CREATE TABLE `user_bookshelf`
     `update_time`    datetime   DEFAULT NULL,
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_userid_bookid` (`user_id`, `book_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=42 DEFAULT CHARSET=utf8mb4 COMMENT='用户书架表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 42
+  DEFAULT CHARSET = utf8mb4 COMMENT ='用户书架表';
 
 -- ----------------------------
 -- Records of user_bookshelf
@@ -2441,7 +2492,9 @@ CREATE TABLE `user_buy_record`
     `create_time`     datetime     DEFAULT NULL COMMENT '购买时间',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_userId_indexId` (`user_id`, `book_index_id`)
-) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COMMENT='用户消费记录表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 3
+  DEFAULT CHARSET = utf8mb4 COMMENT ='用户消费记录表';
 
 -- ----------------------------
 -- Records of user_buy_record
@@ -2464,7 +2517,9 @@ CREATE TABLE `user_feedback`
     `content`     varchar(512) DEFAULT NULL COMMENT '反馈内容',
     `create_time` datetime     DEFAULT NULL COMMENT '反馈时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8mb4;
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 9
+  DEFAULT CHARSET = utf8mb4;
 
 -- ----------------------------
 -- Records of user_feedback
@@ -2486,7 +2541,9 @@ CREATE TABLE `user_read_history`
     `update_time`    datetime   DEFAULT NULL,
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_userid_bookid` (`user_id`, `book_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=119 DEFAULT CHARSET=utf8mb4 COMMENT='用户阅读记录表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 119
+  DEFAULT CHARSET = utf8mb4 COMMENT ='用户阅读记录表';
 
 -- ----------------------------
 -- Records of user_read_history
@@ -2552,7 +2609,9 @@ CREATE TABLE `book_content0`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1155 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1155
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content1
@@ -2565,7 +2624,9 @@ CREATE TABLE `book_content1`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=406 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 406
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content2
@@ -2578,7 +2639,9 @@ CREATE TABLE `book_content2`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1222 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1222
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content3
@@ -2591,7 +2654,9 @@ CREATE TABLE `book_content3`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=410 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 410
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content4
@@ -2604,7 +2669,9 @@ CREATE TABLE `book_content4`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1188 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1188
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content5
@@ -2617,7 +2684,9 @@ CREATE TABLE `book_content5`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=416 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 416
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content6
@@ -2630,7 +2699,9 @@ CREATE TABLE `book_content6`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1180 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1180
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content7
@@ -2643,7 +2714,9 @@ CREATE TABLE `book_content7`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=404 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 404
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content8
@@ -2656,7 +2729,9 @@ CREATE TABLE `book_content8`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=1134 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 1134
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 -- ----------------------------
 -- Table structure for book_content9
@@ -2669,7 +2744,9 @@ CREATE TABLE `book_content9`
     `content`  mediumtext COMMENT '小说章节内容',
     PRIMARY KEY (`id`),
     UNIQUE KEY `key_uq_indexId` (`index_id`) USING BTREE
-) ENGINE=InnoDB AUTO_INCREMENT=415 DEFAULT CHARSET=utf8mb4 COMMENT='小说内容表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 415
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说内容表';
 
 
 DROP TABLE IF EXISTS `crawl_single_task`;
@@ -2686,7 +2763,9 @@ CREATE TABLE `crawl_single_task`
     `exc_count`      tinyint(2)   DEFAULT '0' COMMENT '已经执行次数，最多执行5次',
     `create_time`    datetime     DEFAULT NULL COMMENT '创建时间',
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8mb4 COMMENT='抓取单本小说任务表';
+) ENGINE = InnoDB
+  AUTO_INCREMENT = 8
+  DEFAULT CHARSET = utf8mb4 COMMENT ='抓取单本小说任务表';
 
 -- ----------------------------
 -- Records of crawl_single_task
@@ -2716,7 +2795,8 @@ CREATE TABLE `author_income_detail`
     `income_number`  int(11)    NOT NULL DEFAULT '0' COMMENT '订阅人数',
     `create_time`    datetime            DEFAULT NULL,
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='稿费收入明细统计表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='稿费收入明细统计表';
 
 CREATE TABLE `author_income`
 (
@@ -2732,7 +2812,8 @@ CREATE TABLE `author_income`
     `detail`           varchar(255)        DEFAULT NULL COMMENT '详情',
     `create_time`      datetime            DEFAULT NULL,
     PRIMARY KEY (`id`)
-) ENGINE=InnoDB  DEFAULT CHARSET=utf8mb4  COMMENT='稿费收入统计表';
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='稿费收入统计表';
 
 
 alter table book
@@ -3012,3 +3093,85 @@ update website_info
 set logo     = 'https://youdoc.gitee.io/resource/images/logo%20(1).png',
     logo_dark='https://youdoc.gitee.io/resource/images/logo%20(1).png'
 where id = 1;
+
+
+
+update crawl_source
+set crawl_rule = replace(crawl_rule, 'ibiquge.net', 'ibiquzw.org')
+where id = 16;
+
+update website_info
+set logo     = '/images/logo.png',
+    logo_dark='/images/logo.png'
+where id = 1;
+
+
+INSERT INTO crawl_source (source_name, crawl_rule, source_status, create_time, update_time)
+VALUES ('香书小说网', '{
+  "bookListUrl": "http://www.xbiqugu.net/fenlei/{catId}_{page}.html",
+  "catIdRule": {
+    "catId1": "1",
+    "catId2": "2",
+    "catId3": "3",
+    "catId4": "4",
+    "catId5": "6",
+    "catId6": "5"
+  },
+  "bookIdPatten": "<a\\\\s+href=\\"http://www.xbiqugu.net/(\\\\d+/\\\\d+)/\\"\\\\s+target=\\"_blank\\">",
+  "pagePatten": "<em\\\\s+id=\\"pagestats\\">(\\\\d+)/\\\\d+</em>",
+  "totalPagePatten": "<em\\\\s+id=\\"pagestats\\">\\\\d+/(\\\\d+)</em>",
+  "bookDetailUrl": "http://www.xbiqugu.net/{bookId}/",
+  "bookNamePatten": "<h1>([^/]+)</h1>",
+  "authorNamePatten": "者：([^/]+)</p>",
+  "picUrlPatten": "src=\\"(http://www.xbiqugu.net/files/article/image/\\\\d+/\\\\d+/\\\\d+s\\\\.jpg)\\"",
+  "bookStatusRule": {},
+  "descStart": "<div id=\\"intro\\">",
+  "descEnd": "</div>",
+  "upadateTimePatten": "<p>最后更新：(\\\\d+-\\\\d+-\\\\d+\\\\s\\\\d+:\\\\d+:\\\\d+)</p>",
+  "upadateTimeFormatPatten": "yyyy-MM-dd HH:mm:ss",
+  "bookIndexUrl": "http://www.xbiqugu.net/{bookId}/",
+  "indexIdPatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/(\\\\d+)\\\\.html''\\\\s+>[^/]+</a>",
+  "indexNamePatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/\\\\d+\\\\.html''\\\\s+>([^/]+)</a>",
+  "bookContentUrl": "http://www.xbiqugu.net/{bookId}/{indexId}.html",
+  "contentStart": "<div id=\\"content\\">",
+  "contentEnd": "<p>",
+  "filterContent":"<div\\\\s+id=\\"content_tip\\">\\\\s*<b>([^/]+)</b>\\\\s*</div>"
+}', 0, '2024-06-01 10:11:39', '2024-06-01 10:11:39');
+
+
+update crawl_source
+set crawl_rule = replace(crawl_rule, 'ibiquzw.org', 'biquxs.info')
+where id = 16;
+
+
+update crawl_source
+set crawl_rule = replace(crawl_rule, 'xbiqugu.net', 'xbiqugu.la');
+
+delete
+from sys_menu
+where menu_id = 104;
+
+delete
+from sys_menu
+where menu_id = 57;
+
+
+alter table book_comment add column location varchar(50) DEFAULT NULL COMMENT '地理位置' after comment_content ;
+
+
+alter table crawl_single_task add column crawl_chapters int DEFAULT 0 COMMENT '采集章节数量' after exc_count ;
+
+
+DROP TABLE IF EXISTS `book_comment_reply`;
+CREATE TABLE `book_comment_reply`
+(
+    `id`             bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主键',
+    `comment_id`     bigint(20)   DEFAULT NULL COMMENT '评论ID',
+    `reply_content`  varchar(512) DEFAULT NULL COMMENT '回复内容',
+    `location` varchar(50) DEFAULT NULL COMMENT '地理位置',
+    `audit_status`   tinyint(1)   DEFAULT '0' COMMENT '审核状态，0：待审核，1：审核通过，2：审核不通过',
+    `create_time`    datetime     DEFAULT NULL COMMENT '回复用户ID',
+    `create_user_id` bigint(20)   DEFAULT NULL COMMENT '回复时间',
+    PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+  DEFAULT CHARSET = utf8mb4 COMMENT ='小说评论回复表';

novel-admin/pom.xml

@@ -5,7 +5,7 @@
 
     <groupId>com.java2nb</groupId>
     <artifactId>novel-admin</artifactId>
-    <version>4.2.0</version>
+    <version>5.2.1</version>
     <packaging>jar</packaging>
 
     <name>novel-admin</name>
@@ -14,18 +14,14 @@
     <parent>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-parent</artifactId>
-        <version>2.1.18.RELEASE</version>
+        <version>2.7.18</version>
         <relativePath/>
     </parent>
 
     <properties>
-        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
-        <java.version>1.8</java.version>
+        <java.version>21</java.version>
         <velocity.version>1.7</velocity.version>
-        <activiti.version>5.22.0</activiti.version>
-        <sharding.jdbc.version>3.0.0</sharding.jdbc.version>
-        <jackson.version>2.15.1</jackson.version>
+        <shardingsphere-jdbc.version>5.5.1</shardingsphere-jdbc.version>
     </properties>
 
     <dependencies>
@@ -60,35 +56,24 @@
             <groupId>net.sourceforge.nekohtml</groupId>
             <artifactId>nekohtml</artifactId>
         </dependency>
+        <!-- 请求参数校验相关 -->
+        <dependency>
+            <groupId>org.hibernate.validator</groupId>
+            <artifactId>hibernate-validator</artifactId>
+        </dependency>
         <!--mybatis -->
         <dependency>
             <groupId>mysql</groupId>
             <artifactId>mysql-connector-java</artifactId>
             <version>8.0.29</version>
         </dependency>
-        <dependency>
-            <groupId>org.mybatis</groupId>
-            <artifactId>mybatis</artifactId>
-            <version>3.5.6</version>
-        </dependency>
 
         <dependency>
             <groupId>org.mybatis.spring.boot</groupId>
             <artifactId>mybatis-spring-boot-starter</artifactId>
             <version>1.1.1</version>
         </dependency>
-        <!--druid -->
-        <dependency>
-            <groupId>com.alibaba</groupId>
-            <artifactId>druid</artifactId>
-            <version>1.2.9</version>
-        </dependency>
         <!--commons -->
-        <dependency>
-            <groupId>org.apache.commons</groupId>
-            <artifactId>commons-lang3</artifactId>
-            <version>3.6</version>
-        </dependency>
         <dependency>
             <groupId>commons-configuration</groupId>
             <artifactId>commons-configuration</artifactId>
@@ -138,6 +123,12 @@
             <groupId>org.apache.velocity</groupId>
             <artifactId>velocity</artifactId>
             <version>1.7</version>
+            <exclusions>
+                <exclusion>
+                    <artifactId>commons-lang</artifactId>
+                    <groupId>commons-lang</groupId>
+                </exclusion>
+            </exclusions>
         </dependency>
         <!--<dependency>-->
         <!--<groupId>org.springframework.boot</groupId>-->
@@ -166,6 +157,12 @@
             <groupId>io.springfox</groupId>
             <artifactId>springfox-swagger2</artifactId>
             <version>2.6.1</version>
+            <exclusions>
+                <exclusion>
+                    <artifactId>guava</artifactId>
+                    <groupId>com.google.guava</groupId>
+                </exclusion>
+            </exclusions>
         </dependency>
         <dependency>
             <groupId>io.springfox</groupId>
@@ -195,24 +192,29 @@
             <groupId>org.apache.commons</groupId>
             <artifactId>commons-text</artifactId>
             <version>1.4</version>
+            <exclusions>
+                <exclusion>
+                    <artifactId>commons-lang3</artifactId>
+                    <groupId>org.apache.commons</groupId>
+                </exclusion>
+            </exclusions>
         </dependency>
 
+        <!-- ShardingSphere-JDBC -->
         <dependency>
-            <groupId>io.shardingsphere</groupId>
-            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
-            <version>${sharding.jdbc.version}</version>
+            <groupId>org.apache.shardingsphere</groupId>
+            <artifactId>shardingsphere-jdbc</artifactId>
+            <version>${shardingsphere-jdbc.version}</version>
         </dependency>
-
         <dependency>
-            <groupId>io.shardingsphere</groupId>
-            <artifactId>sharding-jdbc-spring-namespace</artifactId>
-            <version>${sharding.jdbc.version}</version>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>2.2</version>
         </dependency>
-
         <dependency>
-            <groupId>com.fasterxml.jackson.core</groupId>
-            <artifactId>jackson-databind</artifactId>
-            <version>${jackson.version}</version>
+            <groupId>com.h2database</groupId>
+            <artifactId>h2</artifactId>
+            <scope>runtime</scope>
         </dependency>
 
         <!--war包部署需要-->
@@ -296,6 +298,9 @@
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
                                     <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
+
+                                <copy file="${basedir}/src/main/build/docker/Dockerfile"
+                                      tofile="${project.build.directory}/build/Dockerfile"/>
                             </tasks>
                         </configuration>
                     </execution>
@@ -327,4 +332,35 @@
             </snapshots>
         </pluginRepository>
     </pluginRepositories>
+
+    <profiles>
+        <profile>
+            <!-- 定义一个用于切换到中央仓库的profile -->
+            <id>central-repo</id>
+            <repositories>
+                <repository>
+                    <id>central</id>
+                    <url>https://repo.maven.apache.org/maven2/</url>
+                    <releases>
+                        <enabled>true</enabled>
+                    </releases>
+                    <snapshots>
+                        <enabled>false</enabled>
+                    </snapshots>
+                </repository>
+            </repositories>
+            <pluginRepositories>
+                <pluginRepository>
+                    <id>central-plugin</id>
+                    <url>https://repo.maven.apache.org/maven2/</url>
+                    <releases>
+                        <enabled>true</enabled>
+                    </releases>
+                    <snapshots>
+                        <enabled>false</enabled>
+                    </snapshots>
+                </pluginRepository>
+            </pluginRepositories>
+        </profile>
+    </profiles>
 </project>

novel-admin/src/main/build/config/application-prod.yml

@@ -1,20 +0,0 @@
-#端口号
-server:
-  port: 8088
-spring:
-  datasource:
-    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai
-    username: root
-    password: test123456
-  redis:
-    host: 127.0.0.1
-    port: 6379
-    password: test123456
-
-sharding:
-  jdbc:
-    datasource:
-      ds0:
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456

novel-admin/src/main/build/config/application.yml

@@ -0,0 +1,9 @@
+#端口号
+server:
+  port: 8088
+spring:
+  redis:
+    host: 127.0.0.1
+    port: 6379
+    password: test123456
+

novel-admin/src/main/build/config/shardingsphere-jdbc.yml

@@ -0,0 +1,47 @@
+mode:
+  # 单机模式
+  type: Standalone
+  # 元数据持久化
+  repository:
+    # 数据库持久化
+    type: JDBC
+
+# 数据源配置
+dataSources:
+  ds_1:
+    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
+    driverClassName: com.mysql.cj.jdbc.Driver
+    jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+# 规则配置
+rules:
+  - !SINGLE
+    tables:
+      - "*.*"
+  - !SHARDING
+    tables: # 数据分片规则配置
+      book_content:
+        # 分库策略，缺省表示使用默认分库策略
+        actualDataNodes: ds_${1}.book_content${0..9}
+        # 分表策略
+        tableStrategy:
+          standard:
+            # 分片列名称
+            shardingColumn: index_id
+            # 分片算法名称
+            shardingAlgorithmName: bookContentSharding
+
+    shardingAlgorithms:
+      bookContentSharding:
+        # 行表达式分片算法，使用 Groovy 的表达式，提供对 SQL 语句中的 = 和 IN 的分片操作支持
+        type: INLINE
+        props:
+          # 分片算法的行表达式
+          algorithm-expression: book_content${index_id % 10}
+
+
+
+props:
+  # 是否在日志中打印 SQL
+  sql-show: true

novel-admin/src/main/build/docker/Dockerfile

@@ -1,9 +1,9 @@
 FROM openjdk:8
-ADD novel-admin-4.2.0.jar /root
+ADD novel-admin.jar /root
 ENV dburl=""
 ENV username=""
 ENV	password=""
 ENV redishost = ""
 ENV redisport = ""
 ENV redispwd = ""
-ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-admin-4.2.0.jar"]
+ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-admin.jar"]

novel-admin/src/main/java/com/java2nb/AdminApplication.java

@@ -11,7 +11,9 @@ import org.springframework.context.ApplicationContext;
 import org.springframework.context.annotation.Bean;
 import org.springframework.transaction.annotation.EnableTransactionManagement;
 
+import javax.sql.DataSource;
 import java.net.InetAddress;
+import java.sql.Connection;
 
 
 @EnableTransactionManagement
@@ -23,15 +25,23 @@ import java.net.InetAddress;
 @EnableCaching
 @Slf4j
 public class AdminApplication {
+
     public static void main(String[] args) {
         SpringApplication.run(AdminApplication.class, args);
     }
 
     @Bean
-    public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
+    public CommandLineRunner commandLineRunner(ApplicationContext ctx, DataSource dataSource) {
         return args -> {
-            log.info("项目启动啦，访问路径：{}", "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment().getProperty("server.port"));
+            log.info("创建连接池...");
+            try (Connection connection = dataSource.getConnection()) {
+                log.info("连接池已创建.");
+                log.info("数据库：{}", connection.getMetaData().getDatabaseProductName());
+                log.info("数据库版本：{}", connection.getMetaData().getDatabaseProductVersion());
+            }
+            log.info("项目启动啦，访问路径：{}",
+                "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment()
+                    .getProperty("server.port"));
         };
     }
-
 }

novel-admin/src/main/java/com/java2nb/common/aspect/WebLogAspect.java

@@ -1,19 +1,16 @@
 package com.java2nb.common.aspect;
 
-import com.java2nb.common.utils.HttpContextUtils;
+import com.java2nb.common.utils.IPUtils;
 import org.aspectj.lang.JoinPoint;
 import org.aspectj.lang.ProceedingJoinPoint;
 import org.aspectj.lang.annotation.*;
-import org.aspectj.lang.reflect.MethodSignature;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.stereotype.Component;
 import org.springframework.web.context.request.RequestContextHolder;
 import org.springframework.web.context.request.ServletRequestAttributes;
-import sun.net.util.IPAddressUtil;
 
 import javax.servlet.http.HttpServletRequest;
-import java.lang.reflect.Method;
 import java.util.Arrays;
 
 @Aspect
@@ -37,11 +34,10 @@ public class WebLogAspect {
         logger.info("请求地址 : " + request.getRequestURL().toString());
         logger.info("HTTP METHOD : " + request.getMethod());
         // 获取真实的ip地址
-        //logger.info("IP : " + IPAddressUtil.getClientIpAddress(request));
+        logger.info("IP : " + IPUtils.getIpAddr(request));
         logger.info("CLASS_METHOD : " + joinPoint.getSignature().getDeclaringTypeName() + "."
             + joinPoint.getSignature().getName());
         logger.info("参数 : " + Arrays.toString(joinPoint.getArgs()));
-//        loggger.info("参数 : " + joinPoint.getArgs());
 
     }
 

novel-admin/src/main/java/com/java2nb/common/config/DruidDBConfig.java

@@ -1,132 +0,0 @@
-package com.java2nb.common.config;
-import com.alibaba.druid.pool.DruidDataSource;
-import com.alibaba.druid.support.http.StatViewServlet;
-import com.alibaba.druid.support.http.WebStatFilter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.boot.web.servlet.FilterRegistrationBean;
-import org.springframework.boot.web.servlet.ServletRegistrationBean;
-import org.springframework.context.annotation.Bean;
-import org.springframework.context.annotation.Configuration;
-import org.springframework.context.annotation.Primary;
-
-import javax.sql.DataSource;
-import java.sql.SQLException;
-
-/**
- * Created by PrimaryKey on 17/2/4.
- */
-@SuppressWarnings("AlibabaRemoveCommentedCode")
-@Configuration
-public class DruidDBConfig {
-    private Logger logger = LoggerFactory.getLogger(DruidDBConfig.class);
-    @Value("${spring.datasource.url}")
-    private String dbUrl;
-
-    @Value("${spring.datasource.username}")
-    private String username;
-
-    @Value("${spring.datasource.password}")
-    private String password;
-
-    @Value("${spring.datasource.driverClassName}")
-    private String driverClassName;
-
-    @Value("${spring.datasource.initialSize}")
-    private int initialSize;
-
-    @Value("${spring.datasource.minIdle}")
-    private int minIdle;
-
-    @Value("${spring.datasource.maxActive}")
-    private int maxActive;
-
-    @Value("${spring.datasource.maxWait}")
-    private int maxWait;
-
-    @Value("${spring.datasource.timeBetweenEvictionRunsMillis}")
-    private int timeBetweenEvictionRunsMillis;
-
-    @Value("${spring.datasource.minEvictableIdleTimeMillis}")
-    private int minEvictableIdleTimeMillis;
-
-    @Value("${spring.datasource.validationQuery}")
-    private String validationQuery;
-
-    @Value("${spring.datasource.testWhileIdle}")
-    private boolean testWhileIdle;
-
-    @Value("${spring.datasource.testOnBorrow}")
-    private boolean testOnBorrow;
-
-    @Value("${spring.datasource.testOnReturn}")
-    private boolean testOnReturn;
-
-    @Value("${spring.datasource.poolPreparedStatements}")
-    private boolean poolPreparedStatements;
-
-    @Value("${spring.datasource.maxPoolPreparedStatementPerConnectionSize}")
-    private int maxPoolPreparedStatementPerConnectionSize;
-
-    @Value("${spring.datasource.filters}")
-    private String filters;
-
-    @Value("{spring.datasource.connectionProperties}")
-    private String connectionProperties;
-
-    @Bean(initMethod = "init", destroyMethod = "close")   //声明其为Bean实例
-    @Primary  //在同样的DataSource中，首先使用被标注的DataSource
-    public DataSource dataSource() {
-        DruidDataSource datasource = new DruidDataSource();
-
-        datasource.setUrl(this.dbUrl);
-        datasource.setUsername(username);
-        datasource.setPassword(password);
-        datasource.setDriverClassName(driverClassName);
-
-        //configuration
-        datasource.setInitialSize(initialSize);
-        datasource.setMinIdle(minIdle);
-        datasource.setMaxActive(maxActive);
-        datasource.setMaxWait(maxWait);
-        datasource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
-        datasource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
-        datasource.setValidationQuery(validationQuery);
-        datasource.setTestWhileIdle(testWhileIdle);
-        datasource.setTestOnBorrow(testOnBorrow);
-        datasource.setTestOnReturn(testOnReturn);
-        datasource.setPoolPreparedStatements(poolPreparedStatements);
-        datasource.setMaxPoolPreparedStatementPerConnectionSize(maxPoolPreparedStatementPerConnectionSize);
-        try {
-            datasource.setFilters(filters);
-        } catch (SQLException e) {
-            logger.error("druid configuration initialization filter", e);
-        }
-        datasource.setConnectionProperties(connectionProperties);
-
-        return datasource;
-    }
-
-    @Bean
-    public ServletRegistrationBean druidServlet() {
-        ServletRegistrationBean reg = new ServletRegistrationBean();
-        reg.setServlet(new StatViewServlet());
-        reg.addUrlMappings("/druid/*");
-        reg.addInitParameter("allow", ""); //白名单
-        return reg;
-    }
-
-    @Bean public FilterRegistrationBean filterRegistrationBean() {
-        FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean();
-        filterRegistrationBean.setFilter(new WebStatFilter());
-        filterRegistrationBean.addUrlPatterns("/*");
-        filterRegistrationBean.addInitParameter("exclusions", "*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*");
-        filterRegistrationBean.addInitParameter("profileEnable", "true");
-        filterRegistrationBean.addInitParameter("principalCookieName","USER_COOKIE");
-        filterRegistrationBean.addInitParameter("principalSessionName","USER_SESSION");
-        filterRegistrationBean.addInitParameter("DruidWebStatFilter","/*");
-        return filterRegistrationBean;
-    }
-}
-

novel-admin/src/main/java/com/java2nb/common/config/Swagger2Config.java

@@ -1,48 +0,0 @@
-package com.java2nb.common.config;
-
-import org.springframework.context.annotation.Bean;
-import org.springframework.context.annotation.Configuration;
-import springfox.documentation.builders.ApiInfoBuilder;
-import springfox.documentation.builders.PathSelectors;
-import springfox.documentation.builders.RequestHandlerSelectors;
-import springfox.documentation.service.ApiInfo;
-import springfox.documentation.service.Contact;
-import springfox.documentation.spi.DocumentationType;
-import springfox.documentation.spring.web.plugins.Docket;
-import springfox.documentation.swagger2.annotations.EnableSwagger2;
-
-/**
- * ${DESCRIPTION}
- *
- * @author xiongxy
- * @create 2019-11-02 23:53
- */
-@EnableSwagger2
-@Configuration
-public class Swagger2Config {
-
-    @Bean
-    public Docket createRestApi() {
-        return new Docket(DocumentationType.SWAGGER_2)
-                .apiInfo(apiInfo())
-                .select()
-                //为当前包路径
-                .apis(RequestHandlerSelectors.any())
-                .paths(PathSelectors.any())
-                .build();
-    }
-
-    //构建 api文档的详细信息函数
-    private ApiInfo apiInfo() {
-        return new ApiInfoBuilder()
-                //页面标题
-                .title("功能测试")
-                //创建人
-                .contact(new Contact("xiongxy", "1179705413@qq.com", "1179705413@qq.com"))
-                //版本号
-                .version("1.0")
-                //描述
-                .description("API 描述")
-                .build();
-    }
-}

novel-admin/src/main/java/com/java2nb/common/dao/GeneratorMapper.java

@@ -9,25 +9,25 @@ import java.util.Map;
 public interface GeneratorMapper {
 
     @Select(
-        "select table_name tableName, engine, table_comment tableComment, create_time createTime from information_schema.tables"
+        "select table_name tableName, engine, table_comment tableComment, create_time createTime from tables"
             + " where table_schema = 'novel_plus' and table_name like concat('%',#{tableName},'%')")
     List<Map<String, Object>> list(@Param("tableName") String tableName);
 
-    @Select("select count(*) from information_schema.tables where table_schema = 'novel_plus'")
+    @Select("select count(*) from tables where table_schema = 'novel_plus'")
     int count(Map<String, Object> map);
 
     @Select(
-        "select table_name tableName, engine, table_comment tableComment, create_time createTime from information_schema.tables \r\n"
+        "select table_name tableName, engine, table_comment tableComment, create_time createTime from tables \r\n"
             + "	where table_schema = 'novel_plus' and table_name = #{tableName}")
     Map<String, String> get(String tableName);
 
     @Select(
-        "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from information_schema.columns\r\n"
+        "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from columns\r\n"
             + " where table_name = #{tableName} and table_schema = 'novel_plus' order by ordinal_position")
     List<Map<String, String>> listColumns(String tableName);
 
     @Select(
-        "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from information_schema.columns\r\n"
+        "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from columns\r\n"
             + " where table_name = #{tableName} and table_schema = 'novel_plus' and column_key = 'PRI' limit 1")
     Map<String, String> getPriColumn(String tableName);
 }

novel-admin/src/main/java/com/java2nb/common/exception/MainsiteErrorController.java

@@ -2,24 +2,19 @@ package com.java2nb.common.exception;
 
 
 import com.java2nb.common.utils.R;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.boot.web.servlet.error.ErrorAttributes;
 import org.springframework.boot.web.servlet.error.ErrorController;
-import org.springframework.http.HttpStatus;
-import org.springframework.stereotype.Controller;
 import org.springframework.web.bind.annotation.RequestMapping;
 import org.springframework.web.bind.annotation.RestController;
 import org.springframework.web.servlet.ModelAndView;
 
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
-import java.util.Map;
 
 @RestController
 public class MainsiteErrorController implements ErrorController {
-    private Logger logger = LoggerFactory.getLogger(getClass());
+
     private static final String ERROR_PATH = "/error";
 
     @Autowired
@@ -58,9 +53,4 @@ public class MainsiteErrorController implements ErrorController {
         }
     }
 
-    @Override
-    public String getErrorPath() {
-        // TODO Auto-generated method stub
-        return ERROR_PATH;
-    }
 }

novel-admin/src/main/java/com/java2nb/common/utils/JSONUtils.java

@@ -1,6 +1,5 @@
 package com.java2nb.common.utils;
 
-import com.alibaba.druid.util.StringUtils;
 import com.alibaba.fastjson.JSON;
 import com.alibaba.fastjson.JSONObject;
 
@@ -8,6 +7,7 @@ import java.util.HashMap;
 import java.util.Map;
 
 public class JSONUtils {
+
     /**
      * Bean对象转JSON
      *

novel-admin/src/main/java/com/java2nb/novel/controller/FriendLinkController.java

@@ -12,6 +12,7 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.data.redis.core.RedisTemplate;
 import org.springframework.stereotype.Controller;
 import org.springframework.ui.Model;
+import org.springframework.validation.annotation.Validated;
 import org.springframework.web.bind.annotation.*;
 
 import java.util.List;
@@ -83,7 +84,7 @@ public class FriendLinkController {
     @ResponseBody
     @PostMapping("/save")
     @RequiresPermissions("novel:friendLink:add")
-    public R save(FriendLinkDO friendLink) {
+    public R save(@Validated FriendLinkDO friendLink) {
         if (friendLinkService.save(friendLink) > 0) {
             redisTemplate.delete(CacheKey.INDEX_LINK_KEY);
             return R.ok();
@@ -98,7 +99,7 @@ public class FriendLinkController {
     @ResponseBody
     @RequestMapping("/update")
     @RequiresPermissions("novel:friendLink:edit")
-    public R update(FriendLinkDO friendLink) {
+    public R update(@Validated FriendLinkDO friendLink) {
         friendLinkService.update(friendLink);
         redisTemplate.delete(CacheKey.INDEX_LINK_KEY);
         return R.ok();

novel-admin/src/main/java/com/java2nb/novel/domain/FriendLinkDO.java

@@ -1,27 +1,21 @@
 package com.java2nb.novel.domain;
 
-import java.io.Serializable;
-
-
-import java.math.BigDecimal;
-
 import com.fasterxml.jackson.databind.annotation.JsonSerialize;
 import com.java2nb.common.jsonserializer.LongToStringSerializer;
-
-
+import org.hibernate.validator.constraints.URL;
 import org.springframework.format.annotation.DateTimeFormat;
+
+import java.io.Serializable;
 import java.util.Date;
 
 
-
 /**
- * 
- * 
  * @author xiongxy
  * @email 1179705413@qq.com
  * @date 2023-04-14 15:12:25
  */
 public class FriendLinkDO implements Serializable {
+
     private static final long serialVersionUID = 1L;
 
 
@@ -30,6 +24,7 @@ public class FriendLinkDO implements Serializable {
     //链接名
     private String linkName;
     //链接url
+    @URL
     private String linkUrl;
     //排序号
     private Integer sort;
@@ -58,102 +53,119 @@ public class FriendLinkDO implements Serializable {
     public void setId(Integer id) {
         this.id = id;
     }
+
     /**
      * 获取：主键
      */
     public Integer getId() {
         return id;
     }
+
     /**
      * 设置：链接名
      */
     public void setLinkName(String linkName) {
         this.linkName = linkName;
     }
+
     /**
      * 获取：链接名
      */
     public String getLinkName() {
         return linkName;
     }
+
     /**
      * 设置：链接url
      */
     public void setLinkUrl(String linkUrl) {
         this.linkUrl = linkUrl;
     }
+
     /**
      * 获取：链接url
      */
     public String getLinkUrl() {
         return linkUrl;
     }
+
     /**
      * 设置：排序号
      */
     public void setSort(Integer sort) {
         this.sort = sort;
     }
+
     /**
      * 获取：排序号
      */
     public Integer getSort() {
         return sort;
     }
+
     /**
      * 设置：是否开启，0：不开启，1：开启
      */
     public void setIsOpen(Integer isOpen) {
         this.isOpen = isOpen;
     }
+
     /**
      * 获取：是否开启，0：不开启，1：开启
      */
     public Integer getIsOpen() {
         return isOpen;
     }
+
     /**
      * 设置：创建人id
      */
     public void setCreateUserId(Long createUserId) {
         this.createUserId = createUserId;
     }
+
     /**
      * 获取：创建人id
      */
     public Long getCreateUserId() {
         return createUserId;
     }
+
     /**
      * 设置：创建时间
      */
     public void setCreateTime(Date createTime) {
         this.createTime = createTime;
     }
+
     /**
      * 获取：创建时间
      */
     public Date getCreateTime() {
         return createTime;
     }
+
     /**
      * 设置：更新者用户id
      */
     public void setUpdateUserId(Long updateUserId) {
         this.updateUserId = updateUserId;
     }
+
     /**
      * 获取：更新者用户id
      */
     public Long getUpdateUserId() {
         return updateUserId;
     }
+
     /**
      * 设置：更新时间
      */
     public void setUpdateTime(Date updateTime) {
         this.updateTime = updateTime;
     }
+
     /**
      * 获取：更新时间
      */

novel-admin/src/main/resources/application-dev.yml

@@ -7,92 +7,11 @@ logging:
     root: info
     com.java2nb: debug
 spring:
-  datasource:
-    type: com.alibaba.druid.pool.DruidDataSource
-    driverClassName: com.mysql.cj.jdbc.Driver
-    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
-    username: root
-    password: test123456
-    #password:
-    initialSize: 1
-    minIdle: 3
-    maxActive: 20
-    # 配置获取连接等待超时的时间
-    maxWait: 60000
-    # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
-    timeBetweenEvictionRunsMillis: 60000
-    # 配置一个连接在池中最小生存的时间，单位是毫秒
-    minEvictableIdleTimeMillis: 30000
-    validationQuery: select 'x'
-    testWhileIdle: true
-    testOnBorrow: false
-    testOnReturn: false
-    # 打开PSCache，并且指定每个连接上PSCache的大小
-    poolPreparedStatements: true
-    maxPoolPreparedStatementPerConnectionSize: 20
-    # 配置监控统计拦截的filters，去掉后监控界面sql无法统计，'wall'用于防火墙
-    filters: stat,slf4j
-    # 通过connectProperties属性来打开mergeSql功能；慢SQL记录
-    connectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
-    # 合并多个DruidDataSource的监控数据
-    #useGlobalDataSourceStat: true
   redis:
     host: 127.0.0.1
     port: 6379
     password: test123456
     # 连接超时时间（毫秒）
     timeout: 10000
-    jedis:
-      pool:
-        # 连接池中的最大空闲连接
-        max-idle: 8
-        # 连接池中的最小空闲连接
-        min-idle: 10
-        # 连接池最大连接数（使用负值表示没有限制）
-        max-active: 100
-        # 连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-wait: -1
 
 
-
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-
-sharding:
-  jdbc:
-    datasource:
-      names: ds0,ds1
-      ds0:
-        type: com.zaxxer.hikari.HikariDataSource
-        driver-class-name: com.mysql.cj.jdbc.Driver
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-      ds1:
-        type: com.alibaba.druid.pool.DruidDataSource
-        driver-class-name: com.mysql.jdbc.Driver
-        url: jdbc:mysql://localhost:3306/information_schema?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-    config:
-      sharding:
-        props:
-          sql.show: true
-        tables:
-          book_content: #book_content表
-            key-generator-column-name: id  #主键
-            actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
-            #            database-strategy:  #分库策略
-            #              inline:
-            #                sharding-column: book_id
-            #                algorithm-expression: ds${book_id % 10}
-            table-strategy: #分表策略
-              inline:
-                shardingColumn: index_id
-                algorithm-expression: book_content${index_id % 10}
-          tables:
-            actual-data-nodes: ds${1}.tables
-          columns:
-            actual-data-nodes: ds${1}.columns
-        default-data-source-name: ds0

novel-admin/src/main/resources/application-prod.yml

@@ -7,86 +7,11 @@ logging:
     root: error
     com.java2nb: error
 spring:
-  datasource:
-    type: com.alibaba.druid.pool.DruidDataSource
-    driverClassName: com.mysql.cj.jdbc.Driver
-    url: jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
-    username: root
-    password: test123456
-    #password:
-    initialSize: 1
-    minIdle: 3
-    maxActive: 20
-    # 配置获取连接等待超时的时间
-    maxWait: 60000
-    # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
-    timeBetweenEvictionRunsMillis: 60000
-    # 配置一个连接在池中最小生存的时间，单位是毫秒
-    minEvictableIdleTimeMillis: 30000
-    validationQuery: select 'x'
-    testWhileIdle: true
-    testOnBorrow: false
-    testOnReturn: false
-    # 打开PSCache，并且指定每个连接上PSCache的大小
-    poolPreparedStatements: true
-    maxPoolPreparedStatementPerConnectionSize: 20
-    # 配置监控统计拦截的filters，去掉后监控界面sql无法统计，'wall'用于防火墙
-    filters: stat,slf4j
-    # 通过connectProperties属性来打开mergeSql功能；慢SQL记录
-    connectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
-    # 合并多个DruidDataSource的监控数据
-    #useGlobalDataSourceStat: true
   redis:
     host: 127.0.0.1
     port: 6379
-    password: test
+    password: test123456
     # 连接超时时间（毫秒）
     timeout: 10000
-    jedis:
-      pool:
-        # 连接池中的最大空闲连接
-        max-idle: 8
-        # 连接池中的最小空闲连接
-        min-idle: 10
-        # 连接池最大连接数（使用负值表示没有限制）
-        max-active: 100
-        # 连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-wait: -1
 
 
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-
-sharding:
-  jdbc:
-    datasource:
-      names: ds0 #,ds1
-      ds0:
-        type: com.zaxxer.hikari.HikariDataSource
-        driver-class-name: com.mysql.cj.jdbc.Driver
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-    #      ds1:
-    #        type: com.alibaba.druid.pool.DruidDataSource
-    #        driver-class-name: com.mysql.jdbc.Driver
-    #        url: jdbc:mysql://localhost:3306/novel_plus2
-    #        username: root
-    #        password: test123456
-    config:
-      sharding:
-        props:
-          sql.show: true
-        tables:
-          book_content: #book_content表
-            key-generator-column-name: id  #主键
-            actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
-            #            database-strategy:  #分库策略
-            #              inline:
-            #                sharding-column: book_id
-            #                algorithm-expression: ds${book_id % 10}
-            table-strategy: #分表策略
-              inline:
-                shardingColumn: index_id
-                algorithm-expression: book_content${index_id % 10}

novel-admin/src/main/resources/application.yml

@@ -9,6 +9,9 @@ server:
 #  basic:
 #    enabled: false
 spring:
+  datasource:
+    driverClassName: org.apache.shardingsphere.driver.ShardingSphereDriver
+    url: jdbc:shardingsphere:absolutepath:${user.dir}/config/shardingsphere-jdbc.yml
   thymeleaf:
     mode: LEGACYHTML5
     cache: false
@@ -18,14 +21,12 @@ spring:
   profiles:
     active: dev
   
-  #上传文件的最大值（10M）
+  #上传文件的最大值（100M）
   servlet:
     multipart:
-      max-file-size: 10485760
+      max-file-size: 100MB
+      max-request-size: 100MB
 
-  devtools:
-    restart:
-      enabled: true
   main:
     allow-bean-definition-overriding: true
 
@@ -35,9 +36,6 @@ mybatis:
     map-underscore-to-camel-case: true
   mapper-locations: mybatis/**/*Mapper.xml
   typeAliasesPackage: com.java2nb.**.domain
-#[弃用]配置缓存和session存储方式，默认ehcache,可选redis,[弃用]调整至 spring cache type【shiro.用户，权限，session，spring.cache通用】
-#[弃用]cacheType: ehcache
-
 
 logging:
   config: classpath:logback-boot.xml

novel-admin/src/main/resources/static/js/appjs/novel/websiteInfo/edit.js

@@ -79,7 +79,7 @@ function update() {
         },
         success: function (data) {
             if (data.code == 0) {
-                layer.msg("操作成功");
+                layer.msg("操作成功，重启 novel-front 后生效");
             } else {
                 layer.alert(data.msg)
             }

novel-admin/src/main/resources/templates/common/file/file.html

@@ -6,7 +6,7 @@
     <title>小说精品屋 - 文件管理器</title>
     <meta name="keywords" content="">
     <meta name="description" content="">
-    <link rel="shortcut icon" href="favicon.ico">
+    <link rel="shortcut icon" href="/favicon.ico">
     <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
     <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
     <link href="/css/animate.css" rel="stylesheet">

novel-admin/src/main/resources/templates/error/403.html

@@ -6,7 +6,7 @@
 <title>403 页面</title>
 <meta name="keywords" content="">
 <meta name="description" content="">
-<link rel="shortcut icon" href="favicon.ico">
+<link rel="shortcut icon" href="/favicon.ico">
 <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
 <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
 <link href="/css/animate.css" rel="stylesheet">

novel-admin/src/main/resources/templates/error/404.html

@@ -10,7 +10,7 @@
     <meta name="keywords" content="">
     <meta name="description" content="">
 
-    <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
+    <link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
     <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
 
     <link href="/css/animate.css" rel="stylesheet">

novel-admin/src/main/resources/templates/error/500.html

@@ -11,7 +11,7 @@
     <meta name="keywords" content="">
     <meta name="description" content="">
 
-    <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
+    <link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
     <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
 
     <link href="/css/animate.css" rel="stylesheet">

novel-admin/src/main/resources/templates/error/error.html

@@ -6,7 +6,7 @@
     <title>500错误</title>
     <meta name="keywords" content="">
     <meta name="description" content="">
-    <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
+    <link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
     <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
     <link href="/css/animate.css" rel="stylesheet">
     <link href="/css/style.css?v=4.1.0" rel="stylesheet">

novel-admin/src/main/resources/templates/include.html

@@ -4,7 +4,7 @@
 	<title></title>
 	<meta name="keywords" content="">
 	<meta name="description" content="">
-	<link rel="shortcut icon" href="favicon.ico">
+	<link rel="shortcut icon" href="/favicon.ico">
 	<link href="css/bootstrap.min.css?v=3.3.6"
 		  th:href="@{/css/bootstrap.min.css?v=3.3.6}" rel="stylesheet">
 	<link href="/css/font-awesome.css?v=4.4.0"

novel-admin/src/main/resources/templates/index.html

@@ -10,7 +10,7 @@
     <!--[if lt IE 9]>
     <meta http-equiv="refresh" content="0;ie.html"/>
     <![endif]-->
-    <link rel="shortcut icon" href="favicon.ico">
+    <link rel="shortcut icon" href="/favicon.ico">
     <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
     <link href="/css/font-awesome.min.css?v=4.4.0" rel="stylesheet">
     <link href="/css/plugins/toastr/toastr.min.css" rel="stylesheet">

novel-admin/src/main/resources/templates/login.html

@@ -2,7 +2,7 @@
 <html>
 <head>
     <meta charset="UTF-8">
-    <title>后台管理-登陆</title>
+    <title>后台管理-登录</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
     <meta http-equiv="Access-Control-Allow-Origin" content="*">
     <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

novel-admin/src/main/resources/templates/novel/friendLink/add.html

@@ -23,7 +23,7 @@
                             <div class="col-sm-8">
                                 <input id="linkUrl" name="linkUrl"
                                        class="form-control"
-                                       type="text" required>
+                                       type="url" required>
 
                             </div>
                         </div>

novel-admin/src/main/resources/templates/novel/friendLink/edit.html

@@ -26,7 +26,7 @@
                                 <input id="linkUrl" name="linkUrl"
                                        th:value="${friendLink.linkUrl}"
                                        class="form-control"
-                                       type="text" required>
+                                       type="url" required>
                             </div>
                         </div>
                         <div class="form-group">

novel-admin/src/main/resources/templates/system/menu/edit.html

@@ -11,7 +11,7 @@
 <meta name="keywords" content="">
 <meta name="description" content="">
 
-<link rel="shortcut icon" href="favicon.ico">
+<link rel="shortcut icon" href="/favicon.ico">
 <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
 <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
 <link href="/css/animate.css" rel="stylesheet">

novel-admin/src/main/resources/templates/system/user/include.html

@@ -4,7 +4,7 @@
 	<title></title>
 	<meta name="keywords" content="">
 	<meta name="description" content="">
-	<link rel="shortcut icon" href="favicon.ico">
+	<link rel="shortcut icon" href="/favicon.ico">
 	<link rel="stylesheet" href="/css/bootstrap.min.css" />
 	<link rel="stylesheet" href="/css/animate.css" />
 	<link rel="stylesheet" href="/css/font-awesome.css" />

novel-common/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>4.2.0</version>
+        <version>5.2.1</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -52,18 +52,16 @@
         </dependency>
 
         <!-- 分库分表-->
-        <!-- sharding jdbc依赖 -->
-
+        <!-- ShardingSphere-JDBC -->
         <dependency>
-            <groupId>io.shardingsphere</groupId>
-            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
-            <version>${sharding.jdbc.version}</version>
+            <groupId>org.apache.shardingsphere</groupId>
+            <artifactId>shardingsphere-jdbc</artifactId>
+            <version>${shardingsphere-jdbc.version}</version>
         </dependency>
-
         <dependency>
-            <groupId>io.shardingsphere</groupId>
-            <artifactId>sharding-jdbc-spring-namespace</artifactId>
-            <version>${sharding.jdbc.version}</version>
+            <groupId>com.h2database</groupId>
+            <artifactId>h2</artifactId>
+            <scope>runtime</scope>
         </dependency>
 
 
@@ -73,11 +71,6 @@
             <artifactId>pagehelper-spring-boot-starter</artifactId>
             <version>${pagehelper.version}</version>
         </dependency>
-        <dependency>
-            <groupId>com.cuisongliu</groupId>
-            <artifactId>orderbyhelper-spring-boot-starter</artifactId>
-            <version>${orderbyhelper.version}</version>
-        </dependency>
 
         <dependency>
             <groupId>org.apache.commons</groupId>
@@ -86,15 +79,8 @@
         </dependency>
         <!--httpclient-->
         <dependency>
-            <groupId>org.apache.httpcomponents</groupId>
-            <artifactId>httpclient</artifactId>
-            <version>4.5.14</version>
-            <exclusions>
-                <exclusion>
-                    <groupId>commons-logging</groupId>
-                    <artifactId>commons-logging</artifactId>
-                </exclusion>
-            </exclusions>
+            <groupId>org.apache.httpcomponents.client5</groupId>
+            <artifactId>httpclient5</artifactId>
         </dependency>
 
         <dependency>
@@ -117,6 +103,12 @@
             <optional>true</optional>
         </dependency>
 
+        <!-- 请求参数校验相关 -->
+        <dependency>
+            <groupId>org.hibernate.validator</groupId>
+            <artifactId>hibernate-validator</artifactId>
+        </dependency>
+
         <dependency>
             <groupId>io.github.xxyopen</groupId>
             <artifactId>xxy-model</artifactId>
@@ -138,5 +130,29 @@
 
     </dependencies>
 
+    <build>
+        <plugins>
+            <plugin>
+                <artifactId>maven-antrun-plugin</artifactId>
+                <version>1.8</version>
+                <executions>
+                    <execution>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>run</goal>
+                        </goals>
+                        <configuration>
+                            <tasks>
+                                <zip destfile='${project.build.directory}/build/sql.zip'>
+                                    <zipfileset filemode="755" dir='${basedir}/../doc/sql'/>
+                                </zip>
+                            </tasks>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+
 
 </project>

novel-common/src/main/java/com/java2nb/novel/core/advice/CommonExceptionHandler.java

@@ -0,0 +1,55 @@
+package com.java2nb.novel.core.advice;
+
+import io.github.xxyopen.model.resp.RestResult;
+import io.github.xxyopen.model.resp.SysResultCode;
+import io.github.xxyopen.web.exception.BusinessException;
+import jakarta.servlet.http.HttpServletRequest;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.http.MediaType;
+import org.springframework.validation.BindException;
+import org.springframework.web.bind.annotation.ExceptionHandler;
+import org.springframework.web.bind.annotation.RestControllerAdvice;
+import org.springframework.web.servlet.ModelAndView;
+
+/**
+ * 统一异常处理器
+ *
+ * @author xiongxiaoyang
+ */
+@Slf4j
+@RestControllerAdvice
+public class CommonExceptionHandler {
+
+    public CommonExceptionHandler() {
+    }
+
+    @ExceptionHandler({BindException.class})
+    public RestResult<Void> handlerBindException(BindException e) {
+        log.error(e.getMessage(), e);
+        return RestResult.fail(SysResultCode.PARAM_ERROR);
+    }
+
+    @ExceptionHandler({BusinessException.class})
+    public RestResult<Void> handlerBusinessException(BusinessException e) {
+        log.error(e.getMessage(), e);
+        return RestResult.fail(e.getResultCode());
+    }
+
+    @ExceptionHandler(Exception.class)
+    public Object handleException(HttpServletRequest request, Exception e) {
+        log.error(e.getMessage(), e);
+        if (isJsonRequest(request)) {
+            // 如果是REST请求，返回JSON格式的错误响应
+            return RestResult.error();
+        } else {
+            //跳转页面过程中出现异常时统一跳转到404页面
+            return new ModelAndView("404");
+        }
+    }
+
+    private boolean isJsonRequest(HttpServletRequest request) {
+        String acceptHeader = request.getHeader("Accept");
+        return acceptHeader != null && acceptHeader.contains(MediaType.APPLICATION_JSON_VALUE);
+    }
+
+}

novel-common/src/main/java/com/java2nb/novel/core/advice/CustomResponseBodyAdvice.java

@@ -0,0 +1,53 @@
+package com.java2nb.novel.core.advice;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.module.SimpleModule;
+import com.fasterxml.jackson.databind.ser.std.ToStringSerializer;
+import org.springframework.core.MethodParameter;
+import org.springframework.http.MediaType;
+import org.springframework.http.converter.HttpMessageConverter;
+import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;
+import org.springframework.http.server.ServerHttpRequest;
+import org.springframework.http.server.ServerHttpResponse;
+import org.springframework.web.bind.annotation.RestControllerAdvice;
+import org.springframework.web.servlet.mvc.method.annotation.ResponseBodyAdvice;
+
+import java.util.Objects;
+
+/**
+ * 在对 RestController 返回对象 json 序列化时，将所有 Long 类型转为 String 类型返回，避免前端数据精度丢失的问题
+ * 取代 spring.jackson.generator.write-numbers-as-strings=true 配置，避免影响全局的 ObjectMapper
+ *
+ * @author xiongxiaoyang
+ * */
+@RestControllerAdvice
+public class CustomResponseBodyAdvice implements ResponseBodyAdvice<Object> {
+
+    private final ObjectMapper customObjectMapper;
+
+    public CustomResponseBodyAdvice(Jackson2ObjectMapperBuilder builder) {
+        customObjectMapper = builder.createXmlMapper(false).build();
+        SimpleModule simpleModule = new SimpleModule();
+        simpleModule.addSerializer(Long.class, ToStringSerializer.instance);
+        simpleModule.addSerializer(Long.TYPE, ToStringSerializer.instance);
+        customObjectMapper.registerModule(simpleModule);
+    }
+
+    @Override
+    public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
+        // 返回 true 表示对所有 Controller 的响应都生效
+        return true;
+    }
+
+    @Override
+    public Object beforeBodyWrite(Object body, MethodParameter returnType, MediaType selectedContentType, Class<? extends HttpMessageConverter<?>> selectedConverterType, ServerHttpRequest request, ServerHttpResponse response) {
+        // 使用自定义的 ObjectMapper 序列化响应体
+        if(Objects.nonNull(body)) {
+            return customObjectMapper.valueToTree(body);
+        }else{
+            return null;
+        }
+    }
+
+}
+

novel-common/src/main/java/com/java2nb/novel/core/advice/PageExceptionHandler.java

@@ -1,26 +0,0 @@
-package com.java2nb.novel.core.advice;
-
-import lombok.extern.slf4j.Slf4j;
-import org.springframework.web.bind.annotation.ControllerAdvice;
-import org.springframework.web.bind.annotation.ExceptionHandler;
-
-/**
- * 页面异常处理器
- *
- * @author 11797
- */
-@Slf4j
-@ControllerAdvice(basePackages = "com.java2nb.novel.controller.page")
-public class PageExceptionHandler {
-
-
-    /**
-     * 处理所有异常
-     */
-    @ExceptionHandler(Exception.class)
-    public String handlerException(Exception e) {
-        log.error(e.getMessage(), e);
-        //跳转页面过程中出现异常时统一跳转到404页面
-        return "404";
-    }
-}

novel-common/src/main/java/com/java2nb/novel/core/cache/CacheKey.java

@@ -41,11 +41,6 @@ public interface CacheKey {
      * */
     String TEMPLATE_DIR_KEY =  "templateDirKey";;
 
-    /**
-     * 正在运行的爬虫线程存储KEY前缀
-     * */
-    String RUNNING_CRAWL_THREAD_KEY_PREFIX = "runningCrawlTreadDataKeyPrefix";
-
     /**
      * 上一次搜索引擎更新的时间
      * */
@@ -69,4 +64,8 @@ public interface CacheKey {
      * 测试爬虫规则缓存
      */
     String BOOK_TEST_PARSE = "testParse";
+    /**
+     * AI生成图片
+     * */
+    String AI_GEN_PIC = "aiGenPic";
 }

novel-common/src/main/java/com/java2nb/novel/core/config/HttpProxyProperties.java

@@ -19,4 +19,8 @@ public class HttpProxyProperties {
 
     private Integer port;
 
+    private String username;
+
+    private String password;
+
 }

novel-common/src/main/java/com/java2nb/novel/core/enums/ResponseStatus.java

@@ -18,7 +18,7 @@ public enum ResponseStatus implements IResultCode {
     /**
      * 用户相关错误
      * */
-   NO_LOGIN(1001, "未登录或登陆失效！"),
+   NO_LOGIN(1001, "未登录或登录失效！"),
     VEL_CODE_ERROR(1002, "验证码错误！"),
     USERNAME_EXIST(1003,"该手机号已注册！"),
     USERNAME_PASS_ERROR(1004,"手机号或密码错误！"),

novel-common/src/main/java/com/java2nb/novel/core/utils/DateUtil.java

@@ -106,6 +106,47 @@ public class DateUtil {
     }
 
 
+    /**
+     * 将日期格式化成"多久之前"的格式
+     * */
+    public static String formatTimeAgo(Date date){
+        if (date == null) {
+            return null;
+        }
+
+        long now = new Date().getTime();
+        long then = date.getTime();
+
+        long diff = now - then;
+
+        if (diff < 0) {
+            // 未来时间
+            DateUtil.formatDate(date, DateUtil.DATE_TIME_PATTERN);
+        }
+
+        long seconds = diff / 1000;
+        long minutes = seconds / 60;
+        long hours = minutes / 60;
+        long days = hours / 24;
+        long months = days / 30;
+        long years = months / 12;
+
+        if (seconds < 60) {
+            return "刚刚";
+        } else if (minutes < 60) {
+            return minutes + "分钟前";
+        } else if (hours < 24) {
+            return hours + "小时前";
+        } else if (days < 30) {
+            return days + "天前";
+        } else if (months < 12) {
+            return months + "个月前";
+        } else {
+            return years + "年前";
+        }
+    }
+
+
     public static void main(String[] args) {
         System.out.println(formatDate(getYesterday(),DATE_TIME_PATTERN));
         System.out.println(formatDate(getDateStartTime(getYesterday()),DATE_TIME_PATTERN));

novel-common/src/main/java/com/java2nb/novel/core/utils/FileUtil.java

@@ -5,7 +5,7 @@ import lombok.SneakyThrows;
 import lombok.experimental.UtilityClass;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.codec.Charsets;
-import org.apache.http.client.utils.DateUtils;
+import org.apache.hc.client5.http.utils.DateUtils;
 import org.springframework.core.io.Resource;
 import org.springframework.http.HttpEntity;
 import org.springframework.http.HttpHeaders;
@@ -14,7 +14,15 @@ import org.springframework.http.ResponseEntity;
 
 import javax.imageio.ImageIO;
 import java.awt.image.BufferedImage;
-import java.io.*;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.StandardCopyOption;
 import java.util.Date;
 import java.util.Objects;
 
@@ -37,10 +45,13 @@ public class FileUtil {
             //本地图片保存
             HttpHeaders headers = new HttpHeaders();
             HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
-            ResponseEntity<Resource> resEntity = RestTemplateUtil.getInstance(Charsets.ISO_8859_1.name()).exchange(picSrc, HttpMethod.GET, requestEntity, Resource.class);
+            ResponseEntity<Resource> resEntity = RestTemplates.newInstance(Charsets.ISO_8859_1.name())
+                .exchange(picSrc, HttpMethod.GET, requestEntity, Resource.class);
             input = Objects.requireNonNull(resEntity.getBody()).getInputStream();
             Date currentDate = new Date();
-            picSrc = visitPrefix + DateUtils.formatDate(currentDate, "yyyy") + "/" + DateUtils.formatDate(currentDate, "MM") + "/" + DateUtils.formatDate(currentDate, "dd") + "/"
+            picSrc =
+                visitPrefix + DateUtils.formatDate(currentDate, "yyyy") + "/" + DateUtils.formatDate(currentDate, "MM")
+                    + "/" + DateUtils.formatDate(currentDate, "dd") + "/"
                     + UUIDUtil.getUUID32()
                     + picSrc.substring(picSrc.lastIndexOf("."));
             File picFile = new File(picSavePath + picSrc);
@@ -67,7 +78,6 @@ public class FileUtil {
             closeStream(input, out);
         }
 
-
         return picSrc;
     }
 
@@ -120,5 +130,23 @@ public class FileUtil {
 
     }
 
+    /**
+     * 下载文件
+     *
+     * @param downloadUrl 下载的URL
+     * @param savePath    保存的路径
+     */
+    @SneakyThrows
+    public void downloadFile(String downloadUrl, String savePath) {
+        Path path = Paths.get(savePath);
+        Path parentPath = path.getParent();
+        if (Files.notExists(parentPath)) {
+            Files.createDirectories(parentPath);
+        }
+        URL url = new URL(downloadUrl);
+        try (InputStream in = url.openStream()) {
+            Files.copy(in, path, StandardCopyOption.REPLACE_EXISTING);
+        }
+    }
 
 }

novel-common/src/main/java/com/java2nb/novel/core/utils/HttpUtil.java

@@ -1,47 +1,52 @@
 package com.java2nb.novel.core.utils;
 
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.http.*;
 import org.springframework.web.client.RestTemplate;
 
+import java.nio.charset.Charset;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
 /**
  * @author Administrator
  */
+@Slf4j
 public class HttpUtil {
 
-    private static RestTemplate restTemplate = RestTemplateUtil.getInstance("utf-8");
+    private static final String DEFAULT_CHARSET = "utf-8";
 
+    private static final Map<String, RestTemplate> REST_TEMPLATE_MAP = new ConcurrentHashMap<>();
 
-    public static String getByHttpClient(String url) {
+    public static String getByHttpClientWithChrome(String url, String charset) {
+        log.debug("Get url：{}", url);
+        if (!Charset.isSupported(charset)) {
+            log.error("字符编码{}无效！", charset);
+            return null;
+        }
+        RestTemplate restTemplate = REST_TEMPLATE_MAP.computeIfAbsent(charset,
+            k -> RestTemplates.newInstance(charset));
         try {
-
-            ResponseEntity<String> forEntity = restTemplate.getForEntity(url, String.class);
+            HttpHeaders headers = new HttpHeaders();
+            headers.add("user-agent",
+                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36");
+            HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
+            ResponseEntity<String> forEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity,
+                String.class);
+            log.debug("Response code：{}", forEntity.getStatusCode());
             if (forEntity.getStatusCode() == HttpStatus.OK) {
                 return forEntity.getBody();
             } else {
                 return null;
             }
         } catch (Exception e) {
-            e.printStackTrace();
+            log.error(e.getMessage(), e);
             return null;
         }
     }
 
     public static String getByHttpClientWithChrome(String url) {
-        try {
+        return getByHttpClientWithChrome(url, DEFAULT_CHARSET);
+    }
 
-            HttpHeaders headers = new HttpHeaders();
-            headers.add("user-agent","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36");
-            HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
-            ResponseEntity<String> forEntity = restTemplate.exchange(url.toString(), HttpMethod.GET, requestEntity, String.class);
-
-            if (forEntity.getStatusCode() == HttpStatus.OK) {
-                return forEntity.getBody();
-            } else {
-                return null;
-            }
-        } catch (Exception e) {
-            e.printStackTrace();
-            return null;
-        }
-    }
 }

novel-common/src/main/java/com/java2nb/novel/core/utils/IpUtil.java

@@ -1,11 +1,21 @@
 package com.java2nb.novel.core.utils;
 
-import javax.servlet.http.HttpServletRequest;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import jakarta.servlet.http.HttpServletRequest;
+import lombok.extern.slf4j.Slf4j;
 
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.time.Duration;
+
+@Slf4j
 public class IpUtil {
 
     /**
      * 获取真实IP
+     *
      * @param request 请求体
      * @return 真实IP
      */
@@ -31,4 +41,27 @@ public class IpUtil {
         }
         return ip;
     }
+
+    /**
+     * 获取本机公网IP
+     */
+    public static String getPublicIP() {
+        try {
+            HttpClient client = HttpClient.newHttpClient();
+            HttpRequest request = HttpRequest.newBuilder()
+                .uri(URI.create("https://httpbin.org/ip"))
+                .GET()
+                .timeout(Duration.ofSeconds(5))
+                .build();
+
+            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
+
+            if (response.statusCode() == 200) {
+                return new ObjectMapper().readTree(response.body()).get("origin").asText();
+            }
+        } catch (Exception e) {
+            log.error("获取本机公网IP异常", e);
+        }
+        return null;
+    }
 }

novel-common/src/main/java/com/java2nb/novel/core/utils/RestTemplates.java

@@ -2,17 +2,22 @@ package com.java2nb.novel.core.utils;
 
 import com.java2nb.novel.core.config.HttpProxyProperties;
 import lombok.SneakyThrows;
-import org.apache.http.HttpHost;
-import org.apache.http.config.Registry;
-import org.apache.http.config.RegistryBuilder;
-import org.apache.http.conn.socket.ConnectionSocketFactory;
-import org.apache.http.conn.socket.PlainConnectionSocketFactory;
-import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
-import org.apache.http.conn.ssl.TrustStrategy;
-import org.apache.http.impl.client.CloseableHttpClient;
-import org.apache.http.impl.client.HttpClientBuilder;
-import org.apache.http.impl.client.HttpClients;
-import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hc.client5.http.auth.AuthScope;
+import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
+import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
+import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
+import org.apache.hc.client5.http.impl.classic.HttpClientBuilder;
+import org.apache.hc.client5.http.impl.classic.HttpClients;
+import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager;
+import org.apache.hc.client5.http.socket.ConnectionSocketFactory;
+import org.apache.hc.client5.http.socket.PlainConnectionSocketFactory;
+import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory;
+import org.apache.hc.core5.http.HttpHost;
+import org.apache.hc.core5.http.config.Registry;
+import org.apache.hc.core5.http.config.RegistryBuilder;
+import org.apache.hc.core5.ssl.SSLContexts;
+import org.apache.hc.core5.ssl.TrustStrategy;
 import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
 import org.springframework.http.converter.HttpMessageConverter;
 import org.springframework.http.converter.StringHttpMessageConverter;
@@ -26,21 +31,21 @@ import java.util.List;
 import java.util.Objects;
 
 @Component
-public class RestTemplateUtil {
+public class RestTemplates {
 
     private static HttpProxyProperties httpProxyProperties;
 
-    RestTemplateUtil(HttpProxyProperties properties) {
+    RestTemplates(HttpProxyProperties properties) {
         httpProxyProperties = properties;
     }
 
     @SneakyThrows
-    public static RestTemplate getInstance(String charset) {
+    public static RestTemplate newInstance(String charset) {
 
         TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
 
         //忽略证书
-        SSLContext sslContext = org.apache.http.ssl.SSLContexts.custom()
+        SSLContext sslContext = SSLContexts.custom()
             .loadTrustMaterial(null, acceptingTrustStrategy)
             .build();
 
@@ -61,6 +66,15 @@ public class RestTemplateUtil {
         if (Objects.nonNull(httpProxyProperties) && Boolean.TRUE.equals(httpProxyProperties.getEnabled())) {
             HttpHost proxy = new HttpHost(httpProxyProperties.getIp(), httpProxyProperties.getPort());
             clientBuilder.setProxy(proxy);
+            if (StringUtils.isNotBlank(httpProxyProperties.getUsername()) && StringUtils.isNotBlank(
+                httpProxyProperties.getPassword())) {
+                // 创建CredentialsProvider实例并添加代理认证信息
+                BasicCredentialsProvider provider = new BasicCredentialsProvider();
+                UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(
+                    httpProxyProperties.getUsername(), httpProxyProperties.getPassword().toCharArray());
+                provider.setCredentials(new AuthScope(null, -1), credentials);
+                clientBuilder.setDefaultCredentialsProvider(provider);
+            }
         }
         CloseableHttpClient httpClient = clientBuilder.setConnectionManager(connectionManager)
             .build();

novel-common/src/main/java/com/java2nb/novel/core/utils/SpringUtil.java

@@ -0,0 +1,37 @@
+package com.java2nb.novel.core.utils;
+
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.ApplicationContextAware;
+import org.springframework.stereotype.Component;
+
+@Component
+public class SpringUtil implements ApplicationContextAware {
+    private static ApplicationContext applicationContext;
+
+    public SpringUtil() {
+    }
+
+    public void setApplicationContext(ApplicationContext applicationContext) {
+        if (SpringUtil.applicationContext == null) {
+            SpringUtil.applicationContext = applicationContext;
+        }
+
+    }
+
+    public static ApplicationContext getApplicationContext() {
+        return applicationContext;
+    }
+
+    public static Object getBean(String name) {
+        return getApplicationContext().getBean(name);
+    }
+
+    public static <T> T getBean(Class<T> clazz) {
+        return (T)getApplicationContext().getBean(clazz);
+    }
+
+    public static <T> T getBean(String name, Class<T> clazz) {
+        return (T)getApplicationContext().getBean(name, clazz);
+    }
+}
+

novel-common/src/main/java/com/java2nb/novel/entity/BookComment.java

@@ -14,6 +14,9 @@ public class BookComment {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private String commentContent;
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private String location;
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private Integer replyCount;
 
@@ -56,6 +59,16 @@ public class BookComment {
         this.commentContent = commentContent == null ? null : commentContent.trim();
     }
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public String getLocation() {
+        return location;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setLocation(String location) {
+        this.location = location == null ? null : location.trim();
+    }
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public Integer getReplyCount() {
         return replyCount;

novel-common/src/main/java/com/java2nb/novel/entity/BookCommentReply.java

@@ -0,0 +1,97 @@
+package com.java2nb.novel.entity;
+
+import java.util.Date;
+import javax.annotation.Generated;
+
+public class BookCommentReply {
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Long id;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Long commentId;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private String replyContent;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private String location;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Byte auditStatus;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Date createTime;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Long createUserId;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Long getId() {
+        return id;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setId(Long id) {
+        this.id = id;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Long getCommentId() {
+        return commentId;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setCommentId(Long commentId) {
+        this.commentId = commentId;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public String getReplyContent() {
+        return replyContent;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setReplyContent(String replyContent) {
+        this.replyContent = replyContent == null ? null : replyContent.trim();
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public String getLocation() {
+        return location;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setLocation(String location) {
+        this.location = location == null ? null : location.trim();
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Byte getAuditStatus() {
+        return auditStatus;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setAuditStatus(Byte auditStatus) {
+        this.auditStatus = auditStatus;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Date getCreateTime() {
+        return createTime;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setCreateTime(Date createTime) {
+        this.createTime = createTime;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Long getCreateUserId() {
+        return createUserId;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setCreateUserId(Long createUserId) {
+        this.createUserId = createUserId;
+    }
+}

novel-common/src/main/java/com/java2nb/novel/entity/CrawlSingleTask.java

@@ -31,6 +31,9 @@ public class CrawlSingleTask {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private Byte excCount;
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    private Integer crawlChapters;
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private Date createTime;
 
@@ -124,6 +127,16 @@ public class CrawlSingleTask {
         this.excCount = excCount;
     }
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public Integer getCrawlChapters() {
+        return crawlChapters;
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public void setCrawlChapters(Integer crawlChapters) {
+        this.crawlChapters = crawlChapters;
+    }
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public Date getCreateTime() {
         return createTime;

novel-common/src/main/java/com/java2nb/novel/entity/User.java

@@ -2,9 +2,9 @@ package com.java2nb.novel.entity;
 
 import io.github.xxyopen.web.valid.AddGroup;
 import io.github.xxyopen.web.valid.UpdateGroup;
+import jakarta.validation.constraints.*;
 
 import javax.annotation.Generated;
-import javax.validation.constraints.*;
 import java.util.Date;
 
 public class User {
@@ -16,6 +16,7 @@ public class User {
 
     @NotBlank(groups = {AddGroup.class}, message = "手机号不能为空！")
     @Pattern(groups = {AddGroup.class}, regexp = "^1[3|4|5|6|7|8|9][0-9]{9}$", message = "手机号格式不正确！")
+    @Null(groups = {UpdateGroup.class})
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private String username;
 
@@ -25,10 +26,14 @@ public class User {
     private String password;
 
     @Null(groups = {AddGroup.class})
+    @Pattern(groups = {
+        UpdateGroup.class}, regexp = "[\u4E00-\u9FA5A-Za-z0-9_]{1,11}", message = "昵称格式不正确！")
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private String nickName;
 
     @Null(groups = {AddGroup.class})
+    @Pattern(groups = {
+        UpdateGroup.class}, regexp = "^/localPic/\\d{4}/\\d{2}/\\d{2}/[A-Za-z0-9]+\\.(jpg|jpeg|swf|gif|png|JPG|JPEG|SWF|GIF|PNG)$", message = "只能上传图片格式的文件！")
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     private String userPhoto;
 

novel-common/src/main/java/com/java2nb/novel/mapper/BookCommentDynamicSqlSupport.java

@@ -19,6 +19,9 @@ public final class BookCommentDynamicSqlSupport {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public static final SqlColumn<String> commentContent = bookComment.commentContent;
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<String> location = bookComment.location;
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public static final SqlColumn<Integer> replyCount = bookComment.replyCount;
 
@@ -39,6 +42,8 @@ public final class BookCommentDynamicSqlSupport {
 
         public final SqlColumn<String> commentContent = column("comment_content", JDBCType.VARCHAR);
 
+        public final SqlColumn<String> location = column("location", JDBCType.VARCHAR);
+
         public final SqlColumn<Integer> replyCount = column("reply_count", JDBCType.INTEGER);
 
         public final SqlColumn<Byte> auditStatus = column("audit_status", JDBCType.TINYINT);

novel-common/src/main/java/com/java2nb/novel/mapper/BookCommentMapper.java

@@ -29,7 +29,7 @@ import static org.mybatis.dynamic.sql.SqlBuilder.isEqualTo;
 @Mapper
 public interface BookCommentMapper {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
-    BasicColumn[] selectList = BasicColumn.columnList(id, bookId, commentContent, replyCount, auditStatus, createTime, createUserId);
+    BasicColumn[] selectList = BasicColumn.columnList(id, bookId, commentContent, location, replyCount, auditStatus, createTime, createUserId);
 
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     @SelectProvider(type=SqlProviderAdapter.class, method="select")
@@ -58,6 +58,7 @@ public interface BookCommentMapper {
         @Result(column="id", property="id", jdbcType=JdbcType.BIGINT, id=true),
         @Result(column="book_id", property="bookId", jdbcType=JdbcType.BIGINT),
         @Result(column="comment_content", property="commentContent", jdbcType=JdbcType.VARCHAR),
+        @Result(column="location", property="location", jdbcType=JdbcType.VARCHAR),
         @Result(column="reply_count", property="replyCount", jdbcType=JdbcType.INTEGER),
         @Result(column="audit_status", property="auditStatus", jdbcType=JdbcType.TINYINT),
         @Result(column="create_time", property="createTime", jdbcType=JdbcType.TIMESTAMP),
@@ -92,6 +93,7 @@ public interface BookCommentMapper {
             c.map(id).toProperty("id")
             .map(bookId).toProperty("bookId")
             .map(commentContent).toProperty("commentContent")
+            .map(location).toProperty("location")
             .map(replyCount).toProperty("replyCount")
             .map(auditStatus).toProperty("auditStatus")
             .map(createTime).toProperty("createTime")
@@ -105,6 +107,7 @@ public interface BookCommentMapper {
             c.map(id).toProperty("id")
             .map(bookId).toProperty("bookId")
             .map(commentContent).toProperty("commentContent")
+            .map(location).toProperty("location")
             .map(replyCount).toProperty("replyCount")
             .map(auditStatus).toProperty("auditStatus")
             .map(createTime).toProperty("createTime")
@@ -118,6 +121,7 @@ public interface BookCommentMapper {
             c.map(id).toPropertyWhenPresent("id", record::getId)
             .map(bookId).toPropertyWhenPresent("bookId", record::getBookId)
             .map(commentContent).toPropertyWhenPresent("commentContent", record::getCommentContent)
+            .map(location).toPropertyWhenPresent("location", record::getLocation)
             .map(replyCount).toPropertyWhenPresent("replyCount", record::getReplyCount)
             .map(auditStatus).toPropertyWhenPresent("auditStatus", record::getAuditStatus)
             .map(createTime).toPropertyWhenPresent("createTime", record::getCreateTime)
@@ -157,6 +161,7 @@ public interface BookCommentMapper {
         return dsl.set(id).equalTo(record::getId)
                 .set(bookId).equalTo(record::getBookId)
                 .set(commentContent).equalTo(record::getCommentContent)
+                .set(location).equalTo(record::getLocation)
                 .set(replyCount).equalTo(record::getReplyCount)
                 .set(auditStatus).equalTo(record::getAuditStatus)
                 .set(createTime).equalTo(record::getCreateTime)
@@ -168,6 +173,7 @@ public interface BookCommentMapper {
         return dsl.set(id).equalToWhenPresent(record::getId)
                 .set(bookId).equalToWhenPresent(record::getBookId)
                 .set(commentContent).equalToWhenPresent(record::getCommentContent)
+                .set(location).equalToWhenPresent(record::getLocation)
                 .set(replyCount).equalToWhenPresent(record::getReplyCount)
                 .set(auditStatus).equalToWhenPresent(record::getAuditStatus)
                 .set(createTime).equalToWhenPresent(record::getCreateTime)
@@ -179,6 +185,7 @@ public interface BookCommentMapper {
         return update(c ->
             c.set(bookId).equalTo(record::getBookId)
             .set(commentContent).equalTo(record::getCommentContent)
+            .set(location).equalTo(record::getLocation)
             .set(replyCount).equalTo(record::getReplyCount)
             .set(auditStatus).equalTo(record::getAuditStatus)
             .set(createTime).equalTo(record::getCreateTime)
@@ -192,6 +199,7 @@ public interface BookCommentMapper {
         return update(c ->
             c.set(bookId).equalToWhenPresent(record::getBookId)
             .set(commentContent).equalToWhenPresent(record::getCommentContent)
+            .set(location).equalToWhenPresent(record::getLocation)
             .set(replyCount).equalToWhenPresent(record::getReplyCount)
             .set(auditStatus).equalToWhenPresent(record::getAuditStatus)
             .set(createTime).equalToWhenPresent(record::getCreateTime)

novel-common/src/main/java/com/java2nb/novel/mapper/BookCommentReplyDynamicSqlSupport.java

@@ -0,0 +1,54 @@
+package com.java2nb.novel.mapper;
+
+import java.sql.JDBCType;
+import java.util.Date;
+import javax.annotation.Generated;
+import org.mybatis.dynamic.sql.SqlColumn;
+import org.mybatis.dynamic.sql.SqlTable;
+
+public final class BookCommentReplyDynamicSqlSupport {
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final BookCommentReply bookCommentReply = new BookCommentReply();
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Long> id = bookCommentReply.id;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Long> commentId = bookCommentReply.commentId;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<String> replyContent = bookCommentReply.replyContent;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<String> location = bookCommentReply.location;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Byte> auditStatus = bookCommentReply.auditStatus;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Date> createTime = bookCommentReply.createTime;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Long> createUserId = bookCommentReply.createUserId;
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final class BookCommentReply extends SqlTable {
+        public final SqlColumn<Long> id = column("id", JDBCType.BIGINT);
+
+        public final SqlColumn<Long> commentId = column("comment_id", JDBCType.BIGINT);
+
+        public final SqlColumn<String> replyContent = column("reply_content", JDBCType.VARCHAR);
+
+        public final SqlColumn<String> location = column("location", JDBCType.VARCHAR);
+
+        public final SqlColumn<Byte> auditStatus = column("audit_status", JDBCType.TINYINT);
+
+        public final SqlColumn<Date> createTime = column("create_time", JDBCType.TIMESTAMP);
+
+        public final SqlColumn<Long> createUserId = column("create_user_id", JDBCType.BIGINT);
+
+        public BookCommentReply() {
+            super("book_comment_reply");
+        }
+    }
+}

novel-common/src/main/java/com/java2nb/novel/mapper/BookCommentReplyMapper.java

@@ -0,0 +1,208 @@
+package com.java2nb.novel.mapper;
+
+import static com.java2nb.novel.mapper.BookCommentReplyDynamicSqlSupport.*;
+import static org.mybatis.dynamic.sql.SqlBuilder.*;
+
+import com.java2nb.novel.entity.BookCommentReply;
+import java.util.Collection;
+import java.util.List;
+import java.util.Optional;
+import javax.annotation.Generated;
+import org.apache.ibatis.annotations.DeleteProvider;
+import org.apache.ibatis.annotations.InsertProvider;
+import org.apache.ibatis.annotations.Mapper;
+import org.apache.ibatis.annotations.Result;
+import org.apache.ibatis.annotations.ResultMap;
+import org.apache.ibatis.annotations.Results;
+import org.apache.ibatis.annotations.SelectProvider;
+import org.apache.ibatis.annotations.UpdateProvider;
+import org.apache.ibatis.type.JdbcType;
+import org.mybatis.dynamic.sql.BasicColumn;
+import org.mybatis.dynamic.sql.delete.DeleteDSLCompleter;
+import org.mybatis.dynamic.sql.delete.render.DeleteStatementProvider;
+import org.mybatis.dynamic.sql.insert.render.InsertStatementProvider;
+import org.mybatis.dynamic.sql.insert.render.MultiRowInsertStatementProvider;
+import org.mybatis.dynamic.sql.select.CountDSLCompleter;
+import org.mybatis.dynamic.sql.select.SelectDSLCompleter;
+import org.mybatis.dynamic.sql.select.render.SelectStatementProvider;
+import org.mybatis.dynamic.sql.update.UpdateDSL;
+import org.mybatis.dynamic.sql.update.UpdateDSLCompleter;
+import org.mybatis.dynamic.sql.update.UpdateModel;
+import org.mybatis.dynamic.sql.update.render.UpdateStatementProvider;
+import org.mybatis.dynamic.sql.util.SqlProviderAdapter;
+import org.mybatis.dynamic.sql.util.mybatis3.MyBatis3Utils;
+
+@Mapper
+public interface BookCommentReplyMapper {
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    BasicColumn[] selectList = BasicColumn.columnList(id, commentId, replyContent, location, auditStatus, createTime, createUserId);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @SelectProvider(type=SqlProviderAdapter.class, method="select")
+    long count(SelectStatementProvider selectStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @DeleteProvider(type=SqlProviderAdapter.class, method="delete")
+    int delete(DeleteStatementProvider deleteStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @InsertProvider(type=SqlProviderAdapter.class, method="insert")
+    int insert(InsertStatementProvider<BookCommentReply> insertStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @InsertProvider(type=SqlProviderAdapter.class, method="insertMultiple")
+    int insertMultiple(MultiRowInsertStatementProvider<BookCommentReply> multipleInsertStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @SelectProvider(type=SqlProviderAdapter.class, method="select")
+    @ResultMap("BookCommentReplyResult")
+    Optional<BookCommentReply> selectOne(SelectStatementProvider selectStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @SelectProvider(type=SqlProviderAdapter.class, method="select")
+    @Results(id="BookCommentReplyResult", value = {
+        @Result(column="id", property="id", jdbcType=JdbcType.BIGINT, id=true),
+        @Result(column="comment_id", property="commentId", jdbcType=JdbcType.BIGINT),
+        @Result(column="reply_content", property="replyContent", jdbcType=JdbcType.VARCHAR),
+        @Result(column="location", property="location", jdbcType=JdbcType.VARCHAR),
+        @Result(column="audit_status", property="auditStatus", jdbcType=JdbcType.TINYINT),
+        @Result(column="create_time", property="createTime", jdbcType=JdbcType.TIMESTAMP),
+        @Result(column="create_user_id", property="createUserId", jdbcType=JdbcType.BIGINT)
+    })
+    List<BookCommentReply> selectMany(SelectStatementProvider selectStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    @UpdateProvider(type=SqlProviderAdapter.class, method="update")
+    int update(UpdateStatementProvider updateStatement);
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default long count(CountDSLCompleter completer) {
+        return MyBatis3Utils.countFrom(this::count, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int delete(DeleteDSLCompleter completer) {
+        return MyBatis3Utils.deleteFrom(this::delete, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int deleteByPrimaryKey(Long id_) {
+        return delete(c -> 
+            c.where(id, isEqualTo(id_))
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int insert(BookCommentReply record) {
+        return MyBatis3Utils.insert(this::insert, record, bookCommentReply, c ->
+            c.map(id).toProperty("id")
+            .map(commentId).toProperty("commentId")
+            .map(replyContent).toProperty("replyContent")
+            .map(location).toProperty("location")
+            .map(auditStatus).toProperty("auditStatus")
+            .map(createTime).toProperty("createTime")
+            .map(createUserId).toProperty("createUserId")
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int insertMultiple(Collection<BookCommentReply> records) {
+        return MyBatis3Utils.insertMultiple(this::insertMultiple, records, bookCommentReply, c ->
+            c.map(id).toProperty("id")
+            .map(commentId).toProperty("commentId")
+            .map(replyContent).toProperty("replyContent")
+            .map(location).toProperty("location")
+            .map(auditStatus).toProperty("auditStatus")
+            .map(createTime).toProperty("createTime")
+            .map(createUserId).toProperty("createUserId")
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int insertSelective(BookCommentReply record) {
+        return MyBatis3Utils.insert(this::insert, record, bookCommentReply, c ->
+            c.map(id).toPropertyWhenPresent("id", record::getId)
+            .map(commentId).toPropertyWhenPresent("commentId", record::getCommentId)
+            .map(replyContent).toPropertyWhenPresent("replyContent", record::getReplyContent)
+            .map(location).toPropertyWhenPresent("location", record::getLocation)
+            .map(auditStatus).toPropertyWhenPresent("auditStatus", record::getAuditStatus)
+            .map(createTime).toPropertyWhenPresent("createTime", record::getCreateTime)
+            .map(createUserId).toPropertyWhenPresent("createUserId", record::getCreateUserId)
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default Optional<BookCommentReply> selectOne(SelectDSLCompleter completer) {
+        return MyBatis3Utils.selectOne(this::selectOne, selectList, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default List<BookCommentReply> select(SelectDSLCompleter completer) {
+        return MyBatis3Utils.selectList(this::selectMany, selectList, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default List<BookCommentReply> selectDistinct(SelectDSLCompleter completer) {
+        return MyBatis3Utils.selectDistinct(this::selectMany, selectList, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default Optional<BookCommentReply> selectByPrimaryKey(Long id_) {
+        return selectOne(c ->
+            c.where(id, isEqualTo(id_))
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int update(UpdateDSLCompleter completer) {
+        return MyBatis3Utils.update(this::update, bookCommentReply, completer);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    static UpdateDSL<UpdateModel> updateAllColumns(BookCommentReply record, UpdateDSL<UpdateModel> dsl) {
+        return dsl.set(id).equalTo(record::getId)
+                .set(commentId).equalTo(record::getCommentId)
+                .set(replyContent).equalTo(record::getReplyContent)
+                .set(location).equalTo(record::getLocation)
+                .set(auditStatus).equalTo(record::getAuditStatus)
+                .set(createTime).equalTo(record::getCreateTime)
+                .set(createUserId).equalTo(record::getCreateUserId);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    static UpdateDSL<UpdateModel> updateSelectiveColumns(BookCommentReply record, UpdateDSL<UpdateModel> dsl) {
+        return dsl.set(id).equalToWhenPresent(record::getId)
+                .set(commentId).equalToWhenPresent(record::getCommentId)
+                .set(replyContent).equalToWhenPresent(record::getReplyContent)
+                .set(location).equalToWhenPresent(record::getLocation)
+                .set(auditStatus).equalToWhenPresent(record::getAuditStatus)
+                .set(createTime).equalToWhenPresent(record::getCreateTime)
+                .set(createUserId).equalToWhenPresent(record::getCreateUserId);
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int updateByPrimaryKey(BookCommentReply record) {
+        return update(c ->
+            c.set(commentId).equalTo(record::getCommentId)
+            .set(replyContent).equalTo(record::getReplyContent)
+            .set(location).equalTo(record::getLocation)
+            .set(auditStatus).equalTo(record::getAuditStatus)
+            .set(createTime).equalTo(record::getCreateTime)
+            .set(createUserId).equalTo(record::getCreateUserId)
+            .where(id, isEqualTo(record::getId))
+        );
+    }
+
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    default int updateByPrimaryKeySelective(BookCommentReply record) {
+        return update(c ->
+            c.set(commentId).equalToWhenPresent(record::getCommentId)
+            .set(replyContent).equalToWhenPresent(record::getReplyContent)
+            .set(location).equalToWhenPresent(record::getLocation)
+            .set(auditStatus).equalToWhenPresent(record::getAuditStatus)
+            .set(createTime).equalToWhenPresent(record::getCreateTime)
+            .set(createUserId).equalToWhenPresent(record::getCreateUserId)
+            .where(id, isEqualTo(record::getId))
+        );
+    }
+}

novel-common/src/main/java/com/java2nb/novel/mapper/CrawlSingleTaskDynamicSqlSupport.java

@@ -37,6 +37,9 @@ public final class CrawlSingleTaskDynamicSqlSupport {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public static final SqlColumn<Byte> excCount = crawlSingleTask.excCount;
 
+    @Generated("org.mybatis.generator.api.MyBatisGenerator")
+    public static final SqlColumn<Integer> crawlChapters = crawlSingleTask.crawlChapters;
+
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     public static final SqlColumn<Date> createTime = crawlSingleTask.createTime;
 
@@ -60,6 +63,8 @@ public final class CrawlSingleTaskDynamicSqlSupport {
 
         public final SqlColumn<Byte> excCount = column("exc_count", JDBCType.TINYINT);
 
+        public final SqlColumn<Integer> crawlChapters = column("crawl_chapters", JDBCType.INTEGER);
+
         public final SqlColumn<Date> createTime = column("create_time", JDBCType.TIMESTAMP);
 
         public CrawlSingleTask() {

novel-common/src/main/java/com/java2nb/novel/mapper/CrawlSingleTaskMapper.java

@@ -35,7 +35,7 @@ import org.mybatis.dynamic.sql.util.mybatis3.MyBatis3Utils;
 @Mapper
 public interface CrawlSingleTaskMapper {
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
-    BasicColumn[] selectList = BasicColumn.columnList(id, sourceId, sourceName, sourceBookId, catId, bookName, authorName, taskStatus, excCount, createTime);
+    BasicColumn[] selectList = BasicColumn.columnList(id, sourceId, sourceName, sourceBookId, catId, bookName, authorName, taskStatus, excCount, crawlChapters, createTime);
 
     @Generated("org.mybatis.generator.api.MyBatisGenerator")
     @SelectProvider(type=SqlProviderAdapter.class, method="select")
@@ -70,6 +70,7 @@ public interface CrawlSingleTaskMapper {
         @Result(column="author_name", property="authorName", jdbcType=JdbcType.VARCHAR),
         @Result(column="task_status", property="taskStatus", jdbcType=JdbcType.TINYINT),
         @Result(column="exc_count", property="excCount", jdbcType=JdbcType.TINYINT),
+        @Result(column="crawl_chapters", property="crawlChapters", jdbcType=JdbcType.INTEGER),
         @Result(column="create_time", property="createTime", jdbcType=JdbcType.TIMESTAMP)
     })
     List<CrawlSingleTask> selectMany(SelectStatementProvider selectStatement);
@@ -107,6 +108,7 @@ public interface CrawlSingleTaskMapper {
                 .map(authorName).toProperty("authorName")
                 .map(taskStatus).toProperty("taskStatus")
                 .map(excCount).toProperty("excCount")
+                .map(crawlChapters).toProperty("crawlChapters")
                 .map(createTime).toProperty("createTime")
         );
     }
@@ -123,6 +125,7 @@ public interface CrawlSingleTaskMapper {
                 .map(authorName).toProperty("authorName")
                 .map(taskStatus).toProperty("taskStatus")
                 .map(excCount).toProperty("excCount")
+                .map(crawlChapters).toProperty("crawlChapters")
                 .map(createTime).toProperty("createTime")
         );
     }
@@ -139,6 +142,7 @@ public interface CrawlSingleTaskMapper {
                 .map(authorName).toPropertyWhenPresent("authorName", record::getAuthorName)
                 .map(taskStatus).toPropertyWhenPresent("taskStatus", record::getTaskStatus)
                 .map(excCount).toPropertyWhenPresent("excCount", record::getExcCount)
+                .map(crawlChapters).toPropertyWhenPresent("crawlChapters", record::getCrawlChapters)
                 .map(createTime).toPropertyWhenPresent("createTime", record::getCreateTime)
         );
     }
@@ -181,6 +185,7 @@ public interface CrawlSingleTaskMapper {
             .set(authorName).equalTo(record::getAuthorName)
             .set(taskStatus).equalTo(record::getTaskStatus)
             .set(excCount).equalTo(record::getExcCount)
+            .set(crawlChapters).equalTo(record::getCrawlChapters)
             .set(createTime).equalTo(record::getCreateTime);
     }
 
@@ -195,6 +200,7 @@ public interface CrawlSingleTaskMapper {
             .set(authorName).equalToWhenPresent(record::getAuthorName)
             .set(taskStatus).equalToWhenPresent(record::getTaskStatus)
             .set(excCount).equalToWhenPresent(record::getExcCount)
+            .set(crawlChapters).equalToWhenPresent(record::getCrawlChapters)
             .set(createTime).equalToWhenPresent(record::getCreateTime);
     }
 
@@ -209,6 +215,7 @@ public interface CrawlSingleTaskMapper {
                 .set(authorName).equalTo(record::getAuthorName)
                 .set(taskStatus).equalTo(record::getTaskStatus)
                 .set(excCount).equalTo(record::getExcCount)
+                .set(crawlChapters).equalTo(record::getCrawlChapters)
                 .set(createTime).equalTo(record::getCreateTime)
                 .where(id, isEqualTo(record::getId))
         );
@@ -225,6 +232,7 @@ public interface CrawlSingleTaskMapper {
                 .set(authorName).equalToWhenPresent(record::getAuthorName)
                 .set(taskStatus).equalToWhenPresent(record::getTaskStatus)
                 .set(excCount).equalToWhenPresent(record::getExcCount)
+                .set(crawlChapters).equalToWhenPresent(record::getCrawlChapters)
                 .set(createTime).equalToWhenPresent(record::getCreateTime)
                 .where(id, isEqualTo(record::getId))
         );

novel-common/src/main/resources/application-common-dev.yml

@@ -1,71 +1,18 @@
 spring:
-  profiles:
-    include: [ common ]
+  config:
+    import: classpath:application-common.yml
   main:
     allow-bean-definition-overriding: true
   #Redis服务器IP
+  data:
     redis:
       host: 127.0.0.1
       #Redis服务器连接端口
       port: 6379
       #Redis服务器连接密码
       password: test123456
-    jedis:
-      pool:
-        #连接池最大连接数（使用负值表示没有限制）
-        max-active: 8
-        #连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-wait: 1
-        #连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-idle: 8
-        #连接池中的最小空闲连接
-        min-idle: 0
       #连接超时时间（毫秒）
-    timeout: 30000
-  datasource:
-    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-    username: root
-    password: test123456
-    driver-class-name: com.mysql.cj.jdbc.Driver
-
-
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-
-sharding:
-  jdbc:
-    datasource:
-      names: ds0 #,ds1
-      ds0:
-        type: com.zaxxer.hikari.HikariDataSource
-        driver-class-name: com.mysql.cj.jdbc.Driver
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-    #      ds1:
-    #        type: com.alibaba.druid.pool.DruidDataSource
-    #        driver-class-name: com.mysql.jdbc.Driver
-    #        url: jdbc:mysql://localhost:3306/novel_plus2
-    #        username: root
-    #        password: test123456
-    config:
-      sharding:
-        props:
-          sql.show: true
-        tables:
-          book_content: #book_content表
-            key-generator-column-name: id  #主键
-            actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
-            #            database-strategy:  #分库策略
-            #              inline:
-            #                sharding-column: book_id
-            #                algorithm-expression: ds${book_id % 10}
-            table-strategy: #分表策略
-              inline:
-                shardingColumn: index_id
-                algorithm-expression: book_content${index_id % 10}
-
+      timeout: 10000
 
 content:
   save:
@@ -78,6 +25,10 @@ http:
     # 是否开启 HTTP 代理，true-开启，false-不开启
     enabled: false
     # 代理 IP
-    ip: u493.kdltps.com
+    ip: us.swiftproxy.net
     # 代理端口号
-    port: 15818
+    port: 7878
+    # 代理用户名
+    username: swiftproxy_u
+    # 代理密码
+    password: swiftproxy_p

novel-common/src/main/resources/application-common-prod.yml

@@ -1,71 +1,18 @@
 spring:
-  profiles:
-    include: [ common ]
+  config:
+    import: classpath:application-common.yml
   main:
     allow-bean-definition-overriding: true
-  #Redis服务器IP
+  data:
     redis:
+      #Redis服务器IP
       host: 127.0.0.1
       #Redis服务器连接端口
       port: 6379
       #Redis服务器连接密码
-    password: test
-    jedis:
-      pool:
-        #连接池最大连接数（使用负值表示没有限制）
-        max-active: 8
-        #连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-wait: 1
-        #连接池最大阻塞等待时间（使用负值表示没有限制）
-        max-idle: 8
-        #连接池中的最小空闲连接
-        min-idle: 0
+      password: test123456
       #连接超时时间（毫秒）
-    timeout: 30000
-  datasource:
-    url: jdbc:mysql://127.0.0.1:3306/novel_biz?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-    username: root
-    password: test123456
-    driver-class-name: com.mysql.cj.jdbc.Driver
-
-
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-
-sharding:
-  jdbc:
-    datasource:
-      names: ds0 #,ds1
-      ds0:
-        type: com.zaxxer.hikari.HikariDataSource
-        driver-class-name: com.mysql.cj.jdbc.Driver
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-    #      ds1:
-    #        type: com.alibaba.druid.pool.DruidDataSource
-    #        driver-class-name: com.mysql.jdbc.Driver
-    #        url: jdbc:mysql://localhost:3306/novel_plus2
-    #        username: root
-    #        password: test123456
-    config:
-      sharding:
-        props:
-          sql.show: true
-        tables:
-          book_content: #book_content表
-            key-generator-column-name: id  #主键
-            actual-data-nodes: ds${0}.book_content${0..9}  #数据节点
-            #            database-strategy:  #分库策略
-            #              inline:
-            #                sharding-column: book_id
-            #                algorithm-expression: ds${book_id % 10}
-            table-strategy: #分表策略
-              inline:
-                shardingColumn: index_id
-                algorithm-expression: book_content${index_id % 10}
-
+      timeout: 10000
 
 logging:
   level:

novel-common/src/main/resources/application-common.yml

@@ -1,20 +1,16 @@
 spring:
-  cache:
-    ehcache:
-      config: classpath:ehcache.xml
+  datasource:
+    url: jdbc:shardingsphere:absolutepath:${user.dir}/config/shardingsphere-jdbc.yml
+    driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver
   thymeleaf:
     mode: LEGACYHTML5 #去除thymeleaf的html严格校验thymeleaf.mode=LEGACYHTML5
     cache: false # 是否开启模板缓存，默认true,建议在开发时关闭缓存,不然没法看到实时
 
-  # 将所有数字转为 String 类型返回，避免前端数据精度丢失的问题
-  jackson:
-    generator:
-      write-numbers-as-strings: true
-
-  #上传文件的最大值（1M）
+  #上传文件的最大值（100M）
   servlet:
     multipart:
-      max-file-size: 1048576
+      max-file-size: 100MB
+      max-request-size: 100MB
 
 mybatis:
   configuration:
@@ -26,6 +22,8 @@ mybatis:
 logging:
   config: classpath:logback-boot.xml
 
+pagehelper:
+  helper-dialect: mysql
 
 
 

novel-crawl/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>4.2.0</version>
+        <version>5.2.1</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -21,6 +21,12 @@
             <artifactId>novel-common</artifactId>
         </dependency>
 
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>2.2</version>
+        </dependency>
+
         <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-security</artifactId>
@@ -29,7 +35,6 @@
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
-            <version>${jackson.version}</version>
         </dependency>
 
 
@@ -72,6 +77,9 @@
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
                                     <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
+
+                                <copy file="${basedir}/src/main/build/docker/Dockerfile"
+                                      tofile="${project.build.directory}/build/Dockerfile"/>
                             </tasks>
                         </configuration>
                     </execution>

novel-crawl/src/main/build/config/application-common-prod.yml

@@ -1,53 +0,0 @@
-#端口号
-server:
-  port: 8083
-
-#不分表的数据库配置
-spring:
-  datasource:
-    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-    username: root
-    password: root
-  redis:
-    host: 127.0.0.1
-    port: 6379
-    password: test123456
-
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-sharding:
-  jdbc:
-    datasource:
-      ds0:
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-
-#登录用户名密码
-admin:
-  username: admin
-  password: admin
-#
-##爬虫自动更新的线程数
-##建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
-##随着小说数量的增多可以逐渐增加，但建议不要超出CPU的线程数
-crawl:
-  update:
-    thread: 1
-
-#小说内容保存配置
-content:
-  save:
-    storage: db # 小说内容存储方式：db-数据库，txt-TXT文本
-    path: /Users/xiongxiaoyang/books  # 小说TXT文本保存路径
-
-# HTTP 代理配置
-http:
-  proxy:
-    # 是否开启 HTTP 代理，true-开启，false-不开启
-    enabled: false
-    # 代理 IP
-    ip: u493.kdltps.com
-    # 代理端口号
-    port: 15818

novel-crawl/src/main/build/config/application.yml

@@ -0,0 +1,47 @@
+#端口号
+server:
+  port: 8083
+
+spring:
+  data:
+    redis:
+      #Redis服务器IP
+      host: 127.0.0.1
+      #Redis服务器连接端口
+      port: 6379
+      #Redis服务器连接密码
+      password: test123456
+      #连接超时时间（毫秒）
+      timeout: 10000
+
+#登录用户名密码
+admin:
+  username: admin
+  password: admin
+#
+##爬虫自动更新的线程数
+##建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
+##随着小说数量的增多可以逐渐增加，但建议不要超出CPU的线程数
+crawl:
+  update:
+    thread: 1
+
+#小说内容保存配置
+content:
+  save:
+    storage: db # 小说内容存储方式：db-数据库，txt-TXT文本
+    path: /Users/xiongxiaoyang/books  # 小说TXT文本保存路径
+
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: us.swiftproxy.net
+    # 代理端口号
+    port: 7878
+    # 代理用户名
+    username: swiftproxy_u
+    # 代理密码
+    password: swiftproxy_p

novel-crawl/src/main/build/config/shardingsphere-jdbc.yml

@@ -0,0 +1,47 @@
+mode:
+  # 单机模式
+  type: Standalone
+  # 元数据持久化
+  repository:
+    # 数据库持久化
+    type: JDBC
+
+# 数据源配置
+dataSources:
+  ds_1:
+    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
+    driverClassName: com.mysql.cj.jdbc.Driver
+    jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+# 规则配置
+rules:
+  - !SINGLE
+    tables:
+      - "*.*"
+  - !SHARDING
+    tables: # 数据分片规则配置
+      book_content:
+        # 分库策略，缺省表示使用默认分库策略
+        actualDataNodes: ds_${1}.book_content${0..9}
+        # 分表策略
+        tableStrategy:
+          standard:
+            # 分片列名称
+            shardingColumn: index_id
+            # 分片算法名称
+            shardingAlgorithmName: bookContentSharding
+
+    shardingAlgorithms:
+      bookContentSharding:
+        # 行表达式分片算法，使用 Groovy 的表达式，提供对 SQL 语句中的 = 和 IN 的分片操作支持
+        type: INLINE
+        props:
+          # 分片算法的行表达式
+          algorithm-expression: book_content${index_id % 10}
+
+
+
+props:
+  # 是否在日志中打印 SQL
+  sql-show: true

novel-crawl/src/main/build/docker/Dockerfile

@@ -1,9 +1,9 @@
 FROM openjdk:8
-ADD novel-crawl-4.2.0.jar /root
+ADD novel-crawl.jar /root
 ENV dburl=""
 ENV username=""
 ENV	password=""
 ENV redishost = ""
 ENV redisport = ""
 ENV redispwd = ""
-ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-crawl-4.2.0.jar"]
+ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-crawl.jar"]

novel-crawl/src/main/java/com/java2nb/novel/controller/CrawlController.java

@@ -153,6 +153,14 @@ public class CrawlController {
         return RestResult.ok();
     }
 
+    /**
+     * 采集任务进度查询
+     * */
+    @GetMapping("getTaskProgress/{id}")
+    public RestResult<Integer> getTaskProgress(@PathVariable("id") Long id){
+        return RestResult.ok(crawlService.getTaskProgress(id));
+    }
+
 
 
 

novel-crawl/src/main/java/com/java2nb/novel/core/config/SecurityConfiguration.java

@@ -4,14 +4,15 @@ import lombok.RequiredArgsConstructor;
 import org.springframework.beans.factory.annotation.Value;
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Configuration;
-import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
+import org.springframework.security.config.Customizer;
 import org.springframework.security.config.annotation.web.builders.HttpSecurity;
-import org.springframework.security.config.annotation.web.builders.WebSecurity;
 import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
-import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
 import org.springframework.security.core.userdetails.User;
+import org.springframework.security.core.userdetails.UserDetails;
 import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
 import org.springframework.security.crypto.password.PasswordEncoder;
+import org.springframework.security.provisioning.InMemoryUserDetailsManager;
+import org.springframework.security.web.SecurityFilterChain;
 
 /**
  * SpringSecurity配置
@@ -21,7 +22,7 @@ import org.springframework.security.crypto.password.PasswordEncoder;
 @Configuration
 @EnableWebSecurity
 @RequiredArgsConstructor
-public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
+public class SecurityConfiguration {
 
     @Value("${admin.username}")
     private String username;
@@ -29,39 +30,40 @@ public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
     @Value("${admin.password}")
     private String password;
 
-
     @Bean
     public PasswordEncoder passwordEncoder() {
         return new BCryptPasswordEncoder();
     }
 
-
-    @Override
-    public void configure(WebSecurity web) throws Exception {
-        super.configure(web);
+    @Bean
+    public InMemoryUserDetailsManager userDetailsService() {
+        UserDetails admin = User.builder()
+                .username(username)
+                .password(passwordEncoder().encode(password))
+                .roles("ADMIN")
+                .build();
+        return new InMemoryUserDetailsManager(admin);
     }
 
-    @Override
-    public void configure(AuthenticationManagerBuilder auth) throws Exception {
+    @Bean
+    public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
+        http
+                .csrf(csrf -> csrf.disable()) // 禁用 CSRF
+                .authorizeHttpRequests(auth -> auth
+                        .requestMatchers("/css/**", "/favicon.ico").permitAll() // 允许访问静态资源
+                        .anyRequest().hasRole("ADMIN") // 其他请求需要 ADMIN 角色
+                )
+                .formLogin(form -> form
+                        .loginPage("/login.html") // 自定义登录页面
+                        .loginProcessingUrl("/login") // 登录处理 URL
+                        .permitAll()
+                )
+                .logout(logout -> logout
+                        .logoutUrl("/logout") // 登出 URL
+                        .logoutSuccessUrl("/") // 登出成功后跳转的页面
+                )
+                .httpBasic(Customizer.withDefaults()); // 启用 HTTP Basic 认证
 
-        User.UserBuilder builder = User.builder().passwordEncoder(passwordEncoder()::encode);
-        auth.inMemoryAuthentication().withUser(builder.username(username).password(password).roles("ADMIN").build());
+        return http.build();
     }
-
-    @Override
-    protected void configure(HttpSecurity http) throws Exception {
-        http.csrf().disable()
-            .authorizeRequests()
-            .antMatchers("/css/**").permitAll()
-            .antMatchers("/favicon.ico").permitAll()
-            .antMatchers("/**").hasRole("ADMIN")
-            .and().formLogin().loginPage("/login.html").loginProcessingUrl("/login").permitAll()
-            .and().logout()
-            .logoutUrl("/logout")
-            .logoutSuccessUrl("/")
-            .and().httpBasic();
-
-    }
-
-
 }

novel-crawl/src/main/java/com/java2nb/novel/core/crawl/CrawlBookHandler.java

@@ -7,6 +7,6 @@ import com.java2nb.novel.entity.Book;
  * */
 public interface CrawlBookHandler {
 
-    void handle(Book book);
+    void handle(Book book) throws InterruptedException;
 
 }

novel-crawl/src/main/java/com/java2nb/novel/core/crawl/CrawlParser.java

@@ -1,21 +1,20 @@
 package com.java2nb.novel.core.crawl;
 
-import com.java2nb.novel.core.utils.HttpUtil;
 import com.java2nb.novel.core.utils.RandomBookInfoUtil;
-import com.java2nb.novel.core.utils.RestTemplateUtil;
 import com.java2nb.novel.core.utils.StringUtil;
 import com.java2nb.novel.entity.Book;
 import com.java2nb.novel.entity.BookContent;
 import com.java2nb.novel.entity.BookIndex;
+import com.java2nb.novel.entity.CrawlSingleTask;
 import com.java2nb.novel.utils.Constants;
+import com.java2nb.novel.utils.CrawlHttpClient;
 import io.github.xxyopen.util.IdWorker;
-import lombok.SneakyThrows;
+import lombok.RequiredArgsConstructor;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
-import org.springframework.http.HttpStatus;
-import org.springframework.http.ResponseEntity;
-import org.springframework.web.client.RestTemplate;
+import org.springframework.stereotype.Component;
 
+import java.text.ParseException;
 import java.text.SimpleDateFormat;
 import java.util.*;
 import java.util.regex.Matcher;
@@ -27,19 +26,38 @@ import java.util.regex.Pattern;
  * @author Administrator
  */
 @Slf4j
+@Component
+@RequiredArgsConstructor
 public class CrawlParser {
 
-    private static final IdWorker idWorker = IdWorker.INSTANCE;
+    private final IdWorker ID_WORKER = IdWorker.INSTANCE;
 
-    private static final RestTemplate restTemplate = RestTemplateUtil.getInstance("utf-8");
+    private final CrawlHttpClient crawlHttpClient;
 
-    private static final ThreadLocal<Integer> retryCount = new ThreadLocal<>();
+    /**
+     * 爬虫任务进度
+     */
+    private final Map<Long, Integer> crawlTaskProgress = new HashMap<>();
 
-    @SneakyThrows
-    public static void parseBook(RuleBean ruleBean, String bookId, CrawlBookHandler handler) {
+    /**
+     * 获取爬虫任务进度
+     */
+    public Integer getCrawlTaskProgress(Long taskId) {
+        return crawlTaskProgress.get(taskId);
+    }
+
+    /**
+     * 移除爬虫任务进度
+     */
+    public void removeCrawlTaskProgress(Long taskId) {
+        crawlTaskProgress.remove(taskId);
+    }
+
+    public void parseBook(RuleBean ruleBean, String bookId, CrawlBookHandler handler)
+        throws InterruptedException {
         Book book = new Book();
         String bookDetailUrl = ruleBean.getBookDetailUrl().replace("{bookId}", bookId);
-        String bookDetailHtml = getByHttpClientWithChrome(bookDetailUrl);
+        String bookDetailHtml = crawlHttpClient.get(bookDetailUrl, ruleBean.getCharset());
         if (bookDetailHtml != null) {
             Pattern bookNamePatten = PatternFactory.getPattern(ruleBean.getBookNamePatten());
             Matcher bookNameMatch = bookNamePatten.matcher(bookDetailHtml);
@@ -89,7 +107,8 @@ public class CrawlParser {
                         }
                     }
 
-                    String desc = bookDetailHtml.substring(bookDetailHtml.indexOf(ruleBean.getDescStart()) + ruleBean.getDescStart().length());
+                    String desc = bookDetailHtml.substring(
+                        bookDetailHtml.indexOf(ruleBean.getDescStart()) + ruleBean.getDescStart().length());
                     desc = desc.substring(0, desc.indexOf(ruleBean.getDescEnd()));
                     //过滤掉简介中的特殊标签
                     desc = desc.replaceAll("<a[^<]+</a>", "")
@@ -97,6 +116,22 @@ public class CrawlParser {
                         .replaceAll("<p>\\s*</p>", "")
                         .replaceAll("<p>", "")
                         .replaceAll("</p>", "<br/>");
+                    // 小说简介过滤
+                    String filterDesc = ruleBean.getFilterDesc();
+                    if (StringUtils.isNotBlank(filterDesc)) {
+                        String[] filterRules = filterDesc.replace("\r\n", "\n").split("\n");
+                        for (String filterRule : filterRules) {
+                            if (StringUtils.isNotBlank(filterRule)) {
+                                desc = desc.replaceAll(filterRule, "");
+                            }
+                        }
+                    }
+                    // 去除小说简介前后空格
+                    desc = desc.trim();
+                    // 去除小说简介末尾冗余的小说名
+                    if (desc.endsWith(bookName)) {
+                        desc = desc.substring(0, desc.length() - bookName.length());
+                    }
                     //设置书籍简介
                     book.setBookDesc(desc);
                     if (StringUtils.isNotBlank(ruleBean.getStatusPatten())) {
@@ -112,14 +147,20 @@ public class CrawlParser {
                         }
                     }
 
-                    if (StringUtils.isNotBlank(ruleBean.getUpadateTimePatten()) && StringUtils.isNotBlank(ruleBean.getUpadateTimeFormatPatten())) {
+                    if (StringUtils.isNotBlank(ruleBean.getUpadateTimePatten()) && StringUtils.isNotBlank(
+                        ruleBean.getUpadateTimeFormatPatten())) {
                         Pattern updateTimePatten = PatternFactory.getPattern(ruleBean.getUpadateTimePatten());
                         Matcher updateTimeMatch = updateTimePatten.matcher(bookDetailHtml);
                         boolean isFindUpdateTime = updateTimeMatch.find();
                         if (isFindUpdateTime) {
                             String updateTime = updateTimeMatch.group(1);
                             //设置更新时间
-                            book.setLastIndexUpdateTime(new SimpleDateFormat(ruleBean.getUpadateTimeFormatPatten()).parse(updateTime));
+                            try {
+                                book.setLastIndexUpdateTime(
+                                    new SimpleDateFormat(ruleBean.getUpadateTimeFormatPatten()).parse(updateTime));
+                            } catch (ParseException e) {
+                                log.error("解析最新章节更新时间出错", e);
+                            }
 
                         }
                     }
@@ -131,7 +172,7 @@ public class CrawlParser {
                 } else if (book.getVisitCount() != null && book.getScore() == null) {
                     //随机根据访问次数生成评分
                     book.setScore(RandomBookInfoUtil.getScoreByVisitCount(book.getVisitCount()));
-                } else if (book.getVisitCount() == null && book.getScore() == null) {
+                } else if (book.getVisitCount() == null) {
                     //都没有，设置成固定值
                     book.setVisitCount(Constants.VISIT_COUNT_DEFAULT);
                     book.setScore(6.5f);
@@ -141,7 +182,14 @@ public class CrawlParser {
         handler.handle(book);
     }
 
-    public static boolean parseBookIndexAndContent(String sourceBookId, Book book, RuleBean ruleBean, Map<Integer, BookIndex> existBookIndexMap, CrawlBookChapterHandler handler) {
+    public boolean parseBookIndexAndContent(String sourceBookId, Book book, RuleBean ruleBean,
+        Map<Integer, BookIndex> existBookIndexMap, CrawlBookChapterHandler handler, CrawlSingleTask task)
+        throws InterruptedException {
+
+        if (task != null) {
+            // 开始采集
+            crawlTaskProgress.put(task.getId(), 0);
+        }
 
         Date currentDate = new Date();
 
@@ -149,11 +197,12 @@ public class CrawlParser {
         List<BookContent> contentList = new ArrayList<>();
         //读取目录
         String indexListUrl = ruleBean.getBookIndexUrl().replace("{bookId}", sourceBookId);
-        String indexListHtml = getByHttpClientWithChrome(indexListUrl);
+        String indexListHtml = crawlHttpClient.get(indexListUrl, ruleBean.getCharset());
 
         if (indexListHtml != null) {
             if (StringUtils.isNotBlank(ruleBean.getBookIndexStart())) {
-                indexListHtml = indexListHtml.substring(indexListHtml.indexOf(ruleBean.getBookIndexStart()) + ruleBean.getBookIndexStart().length());
+                indexListHtml = indexListHtml.substring(
+                    indexListHtml.indexOf(ruleBean.getBookIndexStart()) + ruleBean.getBookIndexStart().length());
             }
 
             Pattern indexIdPatten = PatternFactory.getPattern(ruleBean.getIndexIdPatten());
@@ -174,14 +223,16 @@ public class CrawlParser {
                 BookIndex hasIndex = existBookIndexMap.get(indexNum);
                 String indexName = indexNameMatch.group(1);
 
-                if (hasIndex == null || !StringUtils.deleteWhitespace(hasIndex.getIndexName()).equals(StringUtils.deleteWhitespace(indexName))) {
+                if (hasIndex == null || !StringUtils.deleteWhitespace(hasIndex.getIndexName())
+                    .equals(StringUtils.deleteWhitespace(indexName))) {
 
                     String sourceIndexId = indexIdMatch.group(1);
                     String bookContentUrl = ruleBean.getBookContentUrl();
                     int calStart = bookContentUrl.indexOf("{cal_");
                     if (calStart != -1) {
                         //内容页URL需要进行计算才能得到
-                        String calStr = bookContentUrl.substring(calStart, calStart + bookContentUrl.substring(calStart).indexOf("}"));
+                        String calStr = bookContentUrl.substring(calStart,
+                            calStart + bookContentUrl.substring(calStart).indexOf("}"));
                         String[] calArr = calStr.split("_");
                         int calType = Integer.parseInt(calArr[1]);
                         if (calType == 1) {
@@ -196,7 +247,7 @@ public class CrawlParser {
                                 calResult = sourceIndexId.substring(0, sourceBookId.length() - y);
                             }
 
-                            if (calResult.length() == 0) {
+                            if (calResult.isEmpty()) {
                                 calResult = "0";
 
                             }
@@ -206,13 +257,27 @@ public class CrawlParser {
 
                     }
 
-                    String contentUrl = bookContentUrl.replace("{bookId}", sourceBookId).replace("{indexId}", sourceIndexId);
+                    String contentUrl = bookContentUrl.replace("{bookId}", sourceBookId)
+                        .replace("{indexId}", sourceIndexId);
 
                     //查询章节内容
-                    String contentHtml = getByHttpClientWithChrome(contentUrl);
+                    String contentHtml = crawlHttpClient.get(contentUrl, ruleBean.getCharset());
                     if (contentHtml != null && !contentHtml.contains("正在手打中")) {
-                        String content = contentHtml.substring(contentHtml.indexOf(ruleBean.getContentStart()) + ruleBean.getContentStart().length());
+                        String content = contentHtml.substring(
+                            contentHtml.indexOf(ruleBean.getContentStart()) + ruleBean.getContentStart().length());
                         content = content.substring(0, content.indexOf(ruleBean.getContentEnd()));
+                        // 小说内容过滤
+                        String filterContent = ruleBean.getFilterContent();
+                        if (StringUtils.isNotBlank(filterContent)) {
+                            String[] filterRules = filterContent.replace("\r\n", "\n").split("\n");
+                            for (String filterRule : filterRules) {
+                                if (StringUtils.isNotBlank(filterRule)) {
+                                    content = content.replaceAll(filterRule, "");
+                                }
+                            }
+                        }
+                        // 去除小说内容末尾的所有换行
+                        content = removeTrailingBrTags(content);
                         //插入章节目录和章节内容
                         BookIndex bookIndex = new BookIndex();
                         bookIndex.setIndexName(indexName);
@@ -235,7 +300,7 @@ public class CrawlParser {
                         } else {
                             //章节插入
                             //设置目录和章节内容
-                            Long indexId = idWorker.nextId();
+                            Long indexId = ID_WORKER.nextId();
                             bookIndex.setId(indexId);
                             bookIndex.setBookId(book.getId());
 
@@ -248,6 +313,11 @@ public class CrawlParser {
                         }
                         bookIndex.setUpdateTime(currentDate);
 
+                        if (task != null) {
+                            // 更新采集进度
+                            crawlTaskProgress.put(task.getId(), indexList.size());
+                        }
+
 
                     }
 
@@ -257,11 +327,10 @@ public class CrawlParser {
                 isFindIndex = indexIdMatch.find() & indexNameMatch.find();
             }
 
-
-            if (indexList.size() > 0) {
+            if (!indexList.isEmpty()) {
                 //如果有爬到最新章节，则设置小说主表的最新章节信息
                 //获取爬取到的最新章节
-                BookIndex lastIndex = indexList.get(indexList.size() - 1);
+                BookIndex lastIndex = indexList.getLast();
                 book.setLastIndexId(lastIndex.getId());
                 book.setLastIndexName(lastIndex.getIndexName());
                 book.setLastIndexUpdateTime(currentDate);
@@ -270,7 +339,7 @@ public class CrawlParser {
             book.setWordCount(totalWordCount);
             book.setUpdateTime(currentDate);
 
-            if (indexList.size() == contentList.size() && indexList.size() > 0) {
+            if (indexList.size() == contentList.size() && !indexList.isEmpty()) {
 
                 handler.handle(new ChapterBean() {{
                     setBookIndexList(indexList);
@@ -291,55 +360,11 @@ public class CrawlParser {
 
     }
 
-
-    private static String getByHttpClient(String url) {
-        try {
-            ResponseEntity<String> forEntity = restTemplate.getForEntity(url, String.class);
-            if (forEntity.getStatusCode() == HttpStatus.OK) {
-                String body = forEntity.getBody();
-                assert body != null;
-                if (body.length() < Constants.INVALID_HTML_LENGTH) {
-                    return processErrorHttpResult(url);
-                }
-                //成功获得html内容
-                return body;
-            }
-        } catch (Exception e) {
-            e.printStackTrace();
-        }
-        return processErrorHttpResult(url);
-
+    /**
+     * 删除字符串末尾的所有 <br> 类似标签（允许各种空格）
+     */
+    public static String removeTrailingBrTags(String str) {
+        return str.replaceAll("(?i)(?:\\s*<\\s*br\\s*/?\\s*>)++(?:\\s|\\u3000)*$", "");
     }
 
-    private static String getByHttpClientWithChrome(String url) {
-        try {
-
-            String body = HttpUtil.getByHttpClientWithChrome(url);
-            if (body != null && body.length() < Constants.INVALID_HTML_LENGTH) {
-                return processErrorHttpResult(url);
-            }
-            //成功获得html内容
-            return body;
-        } catch (Exception e) {
-            e.printStackTrace();
-        }
-        return processErrorHttpResult(url);
-
-    }
-
-    @SneakyThrows
-    private static String processErrorHttpResult(String url) {
-        Integer count = retryCount.get();
-        if (count == null) {
-            count = 0;
-        }
-        if (count < Constants.HTTP_FAIL_RETRY_COUNT) {
-            Thread.sleep(new Random().nextInt(10 * 1000));
-            retryCount.set(++count);
-            return getByHttpClient(url);
-        }
-        return null;
-    }
-
-
 }

novel-crawl/src/main/java/com/java2nb/novel/core/crawl/RuleBean.java

@@ -1,24 +1,32 @@
 package com.java2nb.novel.core.crawl;
 
+import com.java2nb.novel.utils.Constants;
 import lombok.Data;
 
 import java.util.Map;
 
 /**
  * 爬虫解析规则bean
+ *
  * @author Administrator
  */
 @Data
 public class RuleBean {
 
+    /**
+     * 网页字符编码
+     */
+    private String charset = Constants.CRAWL_DEFAULT_CHARSET;
+
+
     /**
      * 小说更新列表url
-     * */
+     */
     private String updateBookListUrl;
 
     /**
      * 分类列表页URL规则
-     * */
+     */
     private String bookListUrl;
 
     private Map<String, String> catIdRule;
@@ -37,6 +45,7 @@ public class RuleBean {
     private String visitCountPatten;
     private String descStart;
     private String descEnd;
+    private String filterDesc;
     private String upadateTimePatten;
     private String upadateTimeFormatPatten;
     private String bookIndexUrl;
@@ -51,5 +60,7 @@ public class RuleBean {
 
     private String bookIndexStart;
 
+    private String filterContent;
+
 
 }

novel-crawl/src/main/java/com/java2nb/novel/core/listener/StarterListener.java

@@ -1,21 +1,23 @@
 package com.java2nb.novel.core.listener;
 
 import com.fasterxml.jackson.databind.ObjectMapper;
-import com.java2nb.novel.core.crawl.ChapterBean;
 import com.java2nb.novel.core.crawl.CrawlParser;
 import com.java2nb.novel.core.crawl.RuleBean;
-import com.java2nb.novel.entity.*;
+import com.java2nb.novel.entity.Book;
+import com.java2nb.novel.entity.BookIndex;
+import com.java2nb.novel.entity.CrawlSingleTask;
+import com.java2nb.novel.entity.CrawlSource;
 import com.java2nb.novel.service.BookService;
 import com.java2nb.novel.service.CrawlService;
 import com.java2nb.novel.utils.Constants;
+import jakarta.servlet.ServletContext;
 import lombok.RequiredArgsConstructor;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.time.DateUtils;
 import org.springframework.beans.factory.annotation.Value;
+import org.springframework.boot.web.servlet.ServletContextInitializer;
+import org.springframework.stereotype.Component;
 
-import javax.servlet.ServletContextEvent;
-import javax.servlet.ServletContextListener;
-import javax.servlet.annotation.WebListener;
 import java.util.Date;
 import java.util.List;
 import java.util.Map;
@@ -24,20 +26,22 @@ import java.util.concurrent.TimeUnit;
 /**
  * @author Administrator
  */
-@WebListener
+@Component
 @Slf4j
 @RequiredArgsConstructor
-public class StarterListener implements ServletContextListener {
+public class StarterListener implements ServletContextInitializer {
 
     private final BookService bookService;
 
     private final CrawlService crawlService;
 
+    private final CrawlParser crawlParser;
+
     @Value("${crawl.update.thread}")
     private int updateThreadCount;
 
     @Override
-    public void contextInitialized(ServletContextEvent sce) {
+    public void onStartup(ServletContext servletContext) {
         for (int i = 0; i < updateThreadCount; i++) {
             new Thread(() -> {
                 log.info("程序启动,开始执行自动更新线程。。。");
@@ -56,20 +60,24 @@ public class StarterListener implements ServletContextListener {
                                 CrawlSource source = crawlService.queryCrawlSource(needUpdateBook.getCrawlSourceId());
                                 RuleBean ruleBean = new ObjectMapper().readValue(source.getCrawlRule(), RuleBean.class);
                                 //解析小说基本信息
-                                CrawlParser.parseBook(ruleBean, needUpdateBook.getCrawlBookId(),book -> {
+                                crawlParser.parseBook(ruleBean, needUpdateBook.getCrawlBookId(), book -> {
                                     //这里只做老书更新
                                     book.setId(needUpdateBook.getId());
                                     book.setWordCount(needUpdateBook.getWordCount());
-                                    if (needUpdateBook.getPicUrl() != null && needUpdateBook.getPicUrl().contains(Constants.LOCAL_PIC_PREFIX)) {
+                                    if (needUpdateBook.getPicUrl() != null && needUpdateBook.getPicUrl()
+                                        .contains(Constants.LOCAL_PIC_PREFIX)) {
                                         //本地图片则不更新
                                         book.setPicUrl(null);
                                     }
                                     //查询已存在的章节
-                                    Map<Integer, BookIndex> existBookIndexMap = bookService.queryExistBookIndexMap(needUpdateBook.getId());
+                                    Map<Integer, BookIndex> existBookIndexMap = bookService.queryExistBookIndexMap(
+                                        needUpdateBook.getId());
                                     //解析章节目录
-                                    CrawlParser.parseBookIndexAndContent(needUpdateBook.getCrawlBookId(), book, ruleBean, existBookIndexMap,chapter -> {
-                                        bookService.updateBookAndIndexAndContent(book, chapter.getBookIndexList(), chapter.getBookContentList(), existBookIndexMap);
-                                    });
+                                    crawlParser.parseBookIndexAndContent(needUpdateBook.getCrawlBookId(), book,
+                                        ruleBean, existBookIndexMap,
+                                        chapter -> bookService.updateBookAndIndexAndContent(book,
+                                            chapter.getBookIndexList(),
+                                            chapter.getBookContentList(), existBookIndexMap), null);
                                 });
                             } catch (Exception e) {
                                 log.error(e.getMessage(), e);
@@ -88,7 +96,6 @@ public class StarterListener implements ServletContextListener {
 
         }
 
-
         new Thread(() -> {
             log.info("程序启动,开始执行单本采集任务线程。。。");
             while (true) {
@@ -102,8 +109,8 @@ public class StarterListener implements ServletContextListener {
                         //查询爬虫规则
                         CrawlSource source = crawlService.queryCrawlSource(task.getSourceId());
                         RuleBean ruleBean = new ObjectMapper().readValue(source.getCrawlRule(), RuleBean.class);
-
-                        if (crawlService.parseBookAndSave(task.getCatId(), ruleBean, task.getSourceId(), task.getSourceBookId())) {
+                        if (crawlService.parseBookAndSave(task.getCatId(), ruleBean, task.getSourceId(),
+                            task.getSourceBookId(), task)) {
                             //采集成功
                             crawlStatus = 1;
                         }
@@ -116,6 +123,7 @@ public class StarterListener implements ServletContextListener {
                 } catch (Exception e) {
                     log.error(e.getMessage(), e);
                 }
+
                 if (task != null) {
                     crawlService.updateCrawlSingleTask(task, crawlStatus);
                 }

novel-crawl/src/main/java/com/java2nb/novel/core/schedule/CrawlThreadMonitor.java

@@ -1,61 +0,0 @@
-package com.java2nb.novel.core.schedule;
-
-
-import com.java2nb.novel.core.cache.CacheKey;
-import com.java2nb.novel.core.cache.CacheService;
-import com.java2nb.novel.entity.CrawlSource;
-import com.java2nb.novel.service.CrawlService;
-import io.github.xxyopen.util.ThreadUtil;
-import lombok.RequiredArgsConstructor;
-import lombok.extern.slf4j.Slf4j;
-import org.springframework.scheduling.annotation.Scheduled;
-import org.springframework.stereotype.Service;
-
-import java.util.List;
-import java.util.Set;
-
-/**
- * 爬虫线程监控器,监控执行完成的爬虫源，并修改状态
- *
- * @author Administrator
- */
-@Service
-@RequiredArgsConstructor
-@Slf4j
-public class CrawlThreadMonitor {
-
-    private final CacheService cacheService;
-
-    private final CrawlService crawlService;
-
-    @Scheduled(fixedRate = 1000 * 60 * 5)
-    public void monitor() {
-
-        //查询需要监控的正在运行的爬虫源
-        List<CrawlSource> sources = crawlService.queryCrawlSourceByStatus((byte) 1);
-
-        for (CrawlSource source : sources) {
-            Set<Long> runningCrawlThreadIds = (Set<Long>) cacheService.getObject(CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + source.getId());
-            boolean sourceStop = true;
-            if (runningCrawlThreadIds != null) {
-                for (Long threadId : runningCrawlThreadIds) {
-                    Thread thread = ThreadUtil.findThread(threadId);
-
-                    if (thread != null && thread.isAlive()) {
-                        //有活跃线程，说明该爬虫源正在运行，数据库中状态正确，不需要修改
-                        sourceStop = false;
-
-                    }
-
-                }
-            }
-
-            if (sourceStop) {
-                crawlService.updateCrawlSourceStatus(source.getId(), (byte) 0);
-            }
-
-
-        }
-
-    }
-}

novel-crawl/src/main/java/com/java2nb/novel/service/CrawlService.java

@@ -47,13 +47,15 @@ public interface CrawlService {
 
     /**
      * 采集并保存小说
+     *
      * @param catId    分类ID
-     * @param bookId 小说ID
-     * @param sourceId 源ID
      * @param ruleBean 采集规则\
+     * @param sourceId 源ID
+     * @param bookId   小说ID
+     * @param task
      * @return true:成功，false:失败
-     * */
-    boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId);
+     */
+    boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId, CrawlSingleTask task) throws InterruptedException;
 
     /**
      * 根据爬虫状态查询爬虫源集合
@@ -117,4 +119,9 @@ public interface CrawlService {
      * @return
      */
     CrawlSource getCrawlSource(Integer id);
+
+    /**
+     * 采集任务进度查询
+     * */
+    Integer getTaskProgress(Long taskId);
 }

novel-crawl/src/main/java/com/java2nb/novel/service/impl/CrawlServiceImpl.java

@@ -2,17 +2,10 @@ package com.java2nb.novel.service.impl;
 
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.github.pagehelper.PageHelper;
-import io.github.xxyopen.model.page.PageBean;
-import com.java2nb.novel.core.cache.CacheKey;
 import com.java2nb.novel.core.cache.CacheService;
 import com.java2nb.novel.core.crawl.CrawlParser;
 import com.java2nb.novel.core.crawl.RuleBean;
 import com.java2nb.novel.core.enums.ResponseStatus;
-import io.github.xxyopen.model.page.builder.pagehelper.PageBuilder;
-import io.github.xxyopen.util.IdWorker;
-import io.github.xxyopen.util.ThreadUtil;
-import io.github.xxyopen.web.exception.BusinessException;
-import io.github.xxyopen.web.util.BeanUtil;
 import com.java2nb.novel.entity.Book;
 import com.java2nb.novel.entity.CrawlSingleTask;
 import com.java2nb.novel.entity.CrawlSource;
@@ -22,9 +15,15 @@ import com.java2nb.novel.mapper.CrawlSourceDynamicSqlSupport;
 import com.java2nb.novel.mapper.CrawlSourceMapper;
 import com.java2nb.novel.service.BookService;
 import com.java2nb.novel.service.CrawlService;
+import com.java2nb.novel.utils.CrawlHttpClient;
 import com.java2nb.novel.vo.CrawlSingleTaskVO;
 import com.java2nb.novel.vo.CrawlSourceVO;
-import io.github.xxyopen.web.util.SpringUtil;
+import io.github.xxyopen.model.page.PageBean;
+import io.github.xxyopen.model.page.builder.pagehelper.PageBuilder;
+import io.github.xxyopen.util.IdWorker;
+import io.github.xxyopen.util.ThreadUtil;
+import io.github.xxyopen.web.exception.BusinessException;
+import io.github.xxyopen.web.util.BeanUtil;
 import lombok.RequiredArgsConstructor;
 import lombok.SneakyThrows;
 import lombok.extern.slf4j.Slf4j;
@@ -33,12 +32,12 @@ import org.mybatis.dynamic.sql.render.RenderingStrategies;
 import org.mybatis.dynamic.sql.select.render.SelectStatementProvider;
 import org.springframework.stereotype.Service;
 
+import java.time.Duration;
 import java.util.*;
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
-import static com.java2nb.novel.core.utils.HttpUtil.getByHttpClientWithChrome;
 import static com.java2nb.novel.mapper.CrawlSourceDynamicSqlSupport.*;
 import static org.mybatis.dynamic.sql.SqlBuilder.isEqualTo;
 import static org.mybatis.dynamic.sql.select.SelectDSL.select;
@@ -51,6 +50,7 @@ import static org.mybatis.dynamic.sql.select.SelectDSL.select;
 @Slf4j
 public class CrawlServiceImpl implements CrawlService {
 
+    private final CrawlParser crawlParser;
 
     private final CrawlSourceMapper crawlSourceMapper;
 
@@ -58,10 +58,14 @@ public class CrawlServiceImpl implements CrawlService {
 
     private final BookService bookService;
 
-    private final CacheService cacheService;
-
     private final IdWorker idWorker = IdWorker.INSTANCE;
 
+    private final CrawlHttpClient crawlHttpClient;
+
+    private final Map<Integer, Byte> crawlSourceStatusMap = new HashMap<>();
+
+    private final Map<Integer, Set<Long>> runningCrawlThread = new HashMap<>();
+
 
     @Override
     public void addCrawlSource(CrawlSource source) {
@@ -71,6 +75,7 @@ public class CrawlServiceImpl implements CrawlService {
         crawlSourceMapper.insertSelective(source);
 
     }
+
     @Override
     public void updateCrawlSource(CrawlSource source) {
         if (source.getId() != null) {
@@ -89,6 +94,7 @@ public class CrawlServiceImpl implements CrawlService {
             }
         }
     }
+
     @Override
     public PageBean<CrawlSource> listCrawlByPage(int page, int pageSize) {
         PageHelper.startPage(page, pageSize);
@@ -98,6 +104,8 @@ public class CrawlServiceImpl implements CrawlService {
             .build()
             .render(RenderingStrategies.MYBATIS3);
         List<CrawlSource> crawlSources = crawlSourceMapper.selectMany(render);
+        crawlSources.forEach(crawlSource -> crawlSource.setSourceStatus(
+            Optional.ofNullable(crawlSourceStatusMap.get(crawlSource.getId())).orElse((byte) 0)));
         PageBean<CrawlSource> pageBean = PageBuilder.build(crawlSources);
         pageBean.setList(BeanUtil.copyList(crawlSources, CrawlSourceVO.class));
         return pageBean;
@@ -107,13 +115,13 @@ public class CrawlServiceImpl implements CrawlService {
     @Override
     public void openOrCloseCrawl(Integer sourceId, Byte sourceStatus) {
 
-        //判断是开启还是关闭，如果是关闭，则修改数据库状态后获取该爬虫正在运行的线程集合并全部停止
-        //如果是开启，先查询数据库中状态，判断该爬虫源是否还在运行，如果在运行，则忽略，
-        // 如果没有则修改数据库状态，并启动线程爬取小说数据加入到runningCrawlThread中
+        // 判断是开启还是关闭，如果是关闭，则获取该爬虫源正在运行的线程集合并全部中断
+        // 如果是开启，先判断该爬虫源是否还在运行，如果在运行，则忽略，如果没有运行则启动线程爬取小说数据并加入到runningCrawlThread中
+        // 最后，保存爬虫源状态
         if (sourceStatus == (byte) 0) {
-            //关闭,直接修改数据库状态，并直接修改数据库状态后获取该爬虫正在运行的线程集合全部停止
-            SpringUtil.getBean(CrawlService.class).updateCrawlSourceStatus(sourceId, sourceStatus);
-            Set<Long> runningCrawlThreadId = (Set<Long>) cacheService.getObject(CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + sourceId);
+            // 关闭
+            // 将该爬虫源正在运行的线程集合全部停止
+            Set<Long> runningCrawlThreadId = runningCrawlThread.get(sourceId);
             if (runningCrawlThreadId != null) {
                 for (Long ThreadId : runningCrawlThreadId) {
                     Thread thread = ThreadUtil.findThread(ThreadId);
@@ -126,15 +134,12 @@ public class CrawlServiceImpl implements CrawlService {
 
         } else {
             // 开启
-            //查询爬虫源状态和规则
-            CrawlSource source = queryCrawlSource(sourceId);
-            Byte realSourceStatus = source.getSourceStatus();
-
+            Byte realSourceStatus = Optional.ofNullable(crawlSourceStatusMap.get(sourceId)).orElse((byte) 0);
             if (realSourceStatus == (byte) 0) {
-                //该爬虫源已经停止运行了,修改数据库状态，并启动线程爬取小说数据加入到runningCrawlThread中
-                SpringUtil.getBean(CrawlService.class).updateCrawlSourceStatus(sourceId, sourceStatus);
+                // 查询爬虫源规则
+                CrawlSource source = queryCrawlSource(sourceId);
+                //该爬虫源已经停止运行了,启动线程爬取小说数据并将线程加入到runningCrawlThread中
                 RuleBean ruleBean = new ObjectMapper().readValue(source.getCrawlRule(), RuleBean.class);
-
                 Set<Long> threadIds = new HashSet<>();
                 //按分类开始爬虫解析任务
                 for (int i = 1; i < 8; i++) {
@@ -143,21 +148,21 @@ public class CrawlServiceImpl implements CrawlService {
                     thread.start();
                     //thread加入到监控缓存中
                     threadIds.add(thread.getId());
-
                 }
-                cacheService.setObject(CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + sourceId, threadIds);
-
+                runningCrawlThread.put(sourceId, threadIds);
+            }
 
         }
 
-
-        }
+        // 保存爬虫源状态
+        crawlSourceStatusMap.put(sourceId, sourceStatus);
 
     }
 
     @Override
     public CrawlSource queryCrawlSource(Integer sourceId) {
-        SelectStatementProvider render = select(CrawlSourceDynamicSqlSupport.sourceStatus, CrawlSourceDynamicSqlSupport.crawlRule)
+        SelectStatementProvider render = select(CrawlSourceDynamicSqlSupport.sourceStatus,
+            CrawlSourceDynamicSqlSupport.crawlRule)
             .from(crawlSource)
             .where(id, isEqualTo(sourceId))
             .build()
@@ -189,6 +194,16 @@ public class CrawlServiceImpl implements CrawlService {
         List<CrawlSingleTask> crawlSingleTasks = crawlSingleTaskMapper.selectMany(render);
         PageBean<CrawlSingleTask> pageBean = PageBuilder.build(crawlSingleTasks);
         pageBean.setList(BeanUtil.copyList(crawlSingleTasks, CrawlSingleTaskVO.class));
+        for (CrawlSingleTask crawlSingleTask : pageBean.getList()) {
+            if (crawlSingleTask.getTaskStatus() == 2
+                && crawlParser.getCrawlTaskProgress(crawlSingleTask.getId()) != null) {
+                // 如果排队中的任务有任务进度，将排队中的任务状态修改成采集中并设置任务进度
+                crawlSingleTask.setTaskStatus((byte) 3);
+                crawlSingleTask.setCrawlChapters(crawlParser.getCrawlTaskProgress(crawlSingleTask.getId()));
+                // 只会有一个任务在采集中
+                break;
+            }
+        }
         return pageBean;
     }
 
@@ -200,7 +215,8 @@ public class CrawlServiceImpl implements CrawlService {
     @Override
     public CrawlSingleTask getCrawlSingleTask() {
 
-        List<CrawlSingleTask> list = crawlSingleTaskMapper.selectMany(select(CrawlSingleTaskDynamicSqlSupport.crawlSingleTask.allColumns())
+        List<CrawlSingleTask> list = crawlSingleTaskMapper.selectMany(
+            select(CrawlSingleTaskDynamicSqlSupport.crawlSingleTask.allColumns())
                 .from(CrawlSingleTaskDynamicSqlSupport.crawlSingleTask)
                 .where(CrawlSingleTaskDynamicSqlSupport.taskStatus, isEqualTo((byte) 2))
                 .orderBy(CrawlSingleTaskDynamicSqlSupport.createTime)
@@ -220,18 +236,24 @@ public class CrawlServiceImpl implements CrawlService {
             // 当采集成功或者采集次数等于5，则更新采集最终状态，并停止采集
             task.setTaskStatus(status);
         }
+        if (status == 1) {
+            // 当采集成功，保存采集的章节数量
+            task.setCrawlChapters(crawlParser.getCrawlTaskProgress(task.getId()));
+        }
         crawlSingleTaskMapper.updateByPrimaryKeySelective(task);
+        // 删除任务进度
+        crawlParser.removeCrawlTaskProgress(task.getId());
 
     }
 
     @Override
     public CrawlSource getCrawlSource(Integer id) {
-            Optional<CrawlSource> opt=crawlSourceMapper.selectByPrimaryKey(id);
-            if(opt.isPresent()) {
-                CrawlSource crawlSource =opt.get();
-                return crawlSource;
+        return crawlSourceMapper.selectByPrimaryKey(id).orElse(null);
     }
-            return null;
+
+    @Override
+    public Integer getTaskProgress(Long taskId) {
+        return Optional.ofNullable(crawlParser.getCrawlTaskProgress(taskId)).orElse(0);
     }
 
     /**
@@ -240,6 +262,11 @@ public class CrawlServiceImpl implements CrawlService {
     @Override
     public void parseBookList(int catId, RuleBean ruleBean, Integer sourceId) {
 
+        String catIdRule = ruleBean.getCatIdRule().get("catId" + catId);
+        if (StringUtils.isBlank(catIdRule)) {
+            return;
+        }
+
         //当前页码1
         int page = 1;
         int totalPage = page;
@@ -247,14 +274,21 @@ public class CrawlServiceImpl implements CrawlService {
         while (page <= totalPage) {
 
             try {
-
-                if (StringUtils.isNotBlank(ruleBean.getCatIdRule().get("catId" + catId))) {
+                String catBookListUrl;
+                if (StringUtils.isNotBlank(ruleBean.getBookListUrl())) {
+                    // 兼容老规则
                     // 拼接分类URL
-                    String catBookListUrl = ruleBean.getBookListUrl()
-                            .replace("{catId}", ruleBean.getCatIdRule().get("catId" + catId))
+                    catBookListUrl = ruleBean.getBookListUrl()
+                        .replace("{catId}", catIdRule)
                         .replace("{page}", page + "");
+                } else {
+                    // 新规则
+                    // 拼接分类URL
+                    catBookListUrl = catIdRule.replace("{page}", page + "");
+                }
+                log.info("catBookListUrl：{}", catBookListUrl);
 
-                    String bookListHtml = getByHttpClientWithChrome(catBookListUrl);
+                String bookListHtml = crawlHttpClient.get(catBookListUrl, ruleBean.getCharset());
                 if (bookListHtml != null) {
                     Pattern bookIdPatten = Pattern.compile(ruleBean.getBookIdPatten());
                     Matcher bookIdMatcher = bookIdPatten.matcher(bookListHtml);
@@ -268,14 +302,18 @@ public class CrawlServiceImpl implements CrawlService {
                                 return;
                             }
 
-
                             String bookId = bookIdMatcher.group(1);
-                                parseBookAndSave(catId, ruleBean, sourceId, bookId);
+                            parseBookAndSave(catId, ruleBean, sourceId, bookId, null);
+                        } catch (InterruptedException e) {
+                            log.error(e.getMessage(), e);
+                            //1.阻塞过程（使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时）
+                            //捕获中断异常InterruptedException来退出线程。
+                            //2.非阻塞过程中通过判断中断标志来退出线程。
+                            return;
                         } catch (Exception e) {
                             log.error(e.getMessage(), e);
                         }
 
-
                         isFindBookId = bookIdMatcher.find();
                     }
 
@@ -287,26 +325,44 @@ public class CrawlServiceImpl implements CrawlService {
                         totalPage = Integer.parseInt(totalPageMatcher.group(1));
 
                     }
-
-
-                    }
                 }
+            } catch (InterruptedException e) {
+                log.error(e.getMessage(), e);
+                //1.阻塞过程（使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时）
+                //捕获中断异常InterruptedException来退出线程。
+                //2.非阻塞过程中通过判断中断标志来退出线程。
+                return;
             } catch (Exception e) {
                 log.error(e.getMessage(), e);
             }
-
+            if (page >= totalPage) {
+                // 第一遍采集完成，翻到第一页，继续第二次采集，适用于分页数比较少的最近更新列表
+                page = 1;
+                try {
+                    // 第一遍采集完成，休眠1分钟
+                    Thread.sleep(Duration.ofMinutes(1));
+                } catch (InterruptedException e) {
+                    log.error(e.getMessage(), e);
+                    //1.阻塞过程（使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时）
+                    //捕获中断异常InterruptedException来退出线程。
+                    //2.非阻塞过程中通过判断中断标志来退出线程。
+                    return;
+                }
+            } else {
                 page += 1;
             }
+        }
 
 
     }
 
     @Override
-    public boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId) {
+    public boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId, CrawlSingleTask task)
+        throws InterruptedException {
 
         final AtomicBoolean parseResult = new AtomicBoolean(false);
 
-        CrawlParser.parseBook(ruleBean, bookId, book -> {
+        crawlParser.parseBook(ruleBean, bookId, book -> {
             if (book.getBookName() == null || book.getAuthorName() == null) {
                 return;
             }
@@ -330,9 +386,11 @@ public class CrawlServiceImpl implements CrawlService {
                 book.setCrawlLastTime(new Date());
                 book.setId(idWorker.nextId());
                 //解析章节目录
-                boolean parseIndexContentResult = CrawlParser.parseBookIndexAndContent(bookId, book, ruleBean, new HashMap<>(0), chapter -> {
-                    bookService.saveBookAndIndexAndContent(book, chapter.getBookIndexList(), chapter.getBookContentList());
-                });
+                boolean parseIndexContentResult = crawlParser.parseBookIndexAndContent(bookId, book, ruleBean,
+                    new HashMap<>(0), chapter -> {
+                        bookService.saveBookAndIndexAndContent(book, chapter.getBookIndexList(),
+                            chapter.getBookContentList());
+                    }, task);
                 parseResult.set(parseIndexContentResult);
 
             } else {
@@ -356,11 +414,13 @@ public class CrawlServiceImpl implements CrawlService {
 
     @Override
     public List<CrawlSource> queryCrawlSourceByStatus(Byte sourceStatus) {
-        SelectStatementProvider render = select(CrawlSourceDynamicSqlSupport.id, CrawlSourceDynamicSqlSupport.sourceStatus, CrawlSourceDynamicSqlSupport.crawlRule)
+        SelectStatementProvider render = select(CrawlSourceDynamicSqlSupport.id,
+            CrawlSourceDynamicSqlSupport.sourceStatus, CrawlSourceDynamicSqlSupport.crawlRule)
             .from(crawlSource)
             .where(CrawlSourceDynamicSqlSupport.sourceStatus, isEqualTo(sourceStatus))
             .build()
             .render(RenderingStrategies.MYBATIS3);
         return crawlSourceMapper.selectMany(render);
     }
+
 }

novel-crawl/src/main/java/com/java2nb/novel/utils/Constants.java

@@ -7,7 +7,7 @@ public class Constants {
 
     /**
      * 本地图片保存前缀
-     * */
+     */
     public static final String LOCAL_PIC_PREFIX = "/localPic/";
 
     /**
@@ -23,5 +23,10 @@ public class Constants {
     /**
      * 爬取小说http请求失败重试次数
      */
-    public static final Integer HTTP_FAIL_RETRY_COUNT = 5;
+    public static final Integer HTTP_FAIL_RETRY_COUNT = 3;
+
+    /**
+     * 爬虫默认编码
+     */
+    public static final String CRAWL_DEFAULT_CHARSET = "UTF-8";
 }

novel-crawl/src/main/java/com/java2nb/novel/utils/CrawlHttpClient.java

@@ -0,0 +1,53 @@
+package com.java2nb.novel.utils;
+
+import com.java2nb.novel.core.utils.HttpUtil;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Component;
+
+import java.util.Objects;
+import java.util.Random;
+
+/**
+ * @author Administrator
+ */
+@Slf4j
+@Component
+public class CrawlHttpClient {
+
+    @Value("${crawl.interval.min}")
+    private Integer intervalMin;
+
+    @Value("${crawl.interval.max}")
+    private Integer intervalMax;
+
+    private final Random random = new Random();
+
+    private static final ThreadLocal<Integer> RETRY_COUNT = new ThreadLocal<>();
+
+    public String get(String url, String charset) throws InterruptedException {
+        if (Objects.nonNull(intervalMin) && Objects.nonNull(intervalMax) && intervalMax > intervalMin) {
+            Thread.sleep(random.nextInt(intervalMax - intervalMin + 1) + intervalMin);
+        }
+        String body = HttpUtil.getByHttpClientWithChrome(url, charset);
+        if (Objects.isNull(body) || body.length() < Constants.INVALID_HTML_LENGTH) {
+            return processErrorHttpResult(url, charset);
+        }
+        //成功获得html内容
+        return body;
+    }
+
+    private String processErrorHttpResult(String url, String charset) throws InterruptedException{
+        Integer count = RETRY_COUNT.get();
+        if (count == null) {
+            count = 0;
+        }
+        if (count < Constants.HTTP_FAIL_RETRY_COUNT) {
+            RETRY_COUNT.set(++count);
+            return get(url, charset);
+        }
+        RETRY_COUNT.remove();
+        return null;
+    }
+
+}

novel-crawl/src/main/resources/application-dev.yml

@@ -1,3 +1,3 @@
 spring:
-    profiles:
-        include: [common-dev]
+    config:
+        import: classpath:application-common-dev.yml

novel-crawl/src/main/resources/application-prod.yml

@@ -1,3 +1,3 @@
 spring:
-    profiles:
-        include: [common-prod]
+    config:
+        import: classpath:application-common-prod.yml

novel-crawl/src/main/resources/application.yml

@@ -14,12 +14,18 @@ admin:
   username: admin
   password: admin
 
+
+
+crawl:
+  update:
     #爬虫自动更新的线程数
     #建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
     #随着小说数量的增多可以逐渐增加，但建议不要超出CPU的线程数
-crawl:
-  update:
     thread: 1
+  # 采集间隔时间，单位：毫秒
+  interval:
+    min: 300
+    max: 500
 
 
 

novel-crawl/src/main/resources/logback-boot.xml

@@ -12,9 +12,6 @@
     <!-- ConsoleAppender：把日志输出到控制台 -->
     <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
         <encoder>
-            <!--
-            <pattern>%d %p (%file:%line\)- %m%n</pattern>
-             -->
             <pattern>${CONSOLE_LOG_PATTERN}</pattern>
             <!-- 控制台也要使用UTF-8，不要使用GBK，否则会中文乱码 -->
             <charset>UTF-8</charset>
@@ -22,40 +19,40 @@
     </appender>
 
     <!-- RollingFileAppender：滚动记录文件，先将日志记录到指定文件，当符合某个条件时，将日志记录到其他文件 -->
-    <!-- 以下的大概意思是：1.先按日期存日志，日期变了，将前一天的日志文件名重命名为XXX%日期%索引，新的日志仍然是demo.log -->
-    <!-- 2.如果日期没有发生变化，但是当前日志的文件大小超过1KB时，对当前日志进行分割 重命名 -->
+    <!-- 以下的大概意思是：1.先按日期存日志，日期变了，将前一天的日志文件名重命名为XXX%日期%索引，新的日志仍然是novel-crawl.log -->
+    <!-- 2.如果日期没有发生变化，但是当前日志的文件大小超过10MB时，对当前日志进行分割 重命名 -->
     <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
-
-        <File>logs/novel-crawl.log</File>
+        <!-- 设置日志文件路径 -->
+        <file>logs/novel-crawl.log</file>
         <!-- rollingPolicy:当发生滚动时，决定 RollingFileAppender 的行为，涉及文件移动和重命名。 -->
-        <!-- TimeBasedRollingPolicy： 最常用的滚动策略，它根据时间来制定滚动策略，既负责滚动也负责出发滚动 -->
-        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
+        <!-- SizeAndTimeBasedRollingPolicy：基于时间和文件大小的滚动策略 -->
+        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
             <!-- 活动文件的名字会根据fileNamePattern的值，每隔一段时间改变一次 -->
-            <!-- 文件名：logs/demo.2017-12-05.0.log -->
-            <fileNamePattern>logs/debug.%d.%i.log</fileNamePattern>
+            <!-- 文件名：logs/debug.2023-10-01.0.log -->
+            <fileNamePattern>logs/debug.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
             <!-- 每产生一个日志文件，该日志文件的保存期限为30天 -->
             <maxHistory>30</maxHistory>
-            <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
-                <!-- maxFileSize:这是活动文件的大小，默认值是10MB，测试时可改成1KB看效果 -->
+            <!-- 单个日志文件的最大大小 -->
             <maxFileSize>10MB</maxFileSize>
-            </timeBasedFileNamingAndTriggeringPolicy>
+            <!-- 所有日志文件的总大小限制 -->
+            <totalSizeCap>1GB</totalSizeCap>
         </rollingPolicy>
         <encoder>
             <!-- pattern节点，用来设置日志的输入格式 -->
-            <pattern>
-                %d %p (%file:%line\)- %m%n
-            </pattern>
+            <pattern>%d %p (%file:%line\)- %m%n</pattern>
             <!-- 记录日志的编码:此处设置字符集 - -->
             <charset>UTF-8</charset>
         </encoder>
     </appender>
+
     <!-- 控制台输出日志级别 -->
     <root level="INFO">
         <appender-ref ref="STDOUT" />
         <appender-ref ref="FILE" />
     </root>
+
     <!-- 指定项目中某个包，当有日志操作行为时的日志记录级别 -->
-    <!-- com.maijinjie.springboot 为根包，也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG -->
+    <!-- com.java2nb 为根包，也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG -->
     <!-- 级别依次为【从高到低】：FATAL > ERROR > WARN > INFO > DEBUG > TRACE -->
     <logger name="com.java2nb" level="DEBUG" additivity="false">
         <appender-ref ref="STDOUT" />

novel-crawl/src/main/resources/static/css/base.css

@@ -331,7 +331,7 @@ input::-webkit-input-placeholder, textarea::-webkit-input-placeholder {
 }
 
 .logo {
-    width: 198px;
+    width: 160px;
     float: left;
     padding: 23px 130px 0 0;
     display: block

novel-crawl/src/main/resources/templates/crawl/crawlSingleTask_add.html

@@ -110,7 +110,7 @@
 
                 } else if (data.code == 1001) {
                     //未登录
-                    location.href = '/user/login.html?originUrl=' + decodeURIComponent(location.href);
+                    location.href = '/user/login.html?originUrl=' + encodeURIComponent(location.href);
 
                 } else {
                     layer.alert(data.msg);

novel-crawl/src/main/resources/templates/crawl/crawlSingleTask_list.html

@@ -48,6 +48,9 @@
                             <th class="name">
                                 采集小说作者名
                             </th>
+                            <th class="goread">
+                                采集进度
+                            </th>
                             <th class="goread">
                                 采集次数
                             </th>
@@ -113,9 +116,15 @@
 <script src="/javascript/header.js" type="text/javascript"></script>
 <script src="/javascript/user.js" type="text/javascript"></script>
 <script language="javascript" type="text/javascript">
-    search(1, 10);
+    let curr = 1;
+    let limit = 10;
 
-    function search(curr, limit) {
+    search();
+    setInterval(function(){
+        search();
+    }, 10000);
+
+    function search() {
 
         $.ajax({
             type: "get",
@@ -140,10 +149,13 @@
                                 "                                " + crawlSource.authorName + "\n" +
                                 "                            </td>\n" +
                                 "                            <td class=\"goread\">\n" +
+                                "                                " + crawlSource.crawlChapters + "\n" + "章" +
+                                "                            </td>\n" +
+                                "                            <td class=\"goread\">\n" +
                                 "                                " + crawlSource.excCount + "\n" +
                                 "                            </td>\n" +
                                 "                            <td class=\"goread\">\n" +
-                                "                                " + (crawlSource.taskStatus == 0 ? '采集失败' : (crawlSource.taskStatus == 1 ? '采集成功' : (crawlSource.excCount > 0 ? '采集中' : '排队中'))) + "\n" +
+                                "                                " + (crawlSource.taskStatus == 0 ? '采集失败' : (crawlSource.taskStatus == 1 ? '采集成功' : (crawlSource.taskStatus == 3 || crawlSource.excCount > 0 ? '采集中' : '排队中'))) + "\n" +
                                 "                            </td>\n" +
                                 "                            <td class=\"name\" valsc=\"291|2037554|1\">"
                                 + crawlSource.createTime + "</td>\n" +
@@ -171,7 +183,9 @@
 
                                     //首次不执行
                                     if (!first) {
-                                        search(obj.curr, obj.limit);
+                                        curr = obj.curr;
+                                        limit = obj.limit;
+                                        search();
                                     } else {
 
                                     }

novel-crawl/src/main/resources/templates/crawl/crawlSource_add.html

@@ -54,6 +54,8 @@
                                 <!--示例：<b>https://m.xdingdiann.com/sort/0/1.html</b>
                                 <li><input type="text" id="updateBookListUrl" class="s_input icon_key"
                                            placeholder="小说更新列表url"></li>-->
+                                示例：<b>utf-8</b>
+                                <li><input type="text" id="charset" class="s_input icon_name" placeholder="网站编码"></li>
                                 示例：<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID，{page}代表分页页码)
                                 <li><input type="text" id="bookListUrl" class="s_input icon_key"
                                            placeholder="分类列表页URL规则"></li>
@@ -116,6 +118,9 @@
                                 示例：<b>&lt;/p&gt;</b>
                                 <li><input type="text" id="descEnd" class="s_input icon_key" placeholder="小说简介结束截取字符串：">
                                 </li>
+                                示例：<b>&lt;span\s+class="allshow"&gt;([^/]+)&lt;/span&gt;</b>
+                                <li><textarea id="filterDesc"
+                                              placeholder="过滤简介（多个内容换行）" rows="5" cols="52"></textarea></li>
                                 示例：<b>更新：(\d+-\d+-\d+\s\d+:\d+:\d+)&lt;/a&gt;</b>
                                 <li><input type="text" id="upadateTimePatten" class="s_input icon_key"
                                            placeholder="小说更新时间的正则表达式："></li>
@@ -144,6 +149,9 @@
                                 示例：<b>&lt;script&gt;</b>
                                 <li><input type="text" id="contentEnd" class="s_input icon_key"
                                            placeholder="小说内容结束截取字符串："></li>
+                                示例：<b>&lt;div\s+id=&quot;content_tip&quot;&gt;\s*&lt;b&gt;([^/]+)&lt;/b&gt;\s*&lt;/div&gt;</b>
+                                <li><textarea id="filterContent"
+                                              placeholder="过滤内容（多个内容换行）" rows="5" cols="52"></textarea></li>
 
                                 <li><input type="button" onclick="addCrawlSource()" name="btnRegister" value="提交"
                                            id="btnRegister" class="btn_red"></li>
@@ -333,6 +341,9 @@
 
         crawlRule.descEnd = descEnd;
 
+        var filterDesc = $("#filterDesc").val();
+        crawlRule.filterDesc = filterDesc;
+
         var upadateTimePatten = $("#upadateTimePatten").val();
 
         if (upadateTimePatten.length > 0) {
@@ -405,6 +416,14 @@
 
         crawlRule.contentEnd = contentEnd;
 
+        var filterContent = $("#filterContent").val();
+        crawlRule.filterContent = filterContent;
+
+        var charset = $('#charset').val();
+        if (charset) {
+            crawlRule.charset = charset;
+        }
+
 
         $.ajax({
             type: "POST",

novel-crawl/src/main/resources/templates/crawl/crawlSource_list.html

@@ -182,7 +182,7 @@
 
                 } else if (data.code == 1001) {
                     //未登录
-                    location.href = '/user/login.html?originUrl=' + decodeURIComponent(location.href);
+                    location.href = '/user/login.html?originUrl=' + encodeURIComponent(location.href);
 
                 } else {
                     layer.alert(data.msg);
@@ -226,7 +226,7 @@
 
                 } else if (data.code == 1001) {
                     //未登录
-                    location.href = '/user/login.html?originUrl=' + decodeURIComponent(location.href);
+                    location.href = '/user/login.html?originUrl=' + encodeURIComponent(location.href);
 
                 } else {
                     layer.alert(data.msg);

novel-crawl/src/main/resources/templates/crawl/crawlSource_update.html

@@ -55,6 +55,8 @@
                                 <!--示例：<b>https://m.xdingdiann.com/sort/0/1.html</b>
                                 <li><input type="text" id="updateBookListUrl" class="s_input icon_key"
                                            placeholder="小说更新列表url"></li>-->
+                                示例：<b>utf-8</b>
+                                <li><input type="text" id="charset" class="s_input icon_name" placeholder="网站编码"></li>
                                 示例：<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID，{page}代表分页页码)
                                 <li><input type="text" id="bookListUrl" class="s_input icon_key"
                                            placeholder="分类列表页URL规则"></li>
@@ -117,6 +119,9 @@
                                 示例：<b>&lt;/p&gt;</b>
                                 <li><input type="text" id="descEnd" class="s_input icon_key" placeholder="小说简介结束截取字符串：">
                                 </li>
+                                示例：<b>&lt;span\s+class="allshow"&gt;([^/]+)&lt;/span&gt;</b>
+                                <li><textarea id="filterDesc"
+                                              placeholder="过滤简介（多个内容换行）" rows="5" cols="52"></textarea></li>
                                 示例：<b>更新：(\d+-\d+-\d+\s\d+:\d+:\d+)&lt;/a&gt;</b>
                                 <li><input type="text" id="upadateTimePatten" class="s_input icon_key"
                                            placeholder="小说更新时间的正则表达式："></li>
@@ -145,6 +150,10 @@
                                 示例：<b>&lt;script&gt;</b>
                                 <li><input type="text" id="contentEnd" class="s_input icon_key"
                                            placeholder="小说内容结束截取字符串："></li>
+                                示例：<b>&lt;div\s+id=&quot;content_tip&quot;&gt;\s*&lt;b&gt;([^/]+)&lt;/b&gt;\s*&lt;/div&gt;</b>
+                                <li><textarea id="filterContent"
+                                              placeholder="过滤内容（多个内容换行）" rows="5" cols="52"></textarea></li>
+
 
                                 <li><input type="button" onclick="updateCrawlSource()" name="btnRegister" value="提交"
                                            id="btnRegister" class="btn_red"></li>
@@ -208,7 +217,7 @@
                         loadPage(data.data);
                     } else if (data.code == 1001) {
                         //未登录
-                        location.href = '/user/login.html?originUrl=' + decodeURIComponent(location.href);
+                        location.href = '/user/login.html?originUrl=' + encodeURIComponent(location.href);
 
                     } else {
                         layer.alert(data.msg);
@@ -260,6 +269,7 @@
             $("#visitCountPatten").val(crawlRule.visitCountPatten);
             $("#descStart").val(crawlRule.descStart);
             $("#descEnd").val(crawlRule.descEnd);
+            $("#filterDesc").val(crawlRule.filterDesc);
             $("#upadateTimePatten").val(crawlRule.upadateTimePatten);
             $("#upadateTimeFormatPatten").val(crawlRule.upadateTimeFormatPatten);
             $("#bookIndexUrl").val(crawlRule.bookIndexUrl);
@@ -269,6 +279,8 @@
             $("#bookContentUrl").val(crawlRule.bookContentUrl);
             $("#contentStart").val(crawlRule.contentStart);
             $("#contentEnd").val(crawlRule.contentEnd);
+            $("#filterContent").val(crawlRule.filterContent);
+            $("#charset").val(crawlRule.charset);
 
         }
     }
@@ -416,6 +428,9 @@
 
         crawlRule.descEnd = descEnd;
 
+        var filterDesc = $("#filterDesc").val();
+        crawlRule.filterDesc = filterDesc;
+
         var upadateTimePatten = $("#upadateTimePatten").val();
 
         if (upadateTimePatten.length > 0) {
@@ -488,6 +503,14 @@
 
         crawlRule.contentEnd = contentEnd;
 
+        var filterContent = $("#filterContent").val();
+        crawlRule.filterContent = filterContent;
+
+        var charset = $('#charset').val();
+        if (charset) {
+            crawlRule.charset = charset;
+        }
+
 
         $.ajax({
             type: "POST",

novel-crawl/src/main/resources/templates/login.html

@@ -6,9 +6,8 @@
     <meta name="description" content="">
     <meta name="author" content="">
     <title>爬虫管理系统登录</title>
-    <link href="/css/bootstrap.min.css" rel="stylesheet"
-          integrity="sha384-/Y6pD6FV/Vv2HJnA6t+vslU6fwYXjCFtcEpHbNJ0lyAFsXTsjBbfaDjzALeQsN6M" crossorigin="anonymous">
-    <link href="/css/signin.css" rel="stylesheet" crossorigin="anonymous"/>
+    <link href="/css/bootstrap.min.css" rel="stylesheet"/>
+    <link href="/css/signin.css" rel="stylesheet"/>
     <style>
         .btn-primary {
             color: #fff;

novel-front/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <artifactId>novel</artifactId>
         <groupId>com.java2nb</groupId>
-        <version>4.2.0</version>
+        <version>5.2.1</version>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
@@ -20,6 +20,12 @@
             <artifactId>novel-common</artifactId>
         </dependency>
 
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>2.2</version>
+        </dependency>
+
         <!--JWT(Json Web Token)登录支持-->
         <dependency>
             <groupId>io.jsonwebtoken</groupId>
@@ -48,7 +54,18 @@
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
-            <version>${jackson.version}</version>
+        </dependency>
+
+        <!-- AI -->
+        <dependency>
+            <groupId>org.springframework.ai</groupId>
+            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>org.lionsoul</groupId>
+            <artifactId>ip2region</artifactId>
+            <version>2.7.0</version>
         </dependency>
     </dependencies>
 
@@ -98,6 +115,9 @@
                                 <zip destfile='${project.build.directory}/build/${project.artifactId}.zip'>
                                     <zipfileset filemode="755" dir='${project.build.directory}/build/'/>
                                 </zip>
+
+                                <copy file="${basedir}/src/main/build/docker/Dockerfile"
+                                      tofile="${project.build.directory}/build/Dockerfile"/>
                             </tasks>
                         </configuration>
                     </execution>

novel-front/src/main/build/config/application-common-prod.yml

@@ -1,56 +0,0 @@
-#端口号
-server:
-  port: 8085
-
-#不分表的数据库配置
-spring:
-  datasource:
-    url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-    username: root
-    password: root
-  redis:
-    host: 127.0.0.1
-    port: 6379
-    password: test123456
-
-####使用shardingJdbc时，
-####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
-##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
-sharding:
-  jdbc:
-    datasource:
-      ds0:
-        jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
-        username: root
-        password: test123456
-
-pic:
-  save:
-    #图片保存方式, 1不保存，使用网络图片 ，2本地保存
-    type: 2
-    #图片保存路径
-    path: /var/pic
-
-
-#模版配置
-templates:
-  name: orange
-
-#小说内容保存配置
-content:
-  save:
-    storage: db # 小说内容存储方式：db-数据库，txt-TXT文本
-    path: /Users/xiongxiaoyang/books  # 小说TXT文本保存路径
-
-
-
-# HTTP 代理配置
-http:
-  proxy:
-    # 是否开启 HTTP 代理，true-开启，false-不开启
-    enabled: false
-    # 代理 IP
-    ip: u493.kdltps.com
-    # 代理端口号
-    port: 15818
-

novel-front/src/main/build/config/application.yml

@@ -0,0 +1,68 @@
+#端口号
+server:
+  port: 8085
+
+spring:
+  data:
+    redis:
+      #Redis服务器IP
+      host: 127.0.0.1
+      #Redis服务器连接端口
+      port: 6379
+      #Redis服务器连接密码
+      password: test123456
+      #连接超时时间（毫秒）
+      timeout: 10000
+
+
+pic:
+  save:
+    #图片保存方式, 1不保存，使用网络图片 ，2本地保存
+    type: 2
+    #图片保存路径
+    path: /var/pic
+
+
+#模版配置
+templates:
+  name: green
+
+#小说内容保存配置
+content:
+  save:
+    storage: db # 小说内容存储方式：db-数据库，txt-TXT文本
+    path: /Users/xiongxiaoyang/books  # 小说TXT文本保存路径
+
+
+
+# HTTP 代理配置
+http:
+  proxy:
+    # 是否开启 HTTP 代理，true-开启，false-不开启
+    enabled: false
+    # 代理 IP
+    ip: us.swiftproxy.net
+    # 代理端口号
+    port: 7878
+    # 代理用户名
+    username: swiftproxy_u
+    # 代理密码
+    password: swiftproxy_p
+
+
+--- #--------------------- Spring AI 配置----------------------
+spring:
+  ai:
+    openai:
+      image:
+        enabled: true
+        base-url: https://api.siliconflow.cn
+        api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
+        options:
+          model: Kwai-Kolors/Kolors
+          response_format: URL
+      api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
+      base-url: https://api.siliconflow.cn
+      chat:
+        options:
+          model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

novel-front/src/main/build/config/shardingsphere-jdbc.yml

@@ -0,0 +1,47 @@
+mode:
+  # 单机模式
+  type: Standalone
+  # 元数据持久化
+  repository:
+    # 数据库持久化
+    type: JDBC
+
+# 数据源配置
+dataSources:
+  ds_1:
+    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
+    driverClassName: com.mysql.cj.jdbc.Driver
+    jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
+    username: root
+    password: test123456
+# 规则配置
+rules:
+  - !SINGLE
+    tables:
+      - "*.*"
+  - !SHARDING
+    tables: # 数据分片规则配置
+      book_content:
+        # 分库策略，缺省表示使用默认分库策略
+        actualDataNodes: ds_${1}.book_content${0..9}
+        # 分表策略
+        tableStrategy:
+          standard:
+            # 分片列名称
+            shardingColumn: index_id
+            # 分片算法名称
+            shardingAlgorithmName: bookContentSharding
+
+    shardingAlgorithms:
+      bookContentSharding:
+        # 行表达式分片算法，使用 Groovy 的表达式，提供对 SQL 语句中的 = 和 IN 的分片操作支持
+        type: INLINE
+        props:
+          # 分片算法的行表达式
+          algorithm-expression: book_content${index_id % 10}
+
+
+
+props:
+  # 是否在日志中打印 SQL
+  sql-show: true

novel-front/src/main/build/docker/Dockerfile

@@ -1,9 +1,9 @@
 FROM openjdk:8
-ADD novel-front-4.2.0.jar /root
+ADD novel-front.jar /root
 ENV dburl=""
 ENV username=""
 ENV	password=""
 ENV redishost = ""
 ENV redisport = ""
 ENV redispwd = ""
-ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-front-4.2.0.jar"]
+ENTRYPOINT ["sh","-c","java -Dspring.datasource.url=${dburl} -Dspring.datasource.username=${username} -Dspring.datasource.password=${password} -Dspring.redis.host=${redishost} -Dspring.redis.port=${redisport} -Dspring.redis.password=${redispwd} -jar /root/novel-front.jar"]