Compare commits

..

No commits in common. "develop_xxy" and "v4.4.0" have entirely different histories.

112 changed files with 1321 additions and 2375 deletions

View File

@ -1,79 +0,0 @@
name: Create novel-plus Maven Release with ZIPs
on:
push:
# 匹配所有以'v'开头的标签
tags:
- 'v*'
jobs:
build-and-release:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Set up JDK 21
uses: actions/setup-java@v3
with:
java-version: '21'
# 可选默认是 temurin也可以选择其他发行版
distribution: 'temurin'
- name: Build project with Maven
run: mvn clean install -DskipTests=true -Pcentral-repo
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
# 使用 tag_name 而不是 github.ref
tag_name: ${{ github.ref_name }}
release_name: novel-plus ${{ github.ref_name }}
draft: false
prerelease: false
# 使用 action 来替代直接 curl 进行上传
- name: Upload sql.zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ github.workspace }}/novel-common/target/build/sql.zip
asset_name: sql.zip
asset_content_type: application/zip
- name: Upload novel-crawl.zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ github.workspace }}/novel-crawl/target/build/novel-crawl.zip
asset_name: novel-crawl.zip
asset_content_type: application/zip
- name: Upload novel-front.zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ github.workspace }}/novel-front/target/build/novel-front.zip
asset_name: novel-front.zip
asset_content_type: application/zip
- name: Upload novel-admin.zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ github.workspace }}/novel-admin/target/build/novel-admin.zip
asset_name: novel-admin.zip
asset_content_type: application/zip

View File

@ -1,11 +1,11 @@
<p align="center"> [![index]( https://youdoc.github.io/img/tencent.jpg )]( https://cloud.tencent.com/act/cps/redirect?redirect=2446&cps_key=736e609d66e0ac4e57813316cec6fd0b&from=console )
<a href="https://cloud.tencent.com/act/cps/redirect?redirect=2446&cps_key=736e609d66e0ac4e57813316cec6fd0b&from=console"><img src="https://youdoc.github.io/img/tencent.jpg" alt="AD" ></a>
</p>
<p align="center"> <p align="center">
<a href='https://github.com/201206030/novel-plus'><img alt="Github stars" src="https://img.shields.io/github/stars/201206030/novel-plus?logo=github"></a> <a href='https://github.com/201206030/novel-plus'><img alt="Github stars" src="https://img.shields.io/github/stars/201206030/novel-plus?logo=github"></a>
<a href='https://github.com/201206030/novel-plus'><img alt="Github forks" src="https://img.shields.io/github/forks/201206030/novel-plus?logo=github"></a> <a href='https://github.com/201206030/novel-plus'><img alt="Github forks" src="https://img.shields.io/github/forks/201206030/novel-plus?logo=github"></a>
<a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee stars" src="https://gitee.com/novel_dev_team/novel-plus/badge/star.svg?theme=gitee"></a> <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee stars" src="https://gitee.com/novel_dev_team/novel-plus/badge/star.svg?theme=gitee"></a>
<a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee forks" src="https://gitee.com/novel_dev_team/novel-plus/badge/fork.svg?theme=gitee"></a> <a href='https://gitee.com/novel_dev_team/novel-plus'><img alt="Gitee forks" src="https://gitee.com/novel_dev_team/novel-plus/badge/fork.svg?theme=gitee"></a>
<a href="https://github.com/201206030/novel-plus"><img src="https://visitor-badge.glitch.me/badge?page_id=201206030.novel-plus" alt="visitors"></a>
</p> </p>
<p align="center"> <p align="center">
@ -16,7 +16,7 @@
novel-plus 是一个多端PCWAP阅读功能完善的原创文学 CMS novel-plus 是一个多端PCWAP阅读功能完善的原创文学 CMS
系统由前台门户系统作家后台管理系统平台后台管理系统和爬虫管理系统等多个子系统构成包括小说推荐作品检索小说排行小说阅读小说评论会员中心作家专区等功能支持自定义多模版可拓展的多种小说内容存储方式内置数据库分表存储和 系统由前台门户系统作家后台管理系统平台后台管理系统和爬虫管理系统等多个子系统构成包括小说推荐作品检索小说排行小说阅读小说评论会员中心作家专区等功能支持自定义多模版可拓展的多种小说内容存储方式内置数据库分表存储和
TXT 文本存储阅读主题切换多爬虫源自动采集和更新数据AI写作会员充值订阅模式新闻发布和实时统计报表 TXT 文本存储阅读主题切换多爬虫源自动采集和更新数据会员充值订阅模式新闻发布和实时统计报表
## 项目地址 ## 项目地址
@ -39,9 +39,8 @@ novel-plus -- 父工程
## 技术选型 ## 技术选型
| 技术 | 说明 | 技术 | 说明
|---------------------|--------------------- |---------------------| ---------------------------
| Spring Boot | Spring 应用快速开发脚手架 | Spring Boot | Spring 应用快速开发脚手架
| Spring AI | Spring 官方 AI 框架
| MyBatis | 持久层 ORM 框架 | MyBatis | 持久层 ORM 框架
| MyBatis Dynamic SQL | Mybatis 动态 sql | MyBatis Dynamic SQL | Mybatis 动态 sql
| PageHelper | MyBatis 分页插件 | PageHelper | MyBatis 分页插件
@ -70,42 +69,6 @@ novel-plus -- 父工程
https://www.bilibili.com/video/BV18e41197xs https://www.bilibili.com/video/BV18e41197xs
## AI 功能
novel-plus 5.x 已集成 Spring 官方最新发布的 Spring AI 框架并推出多项 AI 功能
1. v5.0.0 版本在小说章节发布页面的文本编辑器中集成了多项智能编辑功能包括 AI 扩写缩写续写及文本润色等这些功能的设计灵感来源于百家号文章编辑器中的 AI 助手
2. v5.1.0 版本在小说发布页面新增 AI 生成封面图功能若作家未上传自定义封面图系统将根据小说信息自动生成封面图
目前AI 功能仍处于实验阶段仅实现了基础的核心功能我们非常重视用户的实际使用体验和反馈未来将根据用户需求和使用情况持续优化和调整该功能如果用户反馈积极我们计划进一步开发更高级的
AI 功能例如自动生成有声小说智能情节推荐等以全面提升 novel-plus 的创作能力和用户体验
我们将持续关注 AI 技术的发展并致力于将其与小说创作场景深度融合为用户带来更智能更便捷的创作工具
由于 DeepSeek 官方 API 目前不可用novel-plus 项目默认使用的是第三方[硅基流动](https://cloud.siliconflow.cn/i/DOgMRH9S)
提供的 API采用的 AI 模型有对话模型`deepseek-ai/DeepSeek-R1-Distill-Llama-8B`DeepSeek-R1 的蒸馏版本免费使用和生图模型`Kwai-Kolors/Kolors`快手 Kolors 团队开发的文本到图像生成模型免费使用只需注册一个硅基流动账号创建一个
API 密钥并将其添加到 novel-plus 项目 novel-front 模块的 yaml 配置文件中即可体验 novel-plus 项目的 AI 写作功能
```yaml
spring:
ai:
openai:
image:
enabled: true
base-url: https://api.siliconflow.cn
api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
options:
model: Kwai-Kolors/Kolors
response_format: URL
api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
base-url: https://api.siliconflow.cn
chat:
options:
model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
```
> novel-plus 项目默认使用的都是免费 AI 模型生成效果有限如果对生成内容有更高的要求建议选用付费的 AI 模型
## 增值服务 ## 增值服务
👉 [了解详情](https://novel.xxyopen.com/service.htm) 👉 [了解详情](https://novel.xxyopen.com/service.htm)
@ -130,5 +93,3 @@ spring:
## 免责声明 ## 免责声明
本项目提供的爬虫工具仅用于采集项目初期的测试数据请勿用于商业盈利 用户使用本系统从事任何违法违规的事情一切后果由用户自行承担作者不承担任何责任 本项目提供的爬虫工具仅用于采集项目初期的测试数据请勿用于商业盈利 用户使用本系统从事任何违法违规的事情一切后果由用户自行承担作者不承担任何责任

View File

@ -1,53 +0,0 @@
mode:
# 单机模式
type: Standalone
# 元数据持久化
repository:
# 数据库持久化
type: JDBC
# 数据源配置
dataSources:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
ds_2:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://localhost:3306/information_schema?allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# 规则配置
rules:
- !SINGLE
tables:
- "*.*"
- !SHARDING
tables: # 数据分片规则配置
book_content:
# 分库策略缺省表示使用默认分库策略
actualDataNodes: ds_${1}.book_content${0..9}
# 分表策略
tableStrategy:
standard:
# 分片列名称
shardingColumn: index_id
# 分片算法名称
shardingAlgorithmName: bookContentSharding
shardingAlgorithms:
bookContentSharding:
# 行表达式分片算法使用 Groovy 的表达式提供对 SQL 语句中的 = IN 的分片操作支持
type: INLINE
props:
# 分片算法的行表达式
algorithm-expression: book_content${index_id % 10}
props:
# 是否在日志中打印 SQL
sql-show: true

View File

@ -1,44 +0,0 @@
INSERT INTO crawl_source (source_name, crawl_rule, source_status, create_time, update_time)
VALUES ('香书小说网', '{
"bookListUrl": "http://www.xbiqugu.la/fenlei/{catId}_{page}.html",
"catIdRule": {
"catId1": "1",
"catId2": "2",
"catId3": "3",
"catId4": "4",
"catId5": "6",
"catId6": "5"
},
"bookIdPatten": "<a\\\\s+href=\\"http://www.xbiqugu.la/(\\\\d+/\\\\d+)/\\"\\\\s+target=\\"_blank\\">",
"pagePatten": "<em\\\\s+id=\\"pagestats\\">(\\\\d+)/\\\\d+</em>",
"totalPagePatten": "<em\\\\s+id=\\"pagestats\\">\\\\d+/(\\\\d+)</em>",
"bookDetailUrl": "http://www.xbiqugu.la/{bookId}/",
"bookNamePatten": "<h1>([^/]+)</h1>",
"authorNamePatten": "者:([^/]+)</p>",
"picUrlPatten": "src=\\"(http://www.xbiqugu.la/files/article/image/\\\\d+/\\\\d+/\\\\d+s\\\\.jpg)\\"",
"bookStatusRule": {},
"descStart": "<div id=\\"intro\\">",
"descEnd": "</div>",
"upadateTimePatten": "<p>最后更新:(\\\\d+-\\\\d+-\\\\d+\\\\s\\\\d+:\\\\d+:\\\\d+)</p>",
"upadateTimeFormatPatten": "yyyy-MM-dd HH:mm:ss",
"bookIndexUrl": "http://www.xbiqugu.la/{bookId}/",
"indexIdPatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/(\\\\d+)\\\\.html''\\\\s+>[^/]+</a>",
"indexNamePatten": "<a\\\\s+href=''/\\\\d+/\\\\d+/\\\\d+\\\\.html''\\\\s+>([^/]+)</a>",
"bookContentUrl": "http://www.xbiqugu.la/{bookId}/{indexId}.html",
"contentStart": "<div id=\\"content\\">",
"contentEnd": "<p>",
"filterContent":"<div\\\\s+id=\\"content_tip\\">\\\\s*<b>([^/]+)</b>\\\\s*</div>"
}', 0, '2024-06-01 10:11:39', '2024-06-01 10:11:39');
update crawl_source
set crawl_rule = replace(crawl_rule, 'ibiquzw.org', 'biquxs.info')
where id = 16;
delete
from sys_menu
where menu_id = 104;
delete
from sys_menu
where menu_id = 57;

View File

@ -3140,17 +3140,5 @@ VALUES ('香书小说网', '{
update crawl_source update crawl_source
set crawl_rule = replace(crawl_rule, 'ibiquzw.org', 'biquxs.info') set crawl_rule = replace(crawl_rule, 'ibiquzw.org', 'ibiqugu.net')
where id = 16; where id = 16;
update crawl_source
set crawl_rule = replace(crawl_rule, 'xbiqugu.net', 'xbiqugu.la');
delete
from sys_menu
where menu_id = 104;
delete
from sys_menu
where menu_id = 57;

View File

@ -5,7 +5,7 @@
<groupId>com.java2nb</groupId> <groupId>com.java2nb</groupId>
<artifactId>novel-admin</artifactId> <artifactId>novel-admin</artifactId>
<version>5.1.3</version> <version>4.4.0</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>novel-admin</name> <name>novel-admin</name>
@ -14,14 +14,18 @@
<parent> <parent>
<groupId>org.springframework.boot</groupId> <groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId> <artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.18</version> <version>2.1.18.RELEASE</version>
<relativePath/> <relativePath/>
</parent> </parent>
<properties> <properties>
<java.version>21</java.version> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<velocity.version>1.7</velocity.version> <velocity.version>1.7</velocity.version>
<shardingsphere-jdbc.version>5.5.1</shardingsphere-jdbc.version> <activiti.version>5.22.0</activiti.version>
<sharding.jdbc.version>3.0.0</sharding.jdbc.version>
<jackson.version>2.15.1</jackson.version>
</properties> </properties>
<dependencies> <dependencies>
@ -56,24 +60,35 @@
<groupId>net.sourceforge.nekohtml</groupId> <groupId>net.sourceforge.nekohtml</groupId>
<artifactId>nekohtml</artifactId> <artifactId>nekohtml</artifactId>
</dependency> </dependency>
<!-- 请求参数校验相关 -->
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<!--mybatis --> <!--mybatis -->
<dependency> <dependency>
<groupId>mysql</groupId> <groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId> <artifactId>mysql-connector-java</artifactId>
<version>8.0.29</version> <version>8.0.29</version>
</dependency> </dependency>
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>3.5.6</version>
</dependency>
<dependency> <dependency>
<groupId>org.mybatis.spring.boot</groupId> <groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId> <artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.1.1</version> <version>1.1.1</version>
</dependency> </dependency>
<!--druid -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.2.9</version>
</dependency>
<!--commons --> <!--commons -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
</dependency>
<dependency> <dependency>
<groupId>commons-configuration</groupId> <groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId> <artifactId>commons-configuration</artifactId>
@ -123,12 +138,6 @@
<groupId>org.apache.velocity</groupId> <groupId>org.apache.velocity</groupId>
<artifactId>velocity</artifactId> <artifactId>velocity</artifactId>
<version>1.7</version> <version>1.7</version>
<exclusions>
<exclusion>
<artifactId>commons-lang</artifactId>
<groupId>commons-lang</groupId>
</exclusion>
</exclusions>
</dependency> </dependency>
<!--<dependency>--> <!--<dependency>-->
<!--<groupId>org.springframework.boot</groupId>--> <!--<groupId>org.springframework.boot</groupId>-->
@ -157,12 +166,6 @@
<groupId>io.springfox</groupId> <groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId> <artifactId>springfox-swagger2</artifactId>
<version>2.6.1</version> <version>2.6.1</version>
<exclusions>
<exclusion>
<artifactId>guava</artifactId>
<groupId>com.google.guava</groupId>
</exclusion>
</exclusions>
</dependency> </dependency>
<dependency> <dependency>
<groupId>io.springfox</groupId> <groupId>io.springfox</groupId>
@ -192,29 +195,24 @@
<groupId>org.apache.commons</groupId> <groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId> <artifactId>commons-text</artifactId>
<version>1.4</version> <version>1.4</version>
<exclusions>
<exclusion>
<artifactId>commons-lang3</artifactId>
<groupId>org.apache.commons</groupId>
</exclusion>
</exclusions>
</dependency> </dependency>
<!-- ShardingSphere-JDBC -->
<dependency> <dependency>
<groupId>org.apache.shardingsphere</groupId> <groupId>io.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId> <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${shardingsphere-jdbc.version}</version> <version>${sharding.jdbc.version}</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>org.yaml</groupId> <groupId>io.shardingsphere</groupId>
<artifactId>snakeyaml</artifactId> <artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>2.2</version> <version>${sharding.jdbc.version}</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>com.h2database</groupId> <groupId>com.fasterxml.jackson.core</groupId>
<artifactId>h2</artifactId> <artifactId>jackson-databind</artifactId>
<scope>runtime</scope> <version>${jackson.version}</version>
</dependency> </dependency>
<!--war包部署需要--> <!--war包部署需要-->
@ -332,35 +330,4 @@
</snapshots> </snapshots>
</pluginRepository> </pluginRepository>
</pluginRepositories> </pluginRepositories>
<profiles>
<profile>
<!-- 定义一个用于切换到中央仓库的profile -->
<id>central-repo</id>
<repositories>
<repository>
<id>central</id>
<url>https://repo.maven.apache.org/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central-plugin</id>
<url>https://repo.maven.apache.org/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
</project> </project>

View File

@ -0,0 +1,20 @@
#端口号
server:
port: 8088
spring:
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
redis:
host: 127.0.0.1
port: 6379
password: test123456
sharding:
jdbc:
datasource:
ds0:
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456

View File

@ -1,9 +0,0 @@
#端口号
server:
port: 8088
spring:
redis:
host: 127.0.0.1
port: 6379
password: test123456

View File

@ -1,47 +0,0 @@
mode:
# 单机模式
type: Standalone
# 元数据持久化
repository:
# 数据库持久化
type: JDBC
# 数据源配置
dataSources:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# 规则配置
rules:
- !SINGLE
tables:
- "*.*"
- !SHARDING
tables: # 数据分片规则配置
book_content:
# 分库策略缺省表示使用默认分库策略
actualDataNodes: ds_${1}.book_content${0..9}
# 分表策略
tableStrategy:
standard:
# 分片列名称
shardingColumn: index_id
# 分片算法名称
shardingAlgorithmName: bookContentSharding
shardingAlgorithms:
bookContentSharding:
# 行表达式分片算法使用 Groovy 的表达式提供对 SQL 语句中的 = IN 的分片操作支持
type: INLINE
props:
# 分片算法的行表达式
algorithm-expression: book_content${index_id % 10}
props:
# 是否在日志中打印 SQL
sql-show: true

View File

@ -11,9 +11,7 @@ import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.transaction.annotation.EnableTransactionManagement; import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.sql.DataSource;
import java.net.InetAddress; import java.net.InetAddress;
import java.sql.Connection;
@EnableTransactionManagement @EnableTransactionManagement
@ -25,23 +23,15 @@ import java.sql.Connection;
@EnableCaching @EnableCaching
@Slf4j @Slf4j
public class AdminApplication { public class AdminApplication {
public static void main(String[] args) { public static void main(String[] args) {
SpringApplication.run(AdminApplication.class, args); SpringApplication.run(AdminApplication.class, args);
} }
@Bean @Bean
public CommandLineRunner commandLineRunner(ApplicationContext ctx, DataSource dataSource) { public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
return args -> { return args -> {
log.info("创建连接池..."); log.info("项目启动啦,访问路径:{}", "http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment().getProperty("server.port"));
try (Connection connection = dataSource.getConnection()) {
log.info("连接池已创建.");
log.info("数据库:{}", connection.getMetaData().getDatabaseProductName());
log.info("数据库版本:{}", connection.getMetaData().getDatabaseProductVersion());
}
log.info("项目启动啦,访问路径:{}",
"http://" + InetAddress.getLocalHost().getHostAddress() + ":" + ctx.getEnvironment()
.getProperty("server.port"));
}; };
} }
} }

View File

@ -1,16 +1,19 @@
package com.java2nb.common.aspect; package com.java2nb.common.aspect;
import com.java2nb.common.utils.IPUtils; import com.java2nb.common.utils.HttpContextUtils;
import org.aspectj.lang.JoinPoint; import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.*; import org.aspectj.lang.annotation.*;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component; import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder; import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes; import org.springframework.web.context.request.ServletRequestAttributes;
import sun.net.util.IPAddressUtil;
import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequest;
import java.lang.reflect.Method;
import java.util.Arrays; import java.util.Arrays;
@Aspect @Aspect
@ -34,10 +37,11 @@ public class WebLogAspect {
logger.info("请求地址 : " + request.getRequestURL().toString()); logger.info("请求地址 : " + request.getRequestURL().toString());
logger.info("HTTP METHOD : " + request.getMethod()); logger.info("HTTP METHOD : " + request.getMethod());
// 获取真实的ip地址 // 获取真实的ip地址
logger.info("IP : " + IPUtils.getIpAddr(request)); //logger.info("IP : " + IPAddressUtil.getClientIpAddress(request));
logger.info("CLASS_METHOD : " + joinPoint.getSignature().getDeclaringTypeName() + "." logger.info("CLASS_METHOD : " + joinPoint.getSignature().getDeclaringTypeName() + "."
+ joinPoint.getSignature().getName()); + joinPoint.getSignature().getName());
logger.info("参数 : " + Arrays.toString(joinPoint.getArgs())); logger.info("参数 : " + Arrays.toString(joinPoint.getArgs()));
// loggger.info("参数 : " + joinPoint.getArgs());
} }

View File

@ -0,0 +1,132 @@
package com.java2nb.common.config;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.support.http.StatViewServlet;
import com.alibaba.druid.support.http.WebStatFilter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import javax.sql.DataSource;
import java.sql.SQLException;
/**
* Created by PrimaryKey on 17/2/4.
*/
@SuppressWarnings("AlibabaRemoveCommentedCode")
@Configuration
public class DruidDBConfig {
private Logger logger = LoggerFactory.getLogger(DruidDBConfig.class);
@Value("${spring.datasource.url}")
private String dbUrl;
@Value("${spring.datasource.username}")
private String username;
@Value("${spring.datasource.password}")
private String password;
@Value("${spring.datasource.driverClassName}")
private String driverClassName;
@Value("${spring.datasource.initialSize}")
private int initialSize;
@Value("${spring.datasource.minIdle}")
private int minIdle;
@Value("${spring.datasource.maxActive}")
private int maxActive;
@Value("${spring.datasource.maxWait}")
private int maxWait;
@Value("${spring.datasource.timeBetweenEvictionRunsMillis}")
private int timeBetweenEvictionRunsMillis;
@Value("${spring.datasource.minEvictableIdleTimeMillis}")
private int minEvictableIdleTimeMillis;
@Value("${spring.datasource.validationQuery}")
private String validationQuery;
@Value("${spring.datasource.testWhileIdle}")
private boolean testWhileIdle;
@Value("${spring.datasource.testOnBorrow}")
private boolean testOnBorrow;
@Value("${spring.datasource.testOnReturn}")
private boolean testOnReturn;
@Value("${spring.datasource.poolPreparedStatements}")
private boolean poolPreparedStatements;
@Value("${spring.datasource.maxPoolPreparedStatementPerConnectionSize}")
private int maxPoolPreparedStatementPerConnectionSize;
@Value("${spring.datasource.filters}")
private String filters;
@Value("{spring.datasource.connectionProperties}")
private String connectionProperties;
@Bean(initMethod = "init", destroyMethod = "close") //声明其为Bean实例
@Primary //在同样的DataSource中首先使用被标注的DataSource
public DataSource dataSource() {
DruidDataSource datasource = new DruidDataSource();
datasource.setUrl(this.dbUrl);
datasource.setUsername(username);
datasource.setPassword(password);
datasource.setDriverClassName(driverClassName);
//configuration
datasource.setInitialSize(initialSize);
datasource.setMinIdle(minIdle);
datasource.setMaxActive(maxActive);
datasource.setMaxWait(maxWait);
datasource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
datasource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
datasource.setValidationQuery(validationQuery);
datasource.setTestWhileIdle(testWhileIdle);
datasource.setTestOnBorrow(testOnBorrow);
datasource.setTestOnReturn(testOnReturn);
datasource.setPoolPreparedStatements(poolPreparedStatements);
datasource.setMaxPoolPreparedStatementPerConnectionSize(maxPoolPreparedStatementPerConnectionSize);
try {
datasource.setFilters(filters);
} catch (SQLException e) {
logger.error("druid configuration initialization filter", e);
}
datasource.setConnectionProperties(connectionProperties);
return datasource;
}
@Bean
public ServletRegistrationBean druidServlet() {
ServletRegistrationBean reg = new ServletRegistrationBean();
reg.setServlet(new StatViewServlet());
reg.addUrlMappings("/druid/*");
reg.addInitParameter("allow", ""); //白名单
return reg;
}
@Bean public FilterRegistrationBean filterRegistrationBean() {
FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean();
filterRegistrationBean.setFilter(new WebStatFilter());
filterRegistrationBean.addUrlPatterns("/*");
filterRegistrationBean.addInitParameter("exclusions", "*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*");
filterRegistrationBean.addInitParameter("profileEnable", "true");
filterRegistrationBean.addInitParameter("principalCookieName","USER_COOKIE");
filterRegistrationBean.addInitParameter("principalSessionName","USER_SESSION");
filterRegistrationBean.addInitParameter("DruidWebStatFilter","/*");
return filterRegistrationBean;
}
}

View File

@ -0,0 +1,48 @@
package com.java2nb.common.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.service.Contact;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;
/**
* ${DESCRIPTION}
*
* @author xiongxy
* @create 2019-11-02 23:53
*/
@EnableSwagger2
@Configuration
public class Swagger2Config {
@Bean
public Docket createRestApi() {
return new Docket(DocumentationType.SWAGGER_2)
.apiInfo(apiInfo())
.select()
//为当前包路径
.apis(RequestHandlerSelectors.any())
.paths(PathSelectors.any())
.build();
}
//构建 api文档的详细信息函数
private ApiInfo apiInfo() {
return new ApiInfoBuilder()
//页面标题
.title("功能测试")
//创建人
.contact(new Contact("xiongxy", "1179705413@qq.com", "1179705413@qq.com"))
//版本号
.version("1.0")
//描述
.description("API 描述")
.build();
}
}

View File

@ -9,25 +9,25 @@ import java.util.Map;
public interface GeneratorMapper { public interface GeneratorMapper {
@Select( @Select(
"select table_name tableName, engine, table_comment tableComment, create_time createTime from tables" "select table_name tableName, engine, table_comment tableComment, create_time createTime from information_schema.tables"
+ " where table_schema = 'novel_plus' and table_name like concat('%',#{tableName},'%')") + " where table_schema = 'novel_plus' and table_name like concat('%',#{tableName},'%')")
List<Map<String, Object>> list(@Param("tableName") String tableName); List<Map<String, Object>> list(@Param("tableName") String tableName);
@Select("select count(*) from tables where table_schema = 'novel_plus'") @Select("select count(*) from information_schema.tables where table_schema = 'novel_plus'")
int count(Map<String, Object> map); int count(Map<String, Object> map);
@Select( @Select(
"select table_name tableName, engine, table_comment tableComment, create_time createTime from tables \r\n" "select table_name tableName, engine, table_comment tableComment, create_time createTime from information_schema.tables \r\n"
+ " where table_schema = 'novel_plus' and table_name = #{tableName}") + " where table_schema = 'novel_plus' and table_name = #{tableName}")
Map<String, String> get(String tableName); Map<String, String> get(String tableName);
@Select( @Select(
"select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from columns\r\n" "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from information_schema.columns\r\n"
+ " where table_name = #{tableName} and table_schema = 'novel_plus' order by ordinal_position") + " where table_name = #{tableName} and table_schema = 'novel_plus' order by ordinal_position")
List<Map<String, String>> listColumns(String tableName); List<Map<String, String>> listColumns(String tableName);
@Select( @Select(
"select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from columns\r\n" "select column_name columnName, data_type dataType, column_comment columnComment, column_key columnKey, extra from information_schema.columns\r\n"
+ " where table_name = #{tableName} and table_schema = 'novel_plus' and column_key = 'PRI' limit 1") + " where table_name = #{tableName} and table_schema = 'novel_plus' and column_key = 'PRI' limit 1")
Map<String, String> getPriColumn(String tableName); Map<String, String> getPriColumn(String tableName);
} }

View File

@ -2,19 +2,24 @@ package com.java2nb.common.exception;
import com.java2nb.common.utils.R; import com.java2nb.common.utils.R;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.servlet.error.ErrorAttributes; import org.springframework.boot.web.servlet.error.ErrorAttributes;
import org.springframework.boot.web.servlet.error.ErrorController; import org.springframework.boot.web.servlet.error.ErrorController;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController; import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.ModelAndView; import org.springframework.web.servlet.ModelAndView;
import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse; import javax.servlet.http.HttpServletResponse;
import java.util.Map;
@RestController @RestController
public class MainsiteErrorController implements ErrorController { public class MainsiteErrorController implements ErrorController {
private Logger logger = LoggerFactory.getLogger(getClass());
private static final String ERROR_PATH = "/error"; private static final String ERROR_PATH = "/error";
@Autowired @Autowired
@ -53,4 +58,9 @@ public class MainsiteErrorController implements ErrorController {
} }
} }
@Override
public String getErrorPath() {
// TODO Auto-generated method stub
return ERROR_PATH;
}
} }

View File

@ -1,5 +1,6 @@
package com.java2nb.common.utils; package com.java2nb.common.utils;
import com.alibaba.druid.util.StringUtils;
import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject; import com.alibaba.fastjson.JSONObject;
@ -7,7 +8,6 @@ import java.util.HashMap;
import java.util.Map; import java.util.Map;
public class JSONUtils { public class JSONUtils {
/** /**
* Bean对象转JSON * Bean对象转JSON
* *

View File

@ -7,11 +7,92 @@ logging:
root: info root: info
com.java2nb: debug com.java2nb: debug
spring: spring:
datasource:
type: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
username: root
password: test123456
#password:
initialSize: 1
minIdle: 3
maxActive: 20
# 配置获取连接等待超时的时间
maxWait: 60000
# 配置间隔多久才进行一次检测检测需要关闭的空闲连接单位是毫秒
timeBetweenEvictionRunsMillis: 60000
# 配置一个连接在池中最小生存的时间单位是毫秒
minEvictableIdleTimeMillis: 30000
validationQuery: select 'x'
testWhileIdle: true
testOnBorrow: false
testOnReturn: false
# 打开PSCache并且指定每个连接上PSCache的大小
poolPreparedStatements: true
maxPoolPreparedStatementPerConnectionSize: 20
# 配置监控统计拦截的filters去掉后监控界面sql无法统计'wall'用于防火墙
filters: stat,slf4j
# 通过connectProperties属性来打开mergeSql功能慢SQL记录
connectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
# 合并多个DruidDataSource的监控数据
#useGlobalDataSourceStat: true
redis: redis:
host: 127.0.0.1 host: 127.0.0.1
port: 6379 port: 6379
password: test123456 password: test123456
# 连接超时时间毫秒 # 连接超时时间毫秒
timeout: 10000 timeout: 10000
jedis:
pool:
# 连接池中的最大空闲连接
max-idle: 8
# 连接池中的最小空闲连接
min-idle: 10
# 连接池最大连接数使用负值表示没有限制
max-active: 100
# 连接池最大阻塞等待时间使用负值表示没有限制
max-wait: -1
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
names: ds0,ds1
ds0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
ds1:
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/information_schema?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
config:
sharding:
props:
sql.show: true
tables:
book_content: #book_content表
key-generator-column-name: id #主键
actual-data-nodes: ds${0}.book_content${0..9} #数据节点
# database-strategy: #分库策略
# inline:
# sharding-column: book_id
# algorithm-expression: ds${book_id % 10}
table-strategy: #分表策略
inline:
shardingColumn: index_id
algorithm-expression: book_content${index_id % 10}
tables:
actual-data-nodes: ds${1}.tables
columns:
actual-data-nodes: ds${1}.columns
default-data-source-name: ds0

View File

@ -7,11 +7,86 @@ logging:
root: error root: error
com.java2nb: error com.java2nb: error
spring: spring:
datasource:
type: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
username: root
password: test123456
#password:
initialSize: 1
minIdle: 3
maxActive: 20
# 配置获取连接等待超时的时间
maxWait: 60000
# 配置间隔多久才进行一次检测检测需要关闭的空闲连接单位是毫秒
timeBetweenEvictionRunsMillis: 60000
# 配置一个连接在池中最小生存的时间单位是毫秒
minEvictableIdleTimeMillis: 30000
validationQuery: select 'x'
testWhileIdle: true
testOnBorrow: false
testOnReturn: false
# 打开PSCache并且指定每个连接上PSCache的大小
poolPreparedStatements: true
maxPoolPreparedStatementPerConnectionSize: 20
# 配置监控统计拦截的filters去掉后监控界面sql无法统计'wall'用于防火墙
filters: stat,slf4j
# 通过connectProperties属性来打开mergeSql功能慢SQL记录
connectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
# 合并多个DruidDataSource的监控数据
#useGlobalDataSourceStat: true
redis: redis:
host: 127.0.0.1 host: 127.0.0.1
port: 6379 port: 6379
password: test123456 password: test
# 连接超时时间毫秒 # 连接超时时间毫秒
timeout: 10000 timeout: 10000
jedis:
pool:
# 连接池中的最大空闲连接
max-idle: 8
# 连接池中的最小空闲连接
min-idle: 10
# 连接池最大连接数使用负值表示没有限制
max-active: 100
# 连接池最大阻塞等待时间使用负值表示没有限制
max-wait: -1
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
names: ds0 #,ds1
ds0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# ds1:
# type: com.alibaba.druid.pool.DruidDataSource
# driver-class-name: com.mysql.jdbc.Driver
# url: jdbc:mysql://localhost:3306/novel_plus2
# username: root
# password: test123456
config:
sharding:
props:
sql.show: true
tables:
book_content: #book_content表
key-generator-column-name: id #主键
actual-data-nodes: ds${0}.book_content${0..9} #数据节点
# database-strategy: #分库策略
# inline:
# sharding-column: book_id
# algorithm-expression: ds${book_id % 10}
table-strategy: #分表策略
inline:
shardingColumn: index_id
algorithm-expression: book_content${index_id % 10}

View File

@ -9,9 +9,6 @@ server:
# basic: # basic:
# enabled: false # enabled: false
spring: spring:
datasource:
driverClassName: org.apache.shardingsphere.driver.ShardingSphereDriver
url: jdbc:shardingsphere:absolutepath:${user.dir}/config/shardingsphere-jdbc.yml
thymeleaf: thymeleaf:
mode: LEGACYHTML5 mode: LEGACYHTML5
cache: false cache: false
@ -27,6 +24,9 @@ spring:
max-file-size: 100MB max-file-size: 100MB
max-request-size: 100MB max-request-size: 100MB
devtools:
restart:
enabled: true
main: main:
allow-bean-definition-overriding: true allow-bean-definition-overriding: true
@ -36,6 +36,9 @@ mybatis:
map-underscore-to-camel-case: true map-underscore-to-camel-case: true
mapper-locations: mybatis/**/*Mapper.xml mapper-locations: mybatis/**/*Mapper.xml
typeAliasesPackage: com.java2nb.**.domain typeAliasesPackage: com.java2nb.**.domain
#[弃用]配置缓存和session存储方式默认ehcache,可选redis,[弃用]调整至 spring cache typeshiro.用户权限sessionspring.cache通用
#[弃用]cacheType: ehcache
logging: logging:
config: classpath:logback-boot.xml config: classpath:logback-boot.xml

View File

Before

Width:  |  Height:  |  Size: 162 KiB

After

Width:  |  Height:  |  Size: 162 KiB

View File

@ -79,7 +79,7 @@ function update() {
}, },
success: function (data) { success: function (data) {
if (data.code == 0) { if (data.code == 0) {
layer.msg("操作成功,重启 novel-front 后生效"); layer.msg("操作成功");
} else { } else {
layer.alert(data.msg) layer.alert(data.msg)
} }

View File

@ -6,7 +6,7 @@
<title>小说精品屋 - 文件管理器</title> <title>小说精品屋 - 文件管理器</title>
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">

View File

@ -6,7 +6,7 @@
<title>403 页面</title> <title>403 页面</title>
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">

View File

@ -10,7 +10,7 @@
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">

View File

@ -11,7 +11,7 @@
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">

View File

@ -6,7 +6,7 @@
<title>500错误</title> <title>500错误</title>
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link rel="shortcut icon" href="favicon.ico"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">
<link href="/css/style.css?v=4.1.0" rel="stylesheet"> <link href="/css/style.css?v=4.1.0" rel="stylesheet">

View File

@ -4,7 +4,7 @@
<title></title> <title></title>
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link href="css/bootstrap.min.css?v=3.3.6" <link href="css/bootstrap.min.css?v=3.3.6"
th:href="@{/css/bootstrap.min.css?v=3.3.6}" rel="stylesheet"> th:href="@{/css/bootstrap.min.css?v=3.3.6}" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" <link href="/css/font-awesome.css?v=4.4.0"

View File

@ -10,7 +10,7 @@
<!--[if lt IE 9]> <!--[if lt IE 9]>
<meta http-equiv="refresh" content="0;ie.html"/> <meta http-equiv="refresh" content="0;ie.html"/>
<![endif]--> <![endif]-->
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.min.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.min.css?v=4.4.0" rel="stylesheet">
<link href="/css/plugins/toastr/toastr.min.css" rel="stylesheet"> <link href="/css/plugins/toastr/toastr.min.css" rel="stylesheet">

View File

@ -11,7 +11,7 @@
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet"> <link href="/css/bootstrap.min.css?v=3.3.6" rel="stylesheet">
<link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet"> <link href="/css/font-awesome.css?v=4.4.0" rel="stylesheet">
<link href="/css/animate.css" rel="stylesheet"> <link href="/css/animate.css" rel="stylesheet">

View File

@ -4,7 +4,7 @@
<title></title> <title></title>
<meta name="keywords" content=""> <meta name="keywords" content="">
<meta name="description" content=""> <meta name="description" content="">
<link rel="shortcut icon" href="/favicon.ico"> <link rel="shortcut icon" href="favicon.ico">
<link rel="stylesheet" href="/css/bootstrap.min.css" /> <link rel="stylesheet" href="/css/bootstrap.min.css" />
<link rel="stylesheet" href="/css/animate.css" /> <link rel="stylesheet" href="/css/animate.css" />
<link rel="stylesheet" href="/css/font-awesome.css" /> <link rel="stylesheet" href="/css/font-awesome.css" />

View File

@ -5,7 +5,7 @@
<parent> <parent>
<artifactId>novel</artifactId> <artifactId>novel</artifactId>
<groupId>com.java2nb</groupId> <groupId>com.java2nb</groupId>
<version>5.1.3</version> <version>4.4.0</version>
</parent> </parent>
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
@ -52,16 +52,18 @@
</dependency> </dependency>
<!-- 分库分表--> <!-- 分库分表-->
<!-- ShardingSphere-JDBC --> <!-- sharding jdbc依赖 -->
<dependency> <dependency>
<groupId>org.apache.shardingsphere</groupId> <groupId>io.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId> <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${shardingsphere-jdbc.version}</version> <version>${sharding.jdbc.version}</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>com.h2database</groupId> <groupId>io.shardingsphere</groupId>
<artifactId>h2</artifactId> <artifactId>sharding-jdbc-spring-namespace</artifactId>
<scope>runtime</scope> <version>${sharding.jdbc.version}</version>
</dependency> </dependency>
@ -71,6 +73,11 @@
<artifactId>pagehelper-spring-boot-starter</artifactId> <artifactId>pagehelper-spring-boot-starter</artifactId>
<version>${pagehelper.version}</version> <version>${pagehelper.version}</version>
</dependency> </dependency>
<dependency>
<groupId>com.cuisongliu</groupId>
<artifactId>orderbyhelper-spring-boot-starter</artifactId>
<version>${orderbyhelper.version}</version>
</dependency>
<dependency> <dependency>
<groupId>org.apache.commons</groupId> <groupId>org.apache.commons</groupId>
@ -79,8 +86,15 @@
</dependency> </dependency>
<!--httpclient--> <!--httpclient-->
<dependency> <dependency>
<groupId>org.apache.httpcomponents.client5</groupId> <groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient5</artifactId> <artifactId>httpclient</artifactId>
<version>4.5.14</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency> </dependency>
<dependency> <dependency>
@ -103,12 +117,6 @@
<optional>true</optional> <optional>true</optional>
</dependency> </dependency>
<!-- 请求参数校验相关 -->
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<dependency> <dependency>
<groupId>io.github.xxyopen</groupId> <groupId>io.github.xxyopen</groupId>
<artifactId>xxy-model</artifactId> <artifactId>xxy-model</artifactId>
@ -130,29 +138,5 @@
</dependencies> </dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<zip destfile='${project.build.directory}/build/sql.zip'>
<zipfileset filemode="755" dir='${basedir}/../doc/sql'/>
</zip>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project> </project>

View File

@ -1,55 +0,0 @@
package com.java2nb.novel.core.advice;
import io.github.xxyopen.model.resp.RestResult;
import io.github.xxyopen.model.resp.SysResultCode;
import io.github.xxyopen.web.exception.BusinessException;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.MediaType;
import org.springframework.validation.BindException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.servlet.ModelAndView;
/**
* 统一异常处理器
*
* @author xiongxiaoyang
*/
@Slf4j
@RestControllerAdvice
public class CommonExceptionHandler {
public CommonExceptionHandler() {
}
@ExceptionHandler({BindException.class})
public RestResult<Void> handlerBindException(BindException e) {
log.error(e.getMessage(), e);
return RestResult.fail(SysResultCode.PARAM_ERROR);
}
@ExceptionHandler({BusinessException.class})
public RestResult<Void> handlerBusinessException(BusinessException e) {
log.error(e.getMessage(), e);
return RestResult.fail(e.getResultCode());
}
@ExceptionHandler(Exception.class)
public Object handleException(HttpServletRequest request, Exception e) {
log.error(e.getMessage(), e);
if (isJsonRequest(request)) {
// 如果是REST请求返回JSON格式的错误响应
return RestResult.error();
} else {
//跳转页面过程中出现异常时统一跳转到404页面
return new ModelAndView("404");
}
}
private boolean isJsonRequest(HttpServletRequest request) {
String acceptHeader = request.getHeader("Accept");
return acceptHeader != null && acceptHeader.contains(MediaType.APPLICATION_JSON_VALUE);
}
}

View File

@ -1,53 +0,0 @@
package com.java2nb.novel.core.advice;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.ser.std.ToStringSerializer;
import org.springframework.core.MethodParameter;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;
import org.springframework.http.server.ServerHttpRequest;
import org.springframework.http.server.ServerHttpResponse;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.servlet.mvc.method.annotation.ResponseBodyAdvice;
import java.util.Objects;
/**
* 在对 RestController 返回对象 json 序列化时将所有 Long 类型转为 String 类型返回避免前端数据精度丢失的问题
* 取代 spring.jackson.generator.write-numbers-as-strings=true 配置避免影响全局的 ObjectMapper
*
* @author xiongxiaoyang
* */
@RestControllerAdvice
public class CustomResponseBodyAdvice implements ResponseBodyAdvice<Object> {
private final ObjectMapper customObjectMapper;
public CustomResponseBodyAdvice(Jackson2ObjectMapperBuilder builder) {
customObjectMapper = builder.createXmlMapper(false).build();
SimpleModule simpleModule = new SimpleModule();
simpleModule.addSerializer(Long.class, ToStringSerializer.instance);
simpleModule.addSerializer(Long.TYPE, ToStringSerializer.instance);
customObjectMapper.registerModule(simpleModule);
}
@Override
public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
// 返回 true 表示对所有 Controller 的响应都生效
return true;
}
@Override
public Object beforeBodyWrite(Object body, MethodParameter returnType, MediaType selectedContentType, Class<? extends HttpMessageConverter<?>> selectedConverterType, ServerHttpRequest request, ServerHttpResponse response) {
// 使用自定义的 ObjectMapper 序列化响应体
if(Objects.nonNull(body)) {
return customObjectMapper.valueToTree(body);
}else{
return null;
}
}
}

View File

@ -0,0 +1,26 @@
package com.java2nb.novel.core.advice;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
/**
* 页面异常处理器
*
* @author 11797
*/
@Slf4j
@ControllerAdvice(basePackages = "com.java2nb.novel.controller.page")
public class PageExceptionHandler {
/**
* 处理所有异常
*/
@ExceptionHandler(Exception.class)
public String handlerException(Exception e) {
log.error(e.getMessage(), e);
//跳转页面过程中出现异常时统一跳转到404页面
return "404";
}
}

View File

@ -41,6 +41,11 @@ public interface CacheKey {
* */ * */
String TEMPLATE_DIR_KEY = "templateDirKey";; String TEMPLATE_DIR_KEY = "templateDirKey";;
/**
* 正在运行的爬虫线程存储KEY前缀
* */
String RUNNING_CRAWL_THREAD_KEY_PREFIX = "runningCrawlTreadDataKeyPrefix";
/** /**
* 上一次搜索引擎更新的时间 * 上一次搜索引擎更新的时间
* */ * */
@ -64,8 +69,4 @@ public interface CacheKey {
* 测试爬虫规则缓存 * 测试爬虫规则缓存
*/ */
String BOOK_TEST_PARSE = "testParse"; String BOOK_TEST_PARSE = "testParse";
/**
* AI生成图片
* */
String AI_GEN_PIC = "aiGenPic";
} }

View File

@ -19,8 +19,4 @@ public class HttpProxyProperties {
private Integer port; private Integer port;
private String username;
private String password;
} }

View File

@ -5,7 +5,7 @@ import lombok.SneakyThrows;
import lombok.experimental.UtilityClass; import lombok.experimental.UtilityClass;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.Charsets; import org.apache.commons.codec.Charsets;
import org.apache.hc.client5.http.utils.DateUtils; import org.apache.http.client.utils.DateUtils;
import org.springframework.core.io.Resource; import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity; import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders; import org.springframework.http.HttpHeaders;
@ -18,11 +18,6 @@ import java.io.File;
import java.io.FileOutputStream; import java.io.FileOutputStream;
import java.io.InputStream; import java.io.InputStream;
import java.io.OutputStream; import java.io.OutputStream;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.Date; import java.util.Date;
import java.util.Objects; import java.util.Objects;
@ -130,23 +125,5 @@ public class FileUtil {
} }
/**
* 下载文件
*
* @param downloadUrl 下载的URL
* @param savePath 保存的路径
*/
@SneakyThrows
public void downloadFile(String downloadUrl, String savePath) {
Path path = Paths.get(savePath);
Path parentPath = path.getParent();
if (Files.notExists(parentPath)) {
Files.createDirectories(parentPath);
}
URL url = new URL(downloadUrl);
try (InputStream in = url.openStream()) {
Files.copy(in, path, StandardCopyOption.REPLACE_EXISTING);
}
}
} }

View File

@ -4,36 +4,22 @@ import lombok.extern.slf4j.Slf4j;
import org.springframework.http.*; import org.springframework.http.*;
import org.springframework.web.client.RestTemplate; import org.springframework.web.client.RestTemplate;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Map;
/** /**
* @author Administrator * @author Administrator
*/ */
@Slf4j @Slf4j
public class HttpUtil { public class HttpUtil {
private static final String DEFAULT_CHARSET = "utf-8"; private static final RestTemplate REST_TEMPLATE = RestTemplates.newInstance("utf-8");
private static final Map<String, RestTemplate> REST_TEMPLATE_MAP = new HashMap<>(); public static String getByHttpClientWithChrome(String url) {
public static String getByHttpClientWithChrome(String url, String charset) {
log.debug("Get url{}", url);
if (!Charset.isSupported(charset)) {
log.error("字符编码{}无效!", charset);
return null;
}
RestTemplate restTemplate = REST_TEMPLATE_MAP.computeIfAbsent(charset,
k -> RestTemplates.newInstance(charset));
try { try {
HttpHeaders headers = new HttpHeaders(); HttpHeaders headers = new HttpHeaders();
headers.add("user-agent", headers.add("user-agent",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36"); "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36");
HttpEntity<String> requestEntity = new HttpEntity<>(null, headers); HttpEntity<String> requestEntity = new HttpEntity<>(null, headers);
ResponseEntity<String> forEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity, ResponseEntity<String> forEntity = REST_TEMPLATE.exchange(url, HttpMethod.GET, requestEntity, String.class);
String.class);
log.debug("Response code{}", forEntity.getStatusCode());
if (forEntity.getStatusCode() == HttpStatus.OK) { if (forEntity.getStatusCode() == HttpStatus.OK) {
return forEntity.getBody(); return forEntity.getBody();
} else { } else {
@ -45,8 +31,4 @@ public class HttpUtil {
} }
} }
public static String getByHttpClientWithChrome(String url) {
return getByHttpClientWithChrome(url, DEFAULT_CHARSET);
}
} }

View File

@ -1,6 +1,6 @@
package com.java2nb.novel.core.utils; package com.java2nb.novel.core.utils;
import jakarta.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequest;
public class IpUtil { public class IpUtil {

View File

@ -2,22 +2,17 @@ package com.java2nb.novel.core.utils;
import com.java2nb.novel.core.config.HttpProxyProperties; import com.java2nb.novel.core.config.HttpProxyProperties;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import org.apache.commons.lang3.StringUtils; import org.apache.http.HttpHost;
import org.apache.hc.client5.http.auth.AuthScope; import org.apache.http.config.Registry;
import org.apache.hc.client5.http.auth.UsernamePasswordCredentials; import org.apache.http.config.RegistryBuilder;
import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider; import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient; import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.hc.client5.http.impl.classic.HttpClientBuilder; import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.hc.client5.http.impl.classic.HttpClients; import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager; import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.hc.client5.http.socket.ConnectionSocketFactory; import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.hc.client5.http.socket.PlainConnectionSocketFactory; import org.apache.http.impl.client.HttpClients;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory; import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.hc.core5.http.HttpHost;
import org.apache.hc.core5.http.config.Registry;
import org.apache.hc.core5.http.config.RegistryBuilder;
import org.apache.hc.core5.ssl.SSLContexts;
import org.apache.hc.core5.ssl.TrustStrategy;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory; import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.http.converter.HttpMessageConverter; import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.StringHttpMessageConverter; import org.springframework.http.converter.StringHttpMessageConverter;
@ -45,7 +40,7 @@ public class RestTemplates {
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true; TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
//忽略证书 //忽略证书
SSLContext sslContext = SSLContexts.custom() SSLContext sslContext = org.apache.http.ssl.SSLContexts.custom()
.loadTrustMaterial(null, acceptingTrustStrategy) .loadTrustMaterial(null, acceptingTrustStrategy)
.build(); .build();
@ -66,15 +61,6 @@ public class RestTemplates {
if (Objects.nonNull(httpProxyProperties) && Boolean.TRUE.equals(httpProxyProperties.getEnabled())) { if (Objects.nonNull(httpProxyProperties) && Boolean.TRUE.equals(httpProxyProperties.getEnabled())) {
HttpHost proxy = new HttpHost(httpProxyProperties.getIp(), httpProxyProperties.getPort()); HttpHost proxy = new HttpHost(httpProxyProperties.getIp(), httpProxyProperties.getPort());
clientBuilder.setProxy(proxy); clientBuilder.setProxy(proxy);
if (StringUtils.isNotBlank(httpProxyProperties.getUsername()) && StringUtils.isNotBlank(
httpProxyProperties.getPassword())) {
// 创建CredentialsProvider实例并添加代理认证信息
BasicCredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(
httpProxyProperties.getUsername(), httpProxyProperties.getPassword().toCharArray());
provider.setCredentials(new AuthScope(null, -1), credentials);
clientBuilder.setDefaultCredentialsProvider(provider);
}
} }
CloseableHttpClient httpClient = clientBuilder.setConnectionManager(connectionManager) CloseableHttpClient httpClient = clientBuilder.setConnectionManager(connectionManager)
.build(); .build();

View File

@ -1,37 +0,0 @@
package com.java2nb.novel.core.utils;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.stereotype.Component;
@Component
public class SpringUtil implements ApplicationContextAware {
private static ApplicationContext applicationContext;
public SpringUtil() {
}
public void setApplicationContext(ApplicationContext applicationContext) {
if (SpringUtil.applicationContext == null) {
SpringUtil.applicationContext = applicationContext;
}
}
public static ApplicationContext getApplicationContext() {
return applicationContext;
}
public static Object getBean(String name) {
return getApplicationContext().getBean(name);
}
public static <T> T getBean(Class<T> clazz) {
return (T)getApplicationContext().getBean(clazz);
}
public static <T> T getBean(String name, Class<T> clazz) {
return (T)getApplicationContext().getBean(name, clazz);
}
}

View File

@ -2,9 +2,9 @@ package com.java2nb.novel.entity;
import io.github.xxyopen.web.valid.AddGroup; import io.github.xxyopen.web.valid.AddGroup;
import io.github.xxyopen.web.valid.UpdateGroup; import io.github.xxyopen.web.valid.UpdateGroup;
import jakarta.validation.constraints.*;
import javax.annotation.Generated; import javax.annotation.Generated;
import javax.validation.constraints.*;
import java.util.Date; import java.util.Date;
public class User { public class User {

View File

@ -1,18 +1,71 @@
spring: spring:
config: profiles:
import: classpath:application-common.yml include: [ common ]
main: main:
allow-bean-definition-overriding: true allow-bean-definition-overriding: true
#Redis服务器IP #Redis服务器IP
data:
redis: redis:
host: 127.0.0.1 host: 127.0.0.1
#Redis服务器连接端口 #Redis服务器连接端口
port: 6379 port: 6379
#Redis服务器连接密码 #Redis服务器连接密码
password: test123456 password: test123456
jedis:
pool:
#连接池最大连接数使用负值表示没有限制
max-active: 8
#连接池最大阻塞等待时间使用负值表示没有限制
max-wait: 1
#连接池最大阻塞等待时间使用负值表示没有限制
max-idle: 8
#连接池中的最小空闲连接
min-idle: 0
#连接超时时间毫秒 #连接超时时间毫秒
timeout: 10000 timeout: 30000
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
driver-class-name: com.mysql.cj.jdbc.Driver
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
names: ds0 #,ds1
ds0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# ds1:
# type: com.alibaba.druid.pool.DruidDataSource
# driver-class-name: com.mysql.jdbc.Driver
# url: jdbc:mysql://localhost:3306/novel_plus2
# username: root
# password: test123456
config:
sharding:
props:
sql.show: true
tables:
book_content: #book_content表
key-generator-column-name: id #主键
actual-data-nodes: ds${0}.book_content${0..9} #数据节点
# database-strategy: #分库策略
# inline:
# sharding-column: book_id
# algorithm-expression: ds${book_id % 10}
table-strategy: #分表策略
inline:
shardingColumn: index_id
algorithm-expression: book_content${index_id % 10}
content: content:
save: save:
@ -25,10 +78,6 @@ http:
# 是否开启 HTTP 代理true-开启false-不开启 # 是否开启 HTTP 代理true-开启false-不开启
enabled: false enabled: false
# 代理 IP # 代理 IP
ip: us.swiftproxy.net ip: u493.kdltps.com
# 代理端口号 # 代理端口号
port: 7878 port: 15818
# 代理用户名
username: swiftproxy_u
# 代理密码
password: swiftproxy_p

View File

@ -1,18 +1,71 @@
spring: spring:
config: profiles:
import: classpath:application-common.yml include: [ common ]
main: main:
allow-bean-definition-overriding: true allow-bean-definition-overriding: true
data:
redis:
#Redis服务器IP #Redis服务器IP
redis:
host: 127.0.0.1 host: 127.0.0.1
#Redis服务器连接端口 #Redis服务器连接端口
port: 6379 port: 6379
#Redis服务器连接密码 #Redis服务器连接密码
password: test123456 password: test
jedis:
pool:
#连接池最大连接数使用负值表示没有限制
max-active: 8
#连接池最大阻塞等待时间使用负值表示没有限制
max-wait: 1
#连接池最大阻塞等待时间使用负值表示没有限制
max-idle: 8
#连接池中的最小空闲连接
min-idle: 0
#连接超时时间毫秒 #连接超时时间毫秒
timeout: 10000 timeout: 30000
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_biz?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
driver-class-name: com.mysql.cj.jdbc.Driver
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
names: ds0 #,ds1
ds0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# ds1:
# type: com.alibaba.druid.pool.DruidDataSource
# driver-class-name: com.mysql.jdbc.Driver
# url: jdbc:mysql://localhost:3306/novel_plus2
# username: root
# password: test123456
config:
sharding:
props:
sql.show: true
tables:
book_content: #book_content表
key-generator-column-name: id #主键
actual-data-nodes: ds${0}.book_content${0..9} #数据节点
# database-strategy: #分库策略
# inline:
# sharding-column: book_id
# algorithm-expression: ds${book_id % 10}
table-strategy: #分表策略
inline:
shardingColumn: index_id
algorithm-expression: book_content${index_id % 10}
logging: logging:
level: level:

View File

@ -1,11 +1,16 @@
spring: spring:
datasource: cache:
url: jdbc:shardingsphere:absolutepath:${user.dir}/config/shardingsphere-jdbc.yml ehcache:
driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver config: classpath:ehcache.xml
thymeleaf: thymeleaf:
mode: LEGACYHTML5 #去除thymeleaf的html严格校验thymeleaf.mode=LEGACYHTML5 mode: LEGACYHTML5 #去除thymeleaf的html严格校验thymeleaf.mode=LEGACYHTML5
cache: false # 是否开启模板缓存默认true,建议在开发时关闭缓存,不然没法看到实时 cache: false # 是否开启模板缓存默认true,建议在开发时关闭缓存,不然没法看到实时
# 将所有数字转为 String 类型返回避免前端数据精度丢失的问题
jackson:
generator:
write-numbers-as-strings: true
#上传文件的最大值100M #上传文件的最大值100M
servlet: servlet:
multipart: multipart:
@ -22,8 +27,6 @@ mybatis:
logging: logging:
config: classpath:logback-boot.xml config: classpath:logback-boot.xml
pagehelper:
helper-dialect: mysql

View File

@ -5,7 +5,7 @@
<parent> <parent>
<artifactId>novel</artifactId> <artifactId>novel</artifactId>
<groupId>com.java2nb</groupId> <groupId>com.java2nb</groupId>
<version>5.1.3</version> <version>4.4.0</version>
</parent> </parent>
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
@ -21,12 +21,6 @@
<artifactId>novel-common</artifactId> <artifactId>novel-common</artifactId>
</dependency> </dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>2.2</version>
</dependency>
<dependency> <dependency>
<groupId>org.springframework.boot</groupId> <groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId> <artifactId>spring-boot-starter-security</artifactId>
@ -35,6 +29,7 @@
<dependency> <dependency>
<groupId>com.fasterxml.jackson.core</groupId> <groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId> <artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency> </dependency>

View File

@ -0,0 +1,53 @@
#端口号
server:
port: 8083
#不分表的数据库配置
spring:
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: root
redis:
host: 127.0.0.1
port: 6379
password: test123456
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
ds0:
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
#登录用户名密码
admin:
username: admin
password: admin
#
##爬虫自动更新的线程数
##建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
##随着小说数量的增多可以逐渐增加但建议不要超出CPU的线程数
crawl:
update:
thread: 1
#小说内容保存配置
content:
save:
storage: db # 小说内容存储方式db-数据库txt-TXT文本
path: /Users/xiongxiaoyang/books # 小说TXT文本保存路径
# HTTP 代理配置
http:
proxy:
# 是否开启 HTTP 代理true-开启false-不开启
enabled: false
# 代理 IP
ip: u493.kdltps.com
# 代理端口号
port: 15818

View File

@ -1,47 +0,0 @@
#端口号
server:
port: 8083
spring:
data:
redis:
#Redis服务器IP
host: 127.0.0.1
#Redis服务器连接端口
port: 6379
#Redis服务器连接密码
password: test123456
#连接超时时间毫秒
timeout: 10000
#登录用户名密码
admin:
username: admin
password: admin
#
##爬虫自动更新的线程数
##建议小说数量不多或者正在运行新书入库爬虫的情况下设置为1即可
##随着小说数量的增多可以逐渐增加但建议不要超出CPU的线程数
crawl:
update:
thread: 1
#小说内容保存配置
content:
save:
storage: db # 小说内容存储方式db-数据库txt-TXT文本
path: /Users/xiongxiaoyang/books # 小说TXT文本保存路径
# HTTP 代理配置
http:
proxy:
# 是否开启 HTTP 代理true-开启false-不开启
enabled: false
# 代理 IP
ip: us.swiftproxy.net
# 代理端口号
port: 7878
# 代理用户名
username: swiftproxy_u
# 代理密码
password: swiftproxy_p

View File

@ -1,47 +0,0 @@
mode:
# 单机模式
type: Standalone
# 元数据持久化
repository:
# 数据库持久化
type: JDBC
# 数据源配置
dataSources:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# 规则配置
rules:
- !SINGLE
tables:
- "*.*"
- !SHARDING
tables: # 数据分片规则配置
book_content:
# 分库策略缺省表示使用默认分库策略
actualDataNodes: ds_${1}.book_content${0..9}
# 分表策略
tableStrategy:
standard:
# 分片列名称
shardingColumn: index_id
# 分片算法名称
shardingAlgorithmName: bookContentSharding
shardingAlgorithms:
bookContentSharding:
# 行表达式分片算法使用 Groovy 的表达式提供对 SQL 语句中的 = IN 的分片操作支持
type: INLINE
props:
# 分片算法的行表达式
algorithm-expression: book_content${index_id % 10}
props:
# 是否在日志中打印 SQL
sql-show: true

View File

@ -4,15 +4,14 @@ import lombok.RequiredArgsConstructor;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer; import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity; import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.builders.WebSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.core.userdetails.User; import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder; import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.security.crypto.password.PasswordEncoder; import org.springframework.security.crypto.password.PasswordEncoder;
import org.springframework.security.provisioning.InMemoryUserDetailsManager;
import org.springframework.security.web.SecurityFilterChain;
/** /**
* SpringSecurity配置 * SpringSecurity配置
@ -22,7 +21,7 @@ import org.springframework.security.web.SecurityFilterChain;
@Configuration @Configuration
@EnableWebSecurity @EnableWebSecurity
@RequiredArgsConstructor @RequiredArgsConstructor
public class SecurityConfiguration { public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
@Value("${admin.username}") @Value("${admin.username}")
private String username; private String username;
@ -30,40 +29,39 @@ public class SecurityConfiguration {
@Value("${admin.password}") @Value("${admin.password}")
private String password; private String password;
@Bean @Bean
public PasswordEncoder passwordEncoder() { public PasswordEncoder passwordEncoder() {
return new BCryptPasswordEncoder(); return new BCryptPasswordEncoder();
} }
@Bean
public InMemoryUserDetailsManager userDetailsService() { @Override
UserDetails admin = User.builder() public void configure(WebSecurity web) throws Exception {
.username(username) super.configure(web);
.password(passwordEncoder().encode(password))
.roles("ADMIN")
.build();
return new InMemoryUserDetailsManager(admin);
} }
@Bean @Override
public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception { public void configure(AuthenticationManagerBuilder auth) throws Exception {
http
.csrf(csrf -> csrf.disable()) // 禁用 CSRF
.authorizeHttpRequests(auth -> auth
.requestMatchers("/css/**", "/favicon.ico").permitAll() // 允许访问静态资源
.anyRequest().hasRole("ADMIN") // 其他请求需要 ADMIN 角色
)
.formLogin(form -> form
.loginPage("/login.html") // 自定义登录页面
.loginProcessingUrl("/login") // 登录处理 URL
.permitAll()
)
.logout(logout -> logout
.logoutUrl("/logout") // 登出 URL
.logoutSuccessUrl("/") // 登出成功后跳转的页面
)
.httpBasic(Customizer.withDefaults()); // 启用 HTTP Basic 认证
return http.build(); User.UserBuilder builder = User.builder().passwordEncoder(passwordEncoder()::encode);
auth.inMemoryAuthentication().withUser(builder.username(username).password(password).roles("ADMIN").build());
} }
@Override
protected void configure(HttpSecurity http) throws Exception {
http.csrf().disable()
.authorizeRequests()
.antMatchers("/css/**").permitAll()
.antMatchers("/favicon.ico").permitAll()
.antMatchers("/**").hasRole("ADMIN")
.and().formLogin().loginPage("/login.html").loginProcessingUrl("/login").permitAll()
.and().logout()
.logoutUrl("/logout")
.logoutSuccessUrl("/")
.and().httpBasic();
}
} }

View File

@ -7,6 +7,6 @@ import com.java2nb.novel.entity.Book;
* */ * */
public interface CrawlBookHandler { public interface CrawlBookHandler {
void handle(Book book) throws InterruptedException; void handle(Book book);
} }

View File

@ -10,11 +10,9 @@ import com.java2nb.novel.utils.CrawlHttpClient;
import io.github.xxyopen.util.IdWorker; import io.github.xxyopen.util.IdWorker;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Component; import org.springframework.stereotype.Component;
import java.text.ParseException;
import java.text.SimpleDateFormat; import java.text.SimpleDateFormat;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Date; import java.util.Date;
@ -28,7 +26,6 @@ import java.util.regex.Pattern;
* *
* @author Administrator * @author Administrator
*/ */
@Slf4j
@Component @Component
@RequiredArgsConstructor @RequiredArgsConstructor
public class CrawlParser { public class CrawlParser {
@ -37,11 +34,11 @@ public class CrawlParser {
private final CrawlHttpClient crawlHttpClient; private final CrawlHttpClient crawlHttpClient;
public void parseBook(RuleBean ruleBean, String bookId, CrawlBookHandler handler) @SneakyThrows
throws InterruptedException { public void parseBook(RuleBean ruleBean, String bookId, CrawlBookHandler handler) {
Book book = new Book(); Book book = new Book();
String bookDetailUrl = ruleBean.getBookDetailUrl().replace("{bookId}", bookId); String bookDetailUrl = ruleBean.getBookDetailUrl().replace("{bookId}", bookId);
String bookDetailHtml = crawlHttpClient.get(bookDetailUrl, ruleBean.getCharset()); String bookDetailHtml = crawlHttpClient.get(bookDetailUrl);
if (bookDetailHtml != null) { if (bookDetailHtml != null) {
Pattern bookNamePatten = PatternFactory.getPattern(ruleBean.getBookNamePatten()); Pattern bookNamePatten = PatternFactory.getPattern(ruleBean.getBookNamePatten());
Matcher bookNameMatch = bookNamePatten.matcher(bookDetailHtml); Matcher bookNameMatch = bookNamePatten.matcher(bookDetailHtml);
@ -123,12 +120,8 @@ public class CrawlParser {
if (isFindUpdateTime) { if (isFindUpdateTime) {
String updateTime = updateTimeMatch.group(1); String updateTime = updateTimeMatch.group(1);
//设置更新时间 //设置更新时间
try {
book.setLastIndexUpdateTime( book.setLastIndexUpdateTime(
new SimpleDateFormat(ruleBean.getUpadateTimeFormatPatten()).parse(updateTime)); new SimpleDateFormat(ruleBean.getUpadateTimeFormatPatten()).parse(updateTime));
} catch (ParseException e) {
log.error("解析最新章节更新时间出错", e);
}
} }
} }
@ -151,7 +144,7 @@ public class CrawlParser {
} }
public boolean parseBookIndexAndContent(String sourceBookId, Book book, RuleBean ruleBean, public boolean parseBookIndexAndContent(String sourceBookId, Book book, RuleBean ruleBean,
Map<Integer, BookIndex> existBookIndexMap, CrawlBookChapterHandler handler) throws InterruptedException{ Map<Integer, BookIndex> existBookIndexMap, CrawlBookChapterHandler handler) {
Date currentDate = new Date(); Date currentDate = new Date();
@ -159,7 +152,7 @@ public class CrawlParser {
List<BookContent> contentList = new ArrayList<>(); List<BookContent> contentList = new ArrayList<>();
//读取目录 //读取目录
String indexListUrl = ruleBean.getBookIndexUrl().replace("{bookId}", sourceBookId); String indexListUrl = ruleBean.getBookIndexUrl().replace("{bookId}", sourceBookId);
String indexListHtml = crawlHttpClient.get(indexListUrl, ruleBean.getCharset()); String indexListHtml = crawlHttpClient.get(indexListUrl);
if (indexListHtml != null) { if (indexListHtml != null) {
if (StringUtils.isNotBlank(ruleBean.getBookIndexStart())) { if (StringUtils.isNotBlank(ruleBean.getBookIndexStart())) {
@ -223,7 +216,7 @@ public class CrawlParser {
.replace("{indexId}", sourceIndexId); .replace("{indexId}", sourceIndexId);
//查询章节内容 //查询章节内容
String contentHtml = crawlHttpClient.get(contentUrl, ruleBean.getCharset()); String contentHtml = crawlHttpClient.get(contentUrl);
if (contentHtml != null && !contentHtml.contains("正在手打中")) { if (contentHtml != null && !contentHtml.contains("正在手打中")) {
String content = contentHtml.substring( String content = contentHtml.substring(
contentHtml.indexOf(ruleBean.getContentStart()) + ruleBean.getContentStart().length()); contentHtml.indexOf(ruleBean.getContentStart()) + ruleBean.getContentStart().length());

View File

@ -1,6 +1,5 @@
package com.java2nb.novel.core.crawl; package com.java2nb.novel.core.crawl;
import com.java2nb.novel.utils.Constants;
import lombok.Data; import lombok.Data;
import java.util.Map; import java.util.Map;
@ -13,12 +12,6 @@ import java.util.Map;
@Data @Data
public class RuleBean { public class RuleBean {
/**
* 网页字符编码
*/
private String charset = Constants.CRAWL_DEFAULT_CHARSET;
/** /**
* 小说更新列表url * 小说更新列表url
*/ */

View File

@ -10,14 +10,14 @@ import com.java2nb.novel.entity.CrawlSource;
import com.java2nb.novel.service.BookService; import com.java2nb.novel.service.BookService;
import com.java2nb.novel.service.CrawlService; import com.java2nb.novel.service.CrawlService;
import com.java2nb.novel.utils.Constants; import com.java2nb.novel.utils.Constants;
import jakarta.servlet.ServletContext;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.time.DateUtils; import org.apache.commons.lang3.time.DateUtils;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.servlet.ServletContextInitializer;
import org.springframework.stereotype.Component;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
import java.util.Date; import java.util.Date;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@ -26,10 +26,10 @@ import java.util.concurrent.TimeUnit;
/** /**
* @author Administrator * @author Administrator
*/ */
@Component @WebListener
@Slf4j @Slf4j
@RequiredArgsConstructor @RequiredArgsConstructor
public class StarterListener implements ServletContextInitializer { public class StarterListener implements ServletContextListener {
private final BookService bookService; private final BookService bookService;
@ -41,7 +41,7 @@ public class StarterListener implements ServletContextInitializer {
private int updateThreadCount; private int updateThreadCount;
@Override @Override
public void onStartup(ServletContext servletContext) { public void contextInitialized(ServletContextEvent sce) {
for (int i = 0; i < updateThreadCount; i++) { for (int i = 0; i < updateThreadCount; i++) {
new Thread(() -> { new Thread(() -> {
log.info("程序启动,开始执行自动更新线程。。。"); log.info("程序启动,开始执行自动更新线程。。。");
@ -74,8 +74,10 @@ public class StarterListener implements ServletContextInitializer {
needUpdateBook.getId()); needUpdateBook.getId());
//解析章节目录 //解析章节目录
crawlParser.parseBookIndexAndContent(needUpdateBook.getCrawlBookId(), book, crawlParser.parseBookIndexAndContent(needUpdateBook.getCrawlBookId(), book,
ruleBean, existBookIndexMap, chapter -> bookService.updateBookAndIndexAndContent(book, chapter.getBookIndexList(), ruleBean, existBookIndexMap, chapter -> {
chapter.getBookContentList(), existBookIndexMap)); bookService.updateBookAndIndexAndContent(book, chapter.getBookIndexList(),
chapter.getBookContentList(), existBookIndexMap);
});
}); });
} catch (Exception e) { } catch (Exception e) {
log.error(e.getMessage(), e); log.error(e.getMessage(), e);

View File

@ -0,0 +1,61 @@
package com.java2nb.novel.core.schedule;
import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.entity.CrawlSource;
import com.java2nb.novel.service.CrawlService;
import io.github.xxyopen.util.ThreadUtil;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import java.util.List;
import java.util.Set;
/**
* 爬虫线程监控器,监控执行完成的爬虫源并修改状态
*
* @author Administrator
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class CrawlThreadMonitor {
private final CacheService cacheService;
private final CrawlService crawlService;
@Scheduled(fixedRate = 1000 * 60 * 5)
public void monitor() {
//查询需要监控的正在运行的爬虫源
List<CrawlSource> sources = crawlService.queryCrawlSourceByStatus((byte) 1);
for (CrawlSource source : sources) {
Set<Long> runningCrawlThreadIds = (Set<Long>) cacheService.getObject(CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + source.getId());
boolean sourceStop = true;
if (runningCrawlThreadIds != null) {
for (Long threadId : runningCrawlThreadIds) {
Thread thread = ThreadUtil.findThread(threadId);
if (thread != null && thread.isAlive()) {
//有活跃线程说明该爬虫源正在运行数据库中状态正确不需要修改
sourceStop = false;
}
}
}
if (sourceStop) {
crawlService.updateCrawlSourceStatus(source.getId(), (byte) 0);
}
}
}
}

View File

@ -53,7 +53,7 @@ public interface CrawlService {
* @param ruleBean 采集规则\ * @param ruleBean 采集规则\
* @return true:成功false:失败 * @return true:成功false:失败
* */ * */
boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId) throws InterruptedException; boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId);
/** /**
* 根据爬虫状态查询爬虫源集合 * 根据爬虫状态查询爬虫源集合

View File

@ -7,7 +7,6 @@ import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.crawl.CrawlParser; import com.java2nb.novel.core.crawl.CrawlParser;
import com.java2nb.novel.core.crawl.RuleBean; import com.java2nb.novel.core.crawl.RuleBean;
import com.java2nb.novel.core.enums.ResponseStatus; import com.java2nb.novel.core.enums.ResponseStatus;
import com.java2nb.novel.core.utils.SpringUtil;
import com.java2nb.novel.entity.Book; import com.java2nb.novel.entity.Book;
import com.java2nb.novel.entity.CrawlSingleTask; import com.java2nb.novel.entity.CrawlSingleTask;
import com.java2nb.novel.entity.CrawlSource; import com.java2nb.novel.entity.CrawlSource;
@ -26,6 +25,7 @@ import io.github.xxyopen.util.IdWorker;
import io.github.xxyopen.util.ThreadUtil; import io.github.xxyopen.util.ThreadUtil;
import io.github.xxyopen.web.exception.BusinessException; import io.github.xxyopen.web.exception.BusinessException;
import io.github.xxyopen.web.util.BeanUtil; import io.github.xxyopen.web.util.BeanUtil;
import io.github.xxyopen.web.util.SpringUtil;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@ -34,7 +34,6 @@ import org.mybatis.dynamic.sql.render.RenderingStrategies;
import org.mybatis.dynamic.sql.select.render.SelectStatementProvider; import org.mybatis.dynamic.sql.select.render.SelectStatementProvider;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import java.time.Duration;
import java.util.*; import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher; import java.util.regex.Matcher;
@ -66,10 +65,6 @@ public class CrawlServiceImpl implements CrawlService {
private final CrawlHttpClient crawlHttpClient; private final CrawlHttpClient crawlHttpClient;
private final Map<Integer, Byte> crawlSourceStatusMap = new HashMap<>();
private final Map<Integer, Set<Long>> runningCrawlThread = new HashMap<>();
@Override @Override
public void addCrawlSource(CrawlSource source) { public void addCrawlSource(CrawlSource source) {
@ -108,8 +103,6 @@ public class CrawlServiceImpl implements CrawlService {
.build() .build()
.render(RenderingStrategies.MYBATIS3); .render(RenderingStrategies.MYBATIS3);
List<CrawlSource> crawlSources = crawlSourceMapper.selectMany(render); List<CrawlSource> crawlSources = crawlSourceMapper.selectMany(render);
crawlSources.forEach(crawlSource -> crawlSource.setSourceStatus(
Optional.ofNullable(crawlSourceStatusMap.get(crawlSource.getId())).orElse((byte) 0)));
PageBean<CrawlSource> pageBean = PageBuilder.build(crawlSources); PageBean<CrawlSource> pageBean = PageBuilder.build(crawlSources);
pageBean.setList(BeanUtil.copyList(crawlSources, CrawlSourceVO.class)); pageBean.setList(BeanUtil.copyList(crawlSources, CrawlSourceVO.class));
return pageBean; return pageBean;
@ -119,13 +112,14 @@ public class CrawlServiceImpl implements CrawlService {
@Override @Override
public void openOrCloseCrawl(Integer sourceId, Byte sourceStatus) { public void openOrCloseCrawl(Integer sourceId, Byte sourceStatus) {
// 判断是开启还是关闭如果是关闭获取该爬虫源正在运行的线程集合并全部中断 //判断是开启还是关闭如果是关闭修改数据库状态后获取该爬虫正在运行的线程集合并全部停止
// 如果是开启判断该爬虫源是否还在运行如果在运行则忽略如果没有运行则启动线程爬取小说数据并加入到runningCrawlThread中 //如果是开启查询数据库中状态判断该爬虫源是否还在运行如果在运行则忽略
// 最后保存爬虫源状态 // 如果没有则修改数据库状态并启动线程爬取小说数据加入到runningCrawlThread中
if (sourceStatus == (byte) 0) { if (sourceStatus == (byte) 0) {
// 关闭 //关闭,直接修改数据库状态并直接修改数据库状态后获取该爬虫正在运行的线程集合全部停止
// 将该爬虫源正在运行的线程集合全部停止 SpringUtil.getBean(CrawlService.class).updateCrawlSourceStatus(sourceId, sourceStatus);
Set<Long> runningCrawlThreadId = runningCrawlThread.get(sourceId); Set<Long> runningCrawlThreadId = (Set<Long>) cacheService.getObject(
CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + sourceId);
if (runningCrawlThreadId != null) { if (runningCrawlThreadId != null) {
for (Long ThreadId : runningCrawlThreadId) { for (Long ThreadId : runningCrawlThreadId) {
Thread thread = ThreadUtil.findThread(ThreadId); Thread thread = ThreadUtil.findThread(ThreadId);
@ -137,13 +131,16 @@ public class CrawlServiceImpl implements CrawlService {
} else { } else {
// 开启 //开启
Byte realSourceStatus = Optional.ofNullable(crawlSourceStatusMap.get(sourceId)).orElse((byte) 0); //查询爬虫源状态和规则
if (realSourceStatus == (byte) 0) {
// 查询爬虫源规则
CrawlSource source = queryCrawlSource(sourceId); CrawlSource source = queryCrawlSource(sourceId);
//该爬虫源已经停止运行了,启动线程爬取小说数据并将线程加入到runningCrawlThread中 Byte realSourceStatus = source.getSourceStatus();
if (realSourceStatus == (byte) 0) {
//该爬虫源已经停止运行了,修改数据库状态并启动线程爬取小说数据加入到runningCrawlThread中
SpringUtil.getBean(CrawlService.class).updateCrawlSourceStatus(sourceId, sourceStatus);
RuleBean ruleBean = new ObjectMapper().readValue(source.getCrawlRule(), RuleBean.class); RuleBean ruleBean = new ObjectMapper().readValue(source.getCrawlRule(), RuleBean.class);
Set<Long> threadIds = new HashSet<>(); Set<Long> threadIds = new HashSet<>();
//按分类开始爬虫解析任务 //按分类开始爬虫解析任务
for (int i = 1; i < 8; i++) { for (int i = 1; i < 8; i++) {
@ -152,14 +149,15 @@ public class CrawlServiceImpl implements CrawlService {
thread.start(); thread.start();
//thread加入到监控缓存中 //thread加入到监控缓存中
threadIds.add(thread.getId()); threadIds.add(thread.getId());
} }
runningCrawlThread.put(sourceId, threadIds); cacheService.setObject(CacheKey.RUNNING_CRAWL_THREAD_KEY_PREFIX + sourceId, threadIds);
}
} }
// 保存爬虫源状态
crawlSourceStatusMap.put(sourceId, sourceStatus); }
} }
@ -250,11 +248,6 @@ public class CrawlServiceImpl implements CrawlService {
@Override @Override
public void parseBookList(int catId, RuleBean ruleBean, Integer sourceId) { public void parseBookList(int catId, RuleBean ruleBean, Integer sourceId) {
String catIdRule = ruleBean.getCatIdRule().get("catId" + catId);
if (StringUtils.isBlank(catIdRule)) {
return;
}
//当前页码1 //当前页码1
int page = 1; int page = 1;
int totalPage = page; int totalPage = page;
@ -262,21 +255,14 @@ public class CrawlServiceImpl implements CrawlService {
while (page <= totalPage) { while (page <= totalPage) {
try { try {
String catBookListUrl;
if (StringUtils.isNotBlank(ruleBean.getBookListUrl())) {
// 兼容老规则
// 拼接分类URL
catBookListUrl = ruleBean.getBookListUrl()
.replace("{catId}", catIdRule)
.replace("{page}", page + "");
} else {
// 新规则
// 拼接分类URL
catBookListUrl = catIdRule.replace("{page}", page + "");
}
log.info("catBookListUrl{}", catBookListUrl);
String bookListHtml = crawlHttpClient.get(catBookListUrl, ruleBean.getCharset()); if (StringUtils.isNotBlank(ruleBean.getCatIdRule().get("catId" + catId))) {
//拼接分类URL
String catBookListUrl = ruleBean.getBookListUrl()
.replace("{catId}", ruleBean.getCatIdRule().get("catId" + catId))
.replace("{page}", page + "");
String bookListHtml = crawlHttpClient.get(catBookListUrl);
if (bookListHtml != null) { if (bookListHtml != null) {
Pattern bookIdPatten = Pattern.compile(ruleBean.getBookIdPatten()); Pattern bookIdPatten = Pattern.compile(ruleBean.getBookIdPatten());
Matcher bookIdMatcher = bookIdPatten.matcher(bookListHtml); Matcher bookIdMatcher = bookIdPatten.matcher(bookListHtml);
@ -292,12 +278,6 @@ public class CrawlServiceImpl implements CrawlService {
String bookId = bookIdMatcher.group(1); String bookId = bookIdMatcher.group(1);
parseBookAndSave(catId, ruleBean, sourceId, bookId); parseBookAndSave(catId, ruleBean, sourceId, bookId);
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
//1.阻塞过程使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时
//捕获中断异常InterruptedException来退出线程
//2.非阻塞过程中通过判断中断标志来退出线程
return;
} catch (Exception e) { } catch (Exception e) {
log.error(e.getMessage(), e); log.error(e.getMessage(), e);
} }
@ -313,40 +293,22 @@ public class CrawlServiceImpl implements CrawlService {
totalPage = Integer.parseInt(totalPageMatcher.group(1)); totalPage = Integer.parseInt(totalPageMatcher.group(1));
} }
}
} }
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
//1.阻塞过程使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时
//捕获中断异常InterruptedException来退出线程
//2.非阻塞过程中通过判断中断标志来退出线程
return;
} catch (Exception e) { } catch (Exception e) {
log.error(e.getMessage(), e); log.error(e.getMessage(), e);
} }
if (page >= totalPage) {
// 第一遍采集完成翻到第一页继续第二次采集适用于分页数比较少的最近更新列表
page = 1;
try {
// 第一遍采集完成休眠1分钟
Thread.sleep(Duration.ofMinutes(1));
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
//1.阻塞过程使用了 sleep,同步锁的 wait,socket 中的 receiver,accept 等方法时
//捕获中断异常InterruptedException来退出线程
//2.非阻塞过程中通过判断中断标志来退出线程
return;
}
} else {
page += 1; page += 1;
} }
}
} }
@Override @Override
public boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId) public boolean parseBookAndSave(int catId, RuleBean ruleBean, Integer sourceId, String bookId) {
throws InterruptedException {
final AtomicBoolean parseResult = new AtomicBoolean(false); final AtomicBoolean parseResult = new AtomicBoolean(false);
@ -410,5 +372,4 @@ public class CrawlServiceImpl implements CrawlService {
.render(RenderingStrategies.MYBATIS3); .render(RenderingStrategies.MYBATIS3);
return crawlSourceMapper.selectMany(render); return crawlSourceMapper.selectMany(render);
} }
} }

View File

@ -24,9 +24,4 @@ public class Constants {
* 爬取小说http请求失败重试次数 * 爬取小说http请求失败重试次数
*/ */
public static final Integer HTTP_FAIL_RETRY_COUNT = 3; public static final Integer HTTP_FAIL_RETRY_COUNT = 3;
/**
* 爬虫默认编码
*/
public static final String CRAWL_DEFAULT_CHARSET = "UTF-8";
} }

View File

@ -25,26 +25,30 @@ public class CrawlHttpClient {
private static final ThreadLocal<Integer> RETRY_COUNT = new ThreadLocal<>(); private static final ThreadLocal<Integer> RETRY_COUNT = new ThreadLocal<>();
public String get(String url, String charset) throws InterruptedException { public String get(String url) {
if (Objects.nonNull(intervalMin) && Objects.nonNull(intervalMax) && intervalMax > intervalMin) { if (Objects.nonNull(intervalMin) && Objects.nonNull(intervalMax) && intervalMax > intervalMin) {
try {
Thread.sleep(random.nextInt(intervalMax - intervalMin + 1) + intervalMin); Thread.sleep(random.nextInt(intervalMax - intervalMin + 1) + intervalMin);
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
} }
String body = HttpUtil.getByHttpClientWithChrome(url, charset); }
String body = HttpUtil.getByHttpClientWithChrome(url);
if (Objects.isNull(body) || body.length() < Constants.INVALID_HTML_LENGTH) { if (Objects.isNull(body) || body.length() < Constants.INVALID_HTML_LENGTH) {
return processErrorHttpResult(url, charset); return processErrorHttpResult(url);
} }
//成功获得html内容 //成功获得html内容
return body; return body;
} }
private String processErrorHttpResult(String url, String charset) throws InterruptedException{ private String processErrorHttpResult(String url) {
Integer count = RETRY_COUNT.get(); Integer count = RETRY_COUNT.get();
if (count == null) { if (count == null) {
count = 0; count = 0;
} }
if (count < Constants.HTTP_FAIL_RETRY_COUNT) { if (count < Constants.HTTP_FAIL_RETRY_COUNT) {
RETRY_COUNT.set(++count); RETRY_COUNT.set(++count);
return get(url, charset); return get(url);
} }
RETRY_COUNT.remove(); RETRY_COUNT.remove();
return null; return null;

View File

@ -1,3 +1,3 @@
spring: spring:
config: profiles:
import: classpath:application-common-dev.yml include: [common-dev]

View File

@ -1,3 +1,3 @@
spring: spring:
config: profiles:
import: classpath:application-common-prod.yml include: [common-prod]

View File

Before

Width:  |  Height:  |  Size: 162 KiB

After

Width:  |  Height:  |  Size: 162 KiB

View File

@ -12,6 +12,9 @@
<!-- ConsoleAppender把日志输出到控制台 --> <!-- ConsoleAppender把日志输出到控制台 -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder> <encoder>
<!--
<pattern>%d %p (%file:%line\)- %m%n</pattern>
-->
<pattern>${CONSOLE_LOG_PATTERN}</pattern> <pattern>${CONSOLE_LOG_PATTERN}</pattern>
<!-- 控制台也要使用UTF-8不要使用GBK否则会中文乱码 --> <!-- 控制台也要使用UTF-8不要使用GBK否则会中文乱码 -->
<charset>UTF-8</charset> <charset>UTF-8</charset>
@ -19,40 +22,40 @@
</appender> </appender>
<!-- RollingFileAppender滚动记录文件先将日志记录到指定文件当符合某个条件时将日志记录到其他文件 --> <!-- RollingFileAppender滚动记录文件先将日志记录到指定文件当符合某个条件时将日志记录到其他文件 -->
<!-- 以下的大概意思是1.先按日期存日志日期变了将前一天的日志文件名重命名为XXX%日期%索引新的日志仍然是novel-crawl.log --> <!-- 以下的大概意思是1.先按日期存日志日期变了将前一天的日志文件名重命名为XXX%日期%索引新的日志仍然是demo.log -->
<!-- 2.如果日期没有发生变化但是当前日志的文件大小超过10MB时对当前日志进行分割 重命名 --> <!-- 2.如果日期没有发生变化但是当前日志的文件大小超过1KB时对当前日志进行分割 重命名 -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 设置日志文件路径 -->
<file>logs/novel-crawl.log</file> <File>logs/novel-crawl.log</File>
<!-- rollingPolicy:当发生滚动时决定 RollingFileAppender 的行为涉及文件移动和重命名 --> <!-- rollingPolicy:当发生滚动时决定 RollingFileAppender 的行为涉及文件移动和重命名 -->
<!-- SizeAndTimeBasedRollingPolicy基于时间和文件大小的滚动策略 --> <!-- TimeBasedRollingPolicy 最常用的滚动策略它根据时间来制定滚动策略既负责滚动也负责出发滚动 -->
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 活动文件的名字会根据fileNamePattern的值每隔一段时间改变一次 --> <!-- 活动文件的名字会根据fileNamePattern的值每隔一段时间改变一次 -->
<!-- 文件名logs/debug.2023-10-01.0.log --> <!-- 文件名logs/demo.2017-12-05.0.log -->
<fileNamePattern>logs/debug.%d{yyyy-MM-dd}.%i.log</fileNamePattern> <fileNamePattern>logs/debug.%d.%i.log</fileNamePattern>
<!-- 每产生一个日志文件该日志文件的保存期限为30天 --> <!-- 每产生一个日志文件该日志文件的保存期限为30天 -->
<maxHistory>30</maxHistory> <maxHistory>30</maxHistory>
<!-- 单个日志文件的最大大小 --> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<!-- maxFileSize:这是活动文件的大小默认值是10MB测试时可改成1KB看效果 -->
<maxFileSize>10MB</maxFileSize> <maxFileSize>10MB</maxFileSize>
<!-- 所有日志文件的总大小限制 --> </timeBasedFileNamingAndTriggeringPolicy>
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy> </rollingPolicy>
<encoder> <encoder>
<!-- pattern节点用来设置日志的输入格式 --> <!-- pattern节点用来设置日志的输入格式 -->
<pattern>%d %p (%file:%line\)- %m%n</pattern> <pattern>
%d %p (%file:%line\)- %m%n
</pattern>
<!-- 记录日志的编码:此处设置字符集 - --> <!-- 记录日志的编码:此处设置字符集 - -->
<charset>UTF-8</charset> <charset>UTF-8</charset>
</encoder> </encoder>
</appender> </appender>
<!-- 控制台输出日志级别 --> <!-- 控制台输出日志级别 -->
<root level="INFO"> <root level="INFO">
<appender-ref ref="STDOUT" /> <appender-ref ref="STDOUT" />
<appender-ref ref="FILE" /> <appender-ref ref="FILE" />
</root> </root>
<!-- 指定项目中某个包当有日志操作行为时的日志记录级别 --> <!-- 指定项目中某个包当有日志操作行为时的日志记录级别 -->
<!-- com.java2nb 为根包也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG --> <!-- com.maijinjie.springboot 为根包也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG -->
<!-- 级别依次为从高到低FATAL > ERROR > WARN > INFO > DEBUG > TRACE --> <!-- 级别依次为从高到低FATAL > ERROR > WARN > INFO > DEBUG > TRACE -->
<logger name="com.java2nb" level="DEBUG" additivity="false"> <logger name="com.java2nb" level="DEBUG" additivity="false">
<appender-ref ref="STDOUT" /> <appender-ref ref="STDOUT" />

View File

@ -54,8 +54,6 @@
<!--示例<b>https://m.xdingdiann.com/sort/0/1.html</b> <!--示例<b>https://m.xdingdiann.com/sort/0/1.html</b>
<li><input type="text" id="updateBookListUrl" class="s_input icon_key" <li><input type="text" id="updateBookListUrl" class="s_input icon_key"
placeholder="小说更新列表url"></li>--> placeholder="小说更新列表url"></li>-->
示例<b>utf-8</b>
<li><input type="text" id="charset" class="s_input icon_name" placeholder="网站编码"></li>
示例<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID{page}代表分页页码) 示例<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID{page}代表分页页码)
<li><input type="text" id="bookListUrl" class="s_input icon_key" <li><input type="text" id="bookListUrl" class="s_input icon_key"
placeholder="分类列表页URL规则"></li> placeholder="分类列表页URL规则"></li>
@ -413,11 +411,6 @@
var filterContent = $("#filterContent").val(); var filterContent = $("#filterContent").val();
crawlRule.filterContent = filterContent; crawlRule.filterContent = filterContent;
var charset = $('#charset').val();
if (charset) {
crawlRule.charset = charset;
}
$.ajax({ $.ajax({
type: "POST", type: "POST",

View File

@ -55,8 +55,6 @@
<!--示例<b>https://m.xdingdiann.com/sort/0/1.html</b> <!--示例<b>https://m.xdingdiann.com/sort/0/1.html</b>
<li><input type="text" id="updateBookListUrl" class="s_input icon_key" <li><input type="text" id="updateBookListUrl" class="s_input icon_key"
placeholder="小说更新列表url"></li>--> placeholder="小说更新列表url"></li>-->
示例<b>utf-8</b>
<li><input type="text" id="charset" class="s_input icon_name" placeholder="网站编码"></li>
示例<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID{page}代表分页页码) 示例<b>http://m.xdingdiann.com/sort/{catId}/{page}.html</b> ({catId}代表分类ID{page}代表分页页码)
<li><input type="text" id="bookListUrl" class="s_input icon_key" <li><input type="text" id="bookListUrl" class="s_input icon_key"
placeholder="分类列表页URL规则"></li> placeholder="分类列表页URL规则"></li>
@ -276,7 +274,6 @@
$("#contentStart").val(crawlRule.contentStart); $("#contentStart").val(crawlRule.contentStart);
$("#contentEnd").val(crawlRule.contentEnd); $("#contentEnd").val(crawlRule.contentEnd);
$("#filterContent").val(crawlRule.filterContent); $("#filterContent").val(crawlRule.filterContent);
$("#charset").val(crawlRule.charset);
} }
} }
@ -499,11 +496,6 @@
var filterContent = $("#filterContent").val(); var filterContent = $("#filterContent").val();
crawlRule.filterContent = filterContent; crawlRule.filterContent = filterContent;
var charset = $('#charset').val();
if (charset) {
crawlRule.charset = charset;
}
$.ajax({ $.ajax({
type: "POST", type: "POST",

View File

@ -5,7 +5,7 @@
<parent> <parent>
<artifactId>novel</artifactId> <artifactId>novel</artifactId>
<groupId>com.java2nb</groupId> <groupId>com.java2nb</groupId>
<version>5.1.3</version> <version>4.4.0</version>
</parent> </parent>
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
@ -20,12 +20,6 @@
<artifactId>novel-common</artifactId> <artifactId>novel-common</artifactId>
</dependency> </dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>2.2</version>
</dependency>
<!--JWT(Json Web Token)登录支持--> <!--JWT(Json Web Token)登录支持-->
<dependency> <dependency>
<groupId>io.jsonwebtoken</groupId> <groupId>io.jsonwebtoken</groupId>
@ -54,12 +48,7 @@
<dependency> <dependency>
<groupId>com.fasterxml.jackson.core</groupId> <groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId> <artifactId>jackson-databind</artifactId>
</dependency> <version>${jackson.version}</version>
<!-- AI -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency> </dependency>
</dependencies> </dependencies>

View File

@ -0,0 +1,56 @@
#端口号
server:
port: 8085
#不分表的数据库配置
spring:
datasource:
url: jdbc:mysql://127.0.0.1:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: root
redis:
host: 127.0.0.1
port: 6379
password: test123456
####使用shardingJdbc时
####所有的jdbcType都不能是LONGVARCHAR,否则会导致java.io.NotSerializableException: java.io.StringReader错误
##### 应该替换所有的 LONGVARCHAR 类型为VARCHAR
sharding:
jdbc:
datasource:
ds0:
jdbc-url: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
pic:
save:
#图片保存方式, 1不保存使用网络图片 2本地保存
type: 2
#图片保存路径
path: /var/pic
#模版配置
templates:
name: green
#小说内容保存配置
content:
save:
storage: db # 小说内容存储方式db-数据库txt-TXT文本
path: /Users/xiongxiaoyang/books # 小说TXT文本保存路径
# HTTP 代理配置
http:
proxy:
# 是否开启 HTTP 代理true-开启false-不开启
enabled: false
# 代理 IP
ip: u493.kdltps.com
# 代理端口号
port: 15818

View File

@ -1,68 +0,0 @@
#端口号
server:
port: 8085
spring:
data:
redis:
#Redis服务器IP
host: 127.0.0.1
#Redis服务器连接端口
port: 6379
#Redis服务器连接密码
password: test123456
#连接超时时间毫秒
timeout: 10000
pic:
save:
#图片保存方式, 1不保存使用网络图片 2本地保存
type: 2
#图片保存路径
path: /var/pic
#模版配置
templates:
name: green
#小说内容保存配置
content:
save:
storage: db # 小说内容存储方式db-数据库txt-TXT文本
path: /Users/xiongxiaoyang/books # 小说TXT文本保存路径
# HTTP 代理配置
http:
proxy:
# 是否开启 HTTP 代理true-开启false-不开启
enabled: false
# 代理 IP
ip: us.swiftproxy.net
# 代理端口号
port: 7878
# 代理用户名
username: swiftproxy_u
# 代理密码
password: swiftproxy_p
--- #--------------------- Spring AI 配置----------------------
spring:
ai:
openai:
image:
enabled: true
base-url: https://api.siliconflow.cn
api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
options:
model: Kwai-Kolors/Kolors
response_format: URL
api-key: sk-rrrupturhdofbiqzjutduuiceecpvfqlnvmgcyiaipbdikoi
base-url: https://api.siliconflow.cn
chat:
options:
model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

View File

@ -1,47 +0,0 @@
mode:
# 单机模式
type: Standalone
# 元数据持久化
repository:
# 数据库持久化
type: JDBC
# 数据源配置
dataSources:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/novel_plus?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=Asia/Shanghai
username: root
password: test123456
# 规则配置
rules:
- !SINGLE
tables:
- "*.*"
- !SHARDING
tables: # 数据分片规则配置
book_content:
# 分库策略缺省表示使用默认分库策略
actualDataNodes: ds_${1}.book_content${0..9}
# 分表策略
tableStrategy:
standard:
# 分片列名称
shardingColumn: index_id
# 分片算法名称
shardingAlgorithmName: bookContentSharding
shardingAlgorithms:
bookContentSharding:
# 行表达式分片算法使用 Groovy 的表达式提供对 SQL 语句中的 = IN 的分片操作支持
type: INLINE
props:
# 分片算法的行表达式
algorithm-expression: book_content${index_id % 10}
props:
# 是否在日志中打印 SQL
sql-show: true

View File

@ -12,20 +12,11 @@ import com.java2nb.novel.entity.AuthorIncomeDetail;
import com.java2nb.novel.entity.Book; import com.java2nb.novel.entity.Book;
import com.java2nb.novel.service.AuthorService; import com.java2nb.novel.service.AuthorService;
import com.java2nb.novel.service.BookService; import com.java2nb.novel.service.BookService;
import io.swagger.v3.oas.annotations.Operation;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Flux;
import javax.servlet.http.HttpServletRequest;
import java.util.Date; import java.util.Date;
/** /**
@ -35,67 +26,63 @@ import java.util.Date;
@RestController @RestController
@Slf4j @Slf4j
@RequiredArgsConstructor @RequiredArgsConstructor
public class AuthorController extends BaseController { public class AuthorController extends BaseController{
private final AuthorService authorService; private final AuthorService authorService;
private final BookService bookService; private final BookService bookService;
private final ChatClient chatClient;
private final OpenAiChatModel chatModel;
/** /**
* 校验笔名是否存在 * 校验笔名是否存在
*/ * */
@GetMapping("checkPenName") @GetMapping("checkPenName")
public RestResult<Boolean> checkPenName(String penName) { public RestResult<Boolean> checkPenName(String penName){
return RestResult.ok(authorService.checkPenName(penName)); return RestResult.ok(authorService.checkPenName(penName));
} }
/** /**
* 作家发布小说分页列表查询 * 作家发布小说分页列表查询
*/ * */
@GetMapping("listBookByPage") @GetMapping("listBookByPage")
public RestResult<PageBean<Book>> listBookByPage(@RequestParam(value = "curr", defaultValue = "1") int page, public RestResult<PageBean<Book>> listBookByPage(@RequestParam(value = "curr", defaultValue = "1") int page, @RequestParam(value = "limit", defaultValue = "10") int pageSize , HttpServletRequest request){
@RequestParam(value = "limit", defaultValue = "10") int pageSize, HttpServletRequest request) {
return RestResult.ok(bookService.listBookPageByUserId(getUserDetails(request).getId(), page, pageSize)); return RestResult.ok(bookService.listBookPageByUserId(getUserDetails(request).getId(),page,pageSize));
} }
/** /**
* 发布小说 * 发布小说
*/ * */
@PostMapping("addBook") @PostMapping("addBook")
public RestResult<Void> addBook(@RequestParam("bookDesc") String bookDesc, Book book, HttpServletRequest request) { public RestResult<Void> addBook(@RequestParam("bookDesc") String bookDesc,Book book,HttpServletRequest request){
Author author = checkAuthor(request); Author author = checkAuthor(request);
//bookDesc不能使用book对象来接收否则会自动去掉前面的空格 //bookDesc不能使用book对象来接收否则会自动去掉前面的空格
book.setBookDesc(bookDesc book.setBookDesc(bookDesc
.replaceAll("\\n", "<br>") .replaceAll("\\n","<br>")
.replaceAll("\\s", "&nbsp;")); .replaceAll("\\s","&nbsp;"));
//发布小说 //发布小说
bookService.addBook(book, author.getId(), author.getPenName()); bookService.addBook(book,author.getId(),author.getPenName());
return RestResult.ok(); return RestResult.ok();
} }
/** /**
* 更新小说状态,上架或下架 * 更新小说状态,上架或下架
*/ * */
@PostMapping("updateBookStatus") @PostMapping("updateBookStatus")
public RestResult<Void> updateBookStatus(Long bookId, Byte status, HttpServletRequest request) { public RestResult<Void> updateBookStatus(Long bookId,Byte status,HttpServletRequest request){
Author author = checkAuthor(request); Author author = checkAuthor(request);
//更新小说状态,上架或下架 //更新小说状态,上架或下架
bookService.updateBookStatus(bookId, status, author.getId()); bookService.updateBookStatus(bookId,status,author.getId());
return RestResult.ok(); return RestResult.ok();
} }
/** /**
* 删除章节 * 删除章节
*/ */
@ -125,18 +112,19 @@ public class AuthorController extends BaseController {
} }
/** /**
* 发布章节内容 * 发布章节内容
*/ */
@PostMapping("addBookContent") @PostMapping("addBookContent")
public RestResult<Void> addBookContent(Long bookId, String indexName, String content, Byte isVip, public RestResult<Void> addBookContent(Long bookId, String indexName, String content,Byte isVip, HttpServletRequest request) {
HttpServletRequest request) {
Author author = checkAuthor(request); Author author = checkAuthor(request);
content = content.replaceAll("\\n", "<br>") content = content.replaceAll("\\n", "<br>")
.replaceAll("\\s", "&nbsp;"); .replaceAll("\\s", "&nbsp;");
//发布章节内容 //发布章节内容
bookService.addBookContent(bookId, indexName, content, isVip, author.getId()); bookService.addBookContent(bookId, indexName, content,isVip, author.getId());
return RestResult.ok(); return RestResult.ok();
} }
@ -161,8 +149,7 @@ public class AuthorController extends BaseController {
* 更新章节内容 * 更新章节内容
*/ */
@PostMapping("updateBookContent") @PostMapping("updateBookContent")
public RestResult<Void> updateBookContent(Long indexId, String indexName, String content, public RestResult<Void> updateBookContent(Long indexId, String indexName, String content, HttpServletRequest request) {
HttpServletRequest request) {
Author author = checkAuthor(request); Author author = checkAuthor(request);
content = content.replaceAll("\\n", "<br>") content = content.replaceAll("\\n", "<br>")
@ -177,44 +164,38 @@ public class AuthorController extends BaseController {
* 修改小说封面 * 修改小说封面
*/ */
@PostMapping("updateBookPic") @PostMapping("updateBookPic")
public RestResult<Void> updateBookPic(@RequestParam("bookId") Long bookId, @RequestParam("bookPic") String bookPic, public RestResult<Void> updateBookPic(@RequestParam("bookId") Long bookId,@RequestParam("bookPic") String bookPic,HttpServletRequest request) {
HttpServletRequest request) {
Author author = checkAuthor(request); Author author = checkAuthor(request);
bookService.updateBookPic(bookId, bookPic, author.getId()); bookService.updateBookPic(bookId,bookPic, author.getId());
return RestResult.ok(); return RestResult.ok();
} }
/** /**
* 作家日收入统计数据分页列表查询 * 作家日收入统计数据分页列表查询
*/ * */
@GetMapping("listIncomeDailyByPage") @GetMapping("listIncomeDailyByPage")
public RestResult<PageBean<AuthorIncomeDetail>> listIncomeDailyByPage( public RestResult<PageBean<AuthorIncomeDetail>> listIncomeDailyByPage(@RequestParam(value = "curr", defaultValue = "1") int page,
@RequestParam(value = "curr", defaultValue = "1") int page, @RequestParam(value = "limit", defaultValue = "10") int pageSize ,
@RequestParam(value = "limit", defaultValue = "10") int pageSize,
@RequestParam(value = "bookId", defaultValue = "0") Long bookId, @RequestParam(value = "bookId", defaultValue = "0") Long bookId,
@RequestParam(value = "startTime", defaultValue = "2020-05-01") Date startTime, @RequestParam(value = "startTime",defaultValue = "2020-05-01") Date startTime,
@RequestParam(value = "endTime", defaultValue = "2030-01-01") Date endTime, @RequestParam(value = "endTime",defaultValue = "2030-01-01") Date endTime,
HttpServletRequest request) { HttpServletRequest request){
return RestResult.ok( return RestResult.ok(authorService.listIncomeDailyByPage(page,pageSize,getUserDetails(request).getId(),bookId,startTime,endTime));
authorService.listIncomeDailyByPage(page, pageSize, getUserDetails(request).getId(), bookId, startTime,
endTime));
} }
/** /**
* 作家月收入统计数据分页列表查询 * 作家月收入统计数据分页列表查询
*/ * */
@GetMapping("listIncomeMonthByPage") @GetMapping("listIncomeMonthByPage")
public RestResult<PageBean<AuthorIncome>> listIncomeMonthByPage( public RestResult<PageBean<AuthorIncome>> listIncomeMonthByPage(@RequestParam(value = "curr", defaultValue = "1") int page,
@RequestParam(value = "curr", defaultValue = "1") int page, @RequestParam(value = "limit", defaultValue = "10") int pageSize ,
@RequestParam(value = "limit", defaultValue = "10") int pageSize,
@RequestParam(value = "bookId", defaultValue = "0") Long bookId, @RequestParam(value = "bookId", defaultValue = "0") Long bookId,
HttpServletRequest request) { HttpServletRequest request){
return RestResult.ok( return RestResult.ok(authorService.listIncomeMonthByPage(page,pageSize,getUserDetails(request).getId(),bookId));
authorService.listIncomeMonthByPage(page, pageSize, getUserDetails(request).getId(), bookId));
} }
private Author checkAuthor(HttpServletRequest request) { private Author checkAuthor(HttpServletRequest request) {
@ -233,113 +214,13 @@ public class AuthorController extends BaseController {
throw new BusinessException(ResponseStatus.AUTHOR_STATUS_FORBIDDEN); throw new BusinessException(ResponseStatus.AUTHOR_STATUS_FORBIDDEN);
} }
return author; return author;
} }
/**
* 查询AI生成图片
*/
@GetMapping("queryAiGenPic")
public RestResult<String> queryAiGenPic(@RequestParam("bookId") Long bookId) {
return RestResult.ok(bookService.queryAiGenPic(bookId));
}
/**
* AI扩写
*/
@PostMapping("ai/expand")
public RestResult<String> expandText(@RequestParam("text") String text, @RequestParam("ratio") Double ratio) {
String prompt = "请将以下文本扩写为原长度的" + ratio / 100 + "倍:" + text;
return RestResult.ok(chatClient.prompt()
.user(prompt)
.call()
.content());
}
/**
* AI缩写
*/
@PostMapping("ai/condense")
public RestResult<String> condenseText(@RequestParam("text") String text, @RequestParam("ratio") Integer ratio) {
String prompt = "请将以下文本缩写为原长度的" + 100 / ratio + "分之一:" + text;
return RestResult.ok(chatClient.prompt()
.user(prompt)
.call()
.content());
}
/**
* AI续写
*/
@PostMapping("ai/continue")
public RestResult<String> continueText(@RequestParam("text") String text, @RequestParam("length") Integer length) {
String prompt = "请续写以下文本,续写长度约为" + length + "字:" + text;
return RestResult.ok(chatClient.prompt()
.user(prompt)
.call()
.content());
}
/**
* AI润色
*/
@PostMapping("ai/polish")
public RestResult<String> polishText(@RequestParam("text") String text) {
String prompt = "请润色优化以下文本,保持原意:" + text;
return RestResult.ok(chatClient.prompt()
.user(prompt)
.call()
.content());
}
/**
* AI扩写
*/
@GetMapping(value = "ai/stream/expand", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamExpandText(@RequestParam("text") String text, @RequestParam("ratio") Double ratio) {
String prompt = "请将以下文本扩写为原长度的" + ratio / 100 + "倍:" + text;
return chatClient.prompt()
.user(prompt)
.stream()
.content();
}
/**
* AI缩写
*/
@GetMapping(value = "ai/stream/condense", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamCondenseText(@RequestParam("text") String text, @RequestParam("ratio") Integer ratio) {
String prompt = "请将以下文本缩写为原长度的" + 100 / ratio + "分之一:" + text;
return chatClient.prompt()
.user(prompt)
.stream()
.content();
}
/**
* AI续写
*/
@GetMapping(value = "ai/stream/continue", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamContinueText(@RequestParam("text") String text, @RequestParam("length") Integer length) {
String prompt = "请续写以下文本,续写长度约为" + length + "字:" + text;
return chatClient.prompt()
.user(prompt)
.stream()
.content();
}
/**
* AI润色
*/
@GetMapping(value = "/ai/stream/polish", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamPolishText(@RequestParam("text") String text) {
String prompt = "请润色优化以下文本,保持原意:" + text;
return chatClient.prompt()
.user(prompt)
.stream()
.content();
}
} }

View File

@ -3,10 +3,11 @@ package com.java2nb.novel.controller;
import com.java2nb.novel.core.bean.UserDetails; import com.java2nb.novel.core.bean.UserDetails;
import com.java2nb.novel.core.utils.CookieUtil; import com.java2nb.novel.core.utils.CookieUtil;
import com.java2nb.novel.core.utils.JwtTokenUtil; import com.java2nb.novel.core.utils.JwtTokenUtil;
import jakarta.servlet.http.HttpServletRequest;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServletRequest;
/** /**
* @author 11797 * @author 11797

View File

@ -15,11 +15,11 @@ import com.java2nb.novel.vo.BookVO;
import io.github.xxyopen.model.page.PageBean; import io.github.xxyopen.model.page.PageBean;
import io.github.xxyopen.model.page.builder.pagehelper.PageBuilder; import io.github.xxyopen.model.page.builder.pagehelper.PageBuilder;
import io.github.xxyopen.model.resp.RestResult; import io.github.xxyopen.model.resp.RestResult;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import javax.servlet.http.HttpServletRequest;
import java.util.HashMap; import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;

View File

@ -10,18 +10,17 @@ import com.java2nb.novel.core.utils.RandomValidateCodeUtil;
import io.github.xxyopen.model.resp.RestResult; import io.github.xxyopen.model.resp.RestResult;
import io.github.xxyopen.util.UUIDUtil; import io.github.xxyopen.util.UUIDUtil;
import io.github.xxyopen.web.exception.BusinessException; import io.github.xxyopen.web.exception.BusinessException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.http.client.utils.DateUtils; import org.apache.http.client.utils.DateUtils;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Controller; import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.MultipartFile;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.File; import java.io.File;
import java.util.Date; import java.util.Date;
@ -62,20 +61,11 @@ public class FileController {
/** /**
* 图片上传 * 图片上传
* *
* - 当使用 `$.ajax`发起异步请求时 设置`dataType: "json"`会在请求头中自动添加`Accept: application/json`表示客户端期望服务器返回 * @return
* `JSON`格式的数据
* - 当使用 `$.ajaxFileUpload` 上传文件时它的行为与`$.ajax`不同不会自动修改`Accept`请求头即使设置了`dataType: "json"`
* `$.ajaxFileUpload`也不会在请求头中添加`Accept: application/json`
*
* Spring Boot 默认返回`JSON`格式的响应但它支持内容协商它会根据客户端请求的`Accept`头来决定返回的响应格式
* 如果浏览器发送的请求中`Accept`头包含`application/xml`并且 Spring Boot 支持`XML`格式响应的话Spring Boot 会返回`XML`格式的响应
* Spring Boot 默认不支持`XML`格式的响应当升级`Sharding-JDBC `版本后自动引入了`jackson-dataformat-xml`依赖才开始支持`XML`格式的响应
* 由于`$.ajaxFileUpload`上传文件的默认`Accept`头包含`application/xml`所以需要在后端上传文件接口处明确指定返回的数据类型为`application/json`
*
*/ */
@SneakyThrows @SneakyThrows
@ResponseBody @ResponseBody
@PostMapping(value = "/picUpload", produces = MediaType.APPLICATION_JSON_VALUE) @PostMapping("/picUpload")
RestResult<String> upload(@RequestParam("file") MultipartFile file) { RestResult<String> upload(@RequestParam("file") MultipartFile file) {
Date currentDate = new Date(); Date currentDate = new Date();
String savePath = String savePath =

View File

@ -12,8 +12,6 @@ import com.java2nb.novel.core.bean.UserDetails;
import com.java2nb.novel.core.config.AlipayProperties; import com.java2nb.novel.core.config.AlipayProperties;
import com.java2nb.novel.core.utils.ThreadLocalUtil; import com.java2nb.novel.core.utils.ThreadLocalUtil;
import com.java2nb.novel.service.OrderService; import com.java2nb.novel.service.OrderService;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@ -21,6 +19,8 @@ import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.PrintWriter; import java.io.PrintWriter;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.util.HashMap; import java.util.HashMap;

View File

@ -13,13 +13,13 @@ import com.java2nb.novel.service.UserService;
import io.github.xxyopen.model.resp.RestResult; import io.github.xxyopen.model.resp.RestResult;
import io.github.xxyopen.web.valid.AddGroup; import io.github.xxyopen.web.valid.AddGroup;
import io.github.xxyopen.web.valid.UpdateGroup; import io.github.xxyopen.web.valid.UpdateGroup;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.springframework.validation.annotation.Validated; import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import javax.servlet.http.HttpServletRequest;
import java.util.HashMap; import java.util.HashMap;
import java.util.Map; import java.util.Map;

View File

@ -8,7 +8,6 @@ import com.java2nb.novel.service.*;
import com.java2nb.novel.vo.BookCommentVO; import com.java2nb.novel.vo.BookCommentVO;
import com.java2nb.novel.vo.BookSettingVO; import com.java2nb.novel.vo.BookSettingVO;
import io.github.xxyopen.model.page.PageBean; import io.github.xxyopen.model.page.PageBean;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@ -18,6 +17,7 @@ import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import javax.servlet.http.HttpServletRequest;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletableFuture;

View File

@ -1,38 +0,0 @@
package com.java2nb.novel.core.config;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestClient;
/**
* Ai 相关配置
*
* @author xiongxiaoyang
* @date 2025/2/19
*/
@Configuration
@Slf4j
public class AiConfig {
/**
* 配置自定义的 RestClientBuilder 对象
*/
@Bean
public RestClient.Builder restClientBuilder() {
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
// 连接超时时间
factory.setConnectTimeout(5000);
// 读取超时时间
factory.setReadTimeout(60000);
return RestClient.builder().requestFactory(factory);
}
@Bean
public ChatClient chatClient(ChatClient.Builder chatClientBuilder) {
return chatClientBuilder.build();
}
}

View File

@ -2,13 +2,13 @@ package com.java2nb.novel.core.config;
import com.java2nb.novel.core.filter.NovelFilter; import com.java2nb.novel.core.filter.NovelFilter;
import com.java2nb.novel.core.filter.XssFilter; import com.java2nb.novel.core.filter.XssFilter;
import jakarta.servlet.DispatcherType;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.servlet.FilterRegistrationBean; import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import javax.servlet.DispatcherType;
import java.util.HashMap; import java.util.HashMap;
import java.util.Map; import java.util.Map;

View File

@ -3,13 +3,13 @@ package com.java2nb.novel.core.config;
import com.java2nb.novel.core.converter.DateConverter; import com.java2nb.novel.core.converter.DateConverter;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.format.FormatterRegistry; import org.springframework.format.FormatterRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer; import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
/** /**
* @author xiongxiaoyang * @author xiongxiaoyang
*/ */
@Configuration @Configuration
public class WebMvcConfig implements WebMvcConfigurer { public class WebMvcConfig extends WebMvcConfigurerAdapter {
@Override @Override
public void addFormatters(FormatterRegistry registry) { public void addFormatters(FormatterRegistry registry) {
registry.addConverter(new DateConverter()); registry.addConverter(new DateConverter());

View File

@ -4,14 +4,13 @@ import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService; import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.utils.*; import com.java2nb.novel.core.utils.*;
import io.github.xxyopen.util.UUIDUtil; import io.github.xxyopen.util.UUIDUtil;
import jakarta.servlet.*; import io.github.xxyopen.web.util.SpringUtil;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import java.io.FileInputStream; import javax.servlet.*;
import java.io.IOException; import javax.servlet.http.Cookie;
import java.io.InputStream; import javax.servlet.http.HttpServletRequest;
import java.io.OutputStream; import javax.servlet.http.HttpServletResponse;
import java.io.*;
/** /**
* 项目核心过滤器 * 项目核心过滤器

View File

@ -1,11 +1,11 @@
package com.java2nb.novel.core.filter; package com.java2nb.novel.core.filter;
import com.java2nb.novel.core.wrapper.XssHttpServletRequestWrapper; import com.java2nb.novel.core.wrapper.XssHttpServletRequestWrapper;
import jakarta.servlet.*;
import jakarta.servlet.http.HttpServletRequest;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays; import java.util.Arrays;
import java.util.List; import java.util.List;

View File

@ -3,29 +3,31 @@ package com.java2nb.novel.core.listener;
import com.java2nb.novel.core.config.WebsiteProperties; import com.java2nb.novel.core.config.WebsiteProperties;
import com.java2nb.novel.entity.WebsiteInfo; import com.java2nb.novel.entity.WebsiteInfo;
import com.java2nb.novel.mapper.WebsiteInfoMapper; import com.java2nb.novel.mapper.WebsiteInfoMapper;
import jakarta.servlet.ServletContext;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.web.servlet.ServletContextInitializer;
import org.springframework.stereotype.Component;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
/** /**
* 启动监听器 * 启动监听器
* *
* @author xiongxiaoyang * @author xiongxiaoyang
*/ */
@Component @WebListener
@Slf4j @Slf4j
@RequiredArgsConstructor @RequiredArgsConstructor
public class StarterListener implements ServletContextInitializer { public class StarterListener implements ServletContextListener {
private final WebsiteProperties websiteProperties; private final WebsiteProperties websiteProperties;
private final WebsiteInfoMapper websiteInfoMapper; private final WebsiteInfoMapper websiteInfoMapper;
@Override @Override
public void onStartup(ServletContext servletContext) { public void contextInitialized(ServletContextEvent sce) {
servletContext.setAttribute("website", websiteInfoMapper.selectByPrimaryKey(1L).orElse(new WebsiteInfo() {{ sce.getServletContext()
.setAttribute("website", websiteInfoMapper.selectByPrimaryKey(1L).orElse(new WebsiteInfo() {{
setName(websiteProperties.getName()); setName(websiteProperties.getName());
setDomain(websiteProperties.getDomain()); setDomain(websiteProperties.getDomain());
setKeyword(websiteProperties.getKeyword()); setKeyword(websiteProperties.getKeyword());

View File

@ -1,7 +1,6 @@
package com.java2nb.novel.core.utils; package com.java2nb.novel.core.utils;
import jakarta.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequest;
import java.util.regex.Pattern; import java.util.regex.Pattern;
public class BrowserUtil { public class BrowserUtil {

View File

@ -1,15 +1,15 @@
package com.java2nb.novel.core.utils; package com.java2nb.novel.core.utils;
import jakarta.servlet.http.Cookie; import javax.servlet.http.Cookie;
import jakarta.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import javax.servlet.http.HttpServletResponse;
/** /**
* @author Administrator * @author Administrator
*/ */
public class CookieUtil { public class CookieUtil {
public static String getCookie(HttpServletRequest request, String key){ public static String getCookie(HttpServletRequest request,String key){
Cookie[] cookies = request.getCookies(); Cookie[] cookies = request.getCookies();
if(cookies != null) { if(cookies != null) {
for (Cookie cookie : cookies) { for (Cookie cookie : cookies) {

View File

@ -2,6 +2,7 @@ package com.java2nb.novel.core.utils;
import com.java2nb.novel.core.cache.CacheKey; import com.java2nb.novel.core.cache.CacheKey;
import com.java2nb.novel.core.cache.CacheService; import com.java2nb.novel.core.cache.CacheService;
import io.github.xxyopen.web.util.SpringUtil;
/** /**
* 模板操作工具类 * 模板操作工具类

View File

@ -1,7 +1,11 @@
package com.java2nb.novel.core.wrapper; package com.java2nb.novel.core.wrapper;
import jakarta.servlet.http.HttpServletRequest; import org.apache.commons.lang3.StringUtils;
import jakarta.servlet.http.HttpServletRequestWrapper;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.util.Arrays;
import java.util.List;
/** /**
* XSS过滤处理 * XSS过滤处理

View File

@ -290,9 +290,4 @@ public interface BookService {
* @param authorId * @param authorId
*/ */
void updateBookPic(Long bookId, String bookPic, Long authorId); void updateBookPic(Long bookId, String bookPic, Long authorId);
/**
* 查询AI生成图片
*/
String queryAiGenPic(Long bookId);
} }

View File

@ -7,7 +7,6 @@ import com.java2nb.novel.core.cache.CacheService;
import com.java2nb.novel.core.config.BookPriceProperties; import com.java2nb.novel.core.config.BookPriceProperties;
import com.java2nb.novel.core.enums.ResponseStatus; import com.java2nb.novel.core.enums.ResponseStatus;
import com.java2nb.novel.core.utils.Constants; import com.java2nb.novel.core.utils.Constants;
import com.java2nb.novel.core.utils.FileUtil;
import com.java2nb.novel.core.utils.StringUtil; import com.java2nb.novel.core.utils.StringUtil;
import com.java2nb.novel.entity.Book; import com.java2nb.novel.entity.Book;
import com.java2nb.novel.entity.*; import com.java2nb.novel.entity.*;
@ -28,34 +27,28 @@ import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows; import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.DateUtils;
import org.mybatis.dynamic.sql.SortSpecification; import org.mybatis.dynamic.sql.SortSpecification;
import org.mybatis.dynamic.sql.render.RenderingStrategies; import org.mybatis.dynamic.sql.render.RenderingStrategies;
import org.mybatis.dynamic.sql.select.QueryExpressionDSL;
import org.mybatis.dynamic.sql.select.render.SelectStatementProvider; import org.mybatis.dynamic.sql.select.render.SelectStatementProvider;
import org.springframework.ai.image.Image;
import org.springframework.ai.image.ImagePrompt;
import org.springframework.ai.image.ImageResponse;
import org.springframework.ai.openai.OpenAiImageModel;
import org.springframework.ai.openai.OpenAiImageOptions;
import org.springframework.beans.BeanUtils; import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import tk.mybatis.orderbyhelper.OrderByHelper;
import java.math.BigDecimal; import java.math.BigDecimal;
import java.math.RoundingMode; import java.math.RoundingMode;
import java.util.*; import java.util.ArrayList;
import java.util.concurrent.ThreadPoolExecutor; import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import static com.java2nb.novel.mapper.BookCategoryDynamicSqlSupport.bookCategory; import static com.java2nb.novel.mapper.BookCategoryDynamicSqlSupport.bookCategory;
import static com.java2nb.novel.mapper.BookCategoryDynamicSqlSupport.sort;
import static com.java2nb.novel.mapper.BookCommentDynamicSqlSupport.bookComment; import static com.java2nb.novel.mapper.BookCommentDynamicSqlSupport.bookComment;
import static com.java2nb.novel.mapper.BookContentDynamicSqlSupport.bookContent; import static com.java2nb.novel.mapper.BookContentDynamicSqlSupport.bookContent;
import static com.java2nb.novel.mapper.BookContentDynamicSqlSupport.content; import static com.java2nb.novel.mapper.BookContentDynamicSqlSupport.content;
import static com.java2nb.novel.mapper.BookDynamicSqlSupport.*; import static com.java2nb.novel.mapper.BookDynamicSqlSupport.*;
import static com.java2nb.novel.mapper.BookDynamicSqlSupport.book;
import static com.java2nb.novel.mapper.BookIndexDynamicSqlSupport.bookIndex; import static com.java2nb.novel.mapper.BookIndexDynamicSqlSupport.bookIndex;
import static com.java2nb.novel.mapper.BookSettingDynamicSqlSupport.bookSetting; import static com.java2nb.novel.mapper.BookSettingDynamicSqlSupport.bookSetting;
import static org.mybatis.dynamic.sql.SqlBuilder.*; import static org.mybatis.dynamic.sql.SqlBuilder.*;
@ -97,10 +90,6 @@ public class BookServiceImpl implements BookService {
private final BookPriceProperties bookPriceConfig; private final BookPriceProperties bookPriceConfig;
private final OpenAiImageModel openAiImageModel;
private final ThreadPoolExecutor threadPoolExecutor;
private final IdWorker idWorker = IdWorker.INSTANCE; private final IdWorker idWorker = IdWorker.INSTANCE;
@ -116,7 +105,7 @@ public class BookServiceImpl implements BookService {
} }
result = new ObjectMapper().writeValueAsString( result = new ObjectMapper().writeValueAsString(
list.stream().collect(Collectors.groupingBy(BookSettingVO::getType))); list.stream().collect(Collectors.groupingBy(BookSettingVO::getType)));
cacheService.set(CacheKey.INDEX_BOOK_SETTINGS_KEY, result, 3600 * 24); cacheService.set(CacheKey.INDEX_BOOK_SETTINGS_KEY, result);
} }
return new ObjectMapper().readValue(result, Map.class); return new ObjectMapper().readValue(result, Map.class);
} }
@ -212,6 +201,9 @@ public class BookServiceImpl implements BookService {
PageHelper.startPage(page, pageSize); PageHelper.startPage(page, pageSize);
if (StringUtils.isNotBlank(params.getSort())) {
OrderByHelper.orderBy(params.getSort() + " desc");
}
return PageBuilder.build(bookMapper.searchByPage(params)); return PageBuilder.build(bookMapper.searchByPage(params));
} }
@ -239,22 +231,23 @@ public class BookServiceImpl implements BookService {
@Override @Override
public List<BookIndex> queryIndexList(Long bookId, String orderBy, Integer page, Integer pageSize) { public List<BookIndex> queryIndexList(Long bookId, String orderBy, Integer page, Integer pageSize) {
if (StringUtils.isNotBlank(orderBy)) {
OrderByHelper.orderBy(orderBy);
}
if (page != null && pageSize != null) { if (page != null && pageSize != null) {
PageHelper.startPage(page, pageSize); PageHelper.startPage(page, pageSize);
} }
QueryExpressionDSL<org.mybatis.dynamic.sql.select.SelectModel>.QueryExpressionWhereBuilder where = select(
BookIndexDynamicSqlSupport.id, SelectStatementProvider selectStatement = select(BookIndexDynamicSqlSupport.id,
BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum, BookIndexDynamicSqlSupport.bookId, BookIndexDynamicSqlSupport.indexNum,
BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.updateTime, BookIndexDynamicSqlSupport.indexName, BookIndexDynamicSqlSupport.updateTime,
BookIndexDynamicSqlSupport.isVip) BookIndexDynamicSqlSupport.isVip)
.from(bookIndex) .from(bookIndex)
.where(BookIndexDynamicSqlSupport.bookId, isEqualTo(bookId)); .where(BookIndexDynamicSqlSupport.bookId, isEqualTo(bookId))
if ("index_num desc".equals(orderBy)) {
where.orderBy(BookIndexDynamicSqlSupport.indexNum.descending());
}
return bookIndexMapper.selectMany(where
.build() .build()
.render(RenderingStrategies.MYBATIS3)); .render(RenderingStrategies.MYBATIS3);
return bookIndexMapper.selectMany(selectStatement);
} }
@ -391,6 +384,7 @@ public class BookServiceImpl implements BookService {
@Override @Override
public PageBean<BookCommentVO> listCommentByPage(Long userId, Long bookId, int page, int pageSize) { public PageBean<BookCommentVO> listCommentByPage(Long userId, Long bookId, int page, int pageSize) {
PageHelper.startPage(page, pageSize); PageHelper.startPage(page, pageSize);
OrderByHelper.orderBy("t1.create_time desc");
return PageBuilder.build(bookCommentMapper.listCommentByPage(userId, bookId)); return PageBuilder.build(bookCommentMapper.listCommentByPage(userId, bookId));
} }
@ -516,7 +510,6 @@ public class BookServiceImpl implements BookService {
@Override @Override
public void addBook(Book book, Long authorId, String penName) { public void addBook(Book book, Long authorId, String penName) {
book.setId(IdWorker.INSTANCE.nextId());
//判断小说名是否存在 //判断小说名是否存在
if (queryIdByNameAndAuthor(book.getBookName(), penName) != null) { if (queryIdByNameAndAuthor(book.getBookName(), penName) != null) {
//该作者发布过此书名的小说 //该作者发布过此书名的小说
@ -531,37 +524,7 @@ public class BookServiceImpl implements BookService {
book.setCreateTime(new Date()); book.setCreateTime(new Date());
book.setUpdateTime(book.getCreateTime()); book.setUpdateTime(book.getCreateTime());
bookMapper.insertSelective(book); bookMapper.insertSelective(book);
if (Objects.isNull(book.getPicUrl()) || !book.getPicUrl().startsWith(Constants.LOCAL_PIC_PREFIX)) {
// 用户没有上传封面图片AI自动生成封面图片
threadPoolExecutor.execute(() -> {
String prompt = String.format("生成一本小说的封面图片,图片中间显示书名《%s》书名下方显示作者“%s 著”。",
book.getBookName(), book.getAuthorName());
log.debug("prompt:{}", prompt);
ImageResponse response = openAiImageModel.call(
new ImagePrompt(prompt,
OpenAiImageOptions.builder()
.quality("hd")
.height(800)
.width(600).build())
);
Image output = response.getResult().getOutput();
Date currentDate = new Date();
String picUrl = Constants.LOCAL_PIC_PREFIX +
"aiGen/" + DateUtils.formatDate(currentDate, "yyyy") + "/" +
DateUtils.formatDate(currentDate, "MM") + "/" +
DateUtils.formatDate(currentDate, "dd") + "/" + book.getId() + ".png";
FileUtil.downloadFile(output.getUrl(), picSavePath + picUrl);
bookMapper.update(update(BookDynamicSqlSupport.book)
.set(BookDynamicSqlSupport.picUrl)
.equalTo(picUrl)
.set(updateTime)
.equalTo(currentDate)
.where(id, isEqualTo(book.getId()))
.build()
.render(RenderingStrategies.MYBATIS3));
cacheService.set(CacheKey.AI_GEN_PIC + book.getId(), picUrl, 60 * 60);
});
}
} }
@Override @Override
@ -883,10 +846,5 @@ public class BookServiceImpl implements BookService {
.render(RenderingStrategies.MYBATIS3)); .render(RenderingStrategies.MYBATIS3));
} }
@Override
public String queryAiGenPic(Long bookId) {
return cacheService.get(CacheKey.AI_GEN_PIC + bookId);
}
} }

View File

@ -6,6 +6,7 @@ import com.java2nb.novel.core.serialize.CommentUserNameSerialize;
import com.java2nb.novel.entity.BookComment; import com.java2nb.novel.entity.BookComment;
import lombok.Data; import lombok.Data;
import javax.annotation.Generated;
import java.util.Date; import java.util.Date;
/** /**

View File

@ -1,6 +1,6 @@
spring: spring:
config: profiles:
import: classpath:application-common-dev.yml include: [ common-dev ]
pic: pic:

View File

@ -1,15 +1,14 @@
spring: spring:
config: profiles:
import: classpath:application-common-prod.yml include: [ common-prod ]
#静态文件路径配置 #静态文件路径配置
resources:
static-locations: file:${user.dir}/templates/${templates.name}/static/
#thymeleaf模版路径配置 #thymeleaf模版路径配置
thymeleaf: thymeleaf:
prefix: file:${user.dir}/templates/${templates.name}/html/ prefix: file:${user.dir}/templates/${templates.name}/html/
suffix: .html suffix: .html
web:
resources:
static-locations: file:${user.dir}/templates/${templates.name}/static/
#模版配置 #模版配置
templates: templates:

View File

@ -44,21 +44,9 @@ book:
value: 5 value: 5
--- #--------------------- Spring AI 配置----------------------
spring:
ai:
openai:
image:
enabled: true
base-url: https://api.siliconflow.cn
api-key: sk-jjtixmivxaccndqgkqfkbgkzvmbctdxogcrfbjzfttbouitt
options:
model: Kwai-Kolors/Kolors
response_format: URL
api-key: sk-jjtixmivxaccndqgkqfkbgkzvmbctdxogcrfbjzfttbouitt
base-url: https://api.siliconflow.cn
chat:
options:
model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

View File

Before

Width:  |  Height:  |  Size: 162 KiB

After

Width:  |  Height:  |  Size: 162 KiB

View File

@ -12,6 +12,9 @@
<!-- ConsoleAppender把日志输出到控制台 --> <!-- ConsoleAppender把日志输出到控制台 -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder> <encoder>
<!--
<pattern>%d %p (%file:%line\)- %m%n</pattern>
-->
<pattern>${CONSOLE_LOG_PATTERN}</pattern> <pattern>${CONSOLE_LOG_PATTERN}</pattern>
<!-- 控制台也要使用UTF-8不要使用GBK否则会中文乱码 --> <!-- 控制台也要使用UTF-8不要使用GBK否则会中文乱码 -->
<charset>UTF-8</charset> <charset>UTF-8</charset>
@ -19,40 +22,40 @@
</appender> </appender>
<!-- RollingFileAppender滚动记录文件先将日志记录到指定文件当符合某个条件时将日志记录到其他文件 --> <!-- RollingFileAppender滚动记录文件先将日志记录到指定文件当符合某个条件时将日志记录到其他文件 -->
<!-- 以下的大概意思是1.先按日期存日志日期变了将前一天的日志文件名重命名为XXX%日期%索引新的日志仍然是novel-front.log --> <!-- 以下的大概意思是1.先按日期存日志日期变了将前一天的日志文件名重命名为XXX%日期%索引新的日志仍然是demo.log -->
<!-- 2.如果日期没有发生变化但是当前日志的文件大小超过10MB时对当前日志进行分割 重命名 --> <!-- 2.如果日期没有发生变化但是当前日志的文件大小超过1KB时对当前日志进行分割 重命名 -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 设置日志文件路径 -->
<file>logs/novel-front.log</file> <File>logs/novel-front.log</File>
<!-- rollingPolicy:当发生滚动时决定 RollingFileAppender 的行为涉及文件移动和重命名 --> <!-- rollingPolicy:当发生滚动时决定 RollingFileAppender 的行为涉及文件移动和重命名 -->
<!-- SizeAndTimeBasedRollingPolicy基于时间和文件大小的滚动策略 --> <!-- TimeBasedRollingPolicy 最常用的滚动策略它根据时间来制定滚动策略既负责滚动也负责出发滚动 -->
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 活动文件的名字会根据fileNamePattern的值每隔一段时间改变一次 --> <!-- 活动文件的名字会根据fileNamePattern的值每隔一段时间改变一次 -->
<!-- 文件名logs/debug.2023-10-01.0.log --> <!-- 文件名logs/demo.2017-12-05.0.log -->
<fileNamePattern>logs/debug.%d{yyyy-MM-dd}.%i.log</fileNamePattern> <fileNamePattern>logs/debug.%d.%i.log</fileNamePattern>
<!-- 每产生一个日志文件该日志文件的保存期限为30天 --> <!-- 每产生一个日志文件该日志文件的保存期限为30天 -->
<maxHistory>30</maxHistory> <maxHistory>30</maxHistory>
<!-- 单个日志文件的最大大小 --> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<!-- maxFileSize:这是活动文件的大小默认值是10MB测试时可改成1KB看效果 -->
<maxFileSize>10MB</maxFileSize> <maxFileSize>10MB</maxFileSize>
<!-- 所有日志文件的总大小限制 --> </timeBasedFileNamingAndTriggeringPolicy>
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy> </rollingPolicy>
<encoder> <encoder>
<!-- pattern节点用来设置日志的输入格式 --> <!-- pattern节点用来设置日志的输入格式 -->
<pattern>%d %p (%file:%line\)- %m%n</pattern> <pattern>
%d %p (%file:%line\)- %m%n
</pattern>
<!-- 记录日志的编码:此处设置字符集 - --> <!-- 记录日志的编码:此处设置字符集 - -->
<charset>UTF-8</charset> <charset>UTF-8</charset>
</encoder> </encoder>
</appender> </appender>
<!-- 控制台输出日志级别 --> <!-- 控制台输出日志级别 -->
<root level="INFO"> <root level="INFO">
<appender-ref ref="STDOUT" /> <appender-ref ref="STDOUT" />
<appender-ref ref="FILE" /> <appender-ref ref="FILE" />
</root> </root>
<!-- 指定项目中某个包当有日志操作行为时的日志记录级别 --> <!-- 指定项目中某个包当有日志操作行为时的日志记录级别 -->
<!-- com.java2nb 为根包也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG --> <!-- com.maijinjie.springboot 为根包也就是只要是发生在这个根包下面的所有日志操作行为的权限都是DEBUG -->
<!-- 级别依次为从高到低FATAL > ERROR > WARN > INFO > DEBUG > TRACE --> <!-- 级别依次为从高到低FATAL > ERROR > WARN > INFO > DEBUG > TRACE -->
<logger name="com.java2nb" level="DEBUG" additivity="false"> <logger name="com.java2nb" level="DEBUG" additivity="false">
<appender-ref ref="STDOUT" /> <appender-ref ref="STDOUT" />

View File

@ -14,7 +14,7 @@
and t1.create_user_id = #{userId} and t1.create_user_id = #{userId}
</if> </if>
</trim> </trim>
order by t1.create_time desc
</select> </select>

View File

@ -31,9 +31,6 @@
<if test="updateTimeMin != null"> <if test="updateTimeMin != null">
and last_index_update_time >= #{updateTimeMin} and last_index_update_time >= #{updateTimeMin}
</if> </if>
<if test="sort != null">
order by ${sort} desc
</if>
</select> </select>

View File

@ -8,79 +8,6 @@
<title>作家管理系统-小说精品屋</title> <title>作家管理系统-小说精品屋</title>
<link rel="stylesheet" href="/css/base.css?v=1"/> <link rel="stylesheet" href="/css/base.css?v=1"/>
<link rel="stylesheet" href="/css/user.css"/> <link rel="stylesheet" href="/css/user.css"/>
<style>
/* 编辑器容器样式 */
.editor-container {
margin: 10px 0px;
padding: 20px;
background: #fff;
border-radius: 8px;
box-shadow: 0 2px 12px 0 rgba(0, 0, 0, .1);
}
/* 文本域样式 */
#bookContent {
width: 93%;
height: 400px;
padding: 15px;
border: 1px solid #e6e6e6;
border-radius: 4px;
font-size: 14px;
line-height: 1.6;
resize: vertical;
}
/* 工具栏样式 */
.ai-toolbar {
margin-bottom: 15px;
display: flex;
gap: 10px; /* 按钮间距 */
}
/* 自定义链接按钮样式 */
.ai-link {
display: inline-block;
padding: 10px 20px;
border-radius: 8px;
font-size: 14px;
font-weight: 500;
color: #fff;
text-decoration: none;
cursor: pointer;
transition: all 0.3s ease;
background: linear-gradient(135deg, #6a11cb, #2575fc);
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
/* 链接按钮悬停效果 */
.ai-link:hover {
transform: translateY(-2px);
box-shadow: 0 6px 12px rgba(0, 0, 0, 0.15);
}
/* 链接按钮点击效果 */
.ai-link:active {
transform: translateY(0);
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
/* 不同按钮的颜色 */
.ai-link.expand {
background: linear-gradient(135deg, #ff9a9e, #fad0c4);
}
.ai-link.condense {
background: linear-gradient(135deg, #a18cd1, #fbc2eb);
}
.ai-link.continue {
background: linear-gradient(135deg, #f6d365, #fda085);
}
.ai-link.polish {
background: linear-gradient(135deg, #ff6f61, #ffcc00);
}
</style>
</head> </head>
</head> </head>
<body class=""> <body class="">
@ -119,28 +46,17 @@
<ul class="log_list"> <ul class="log_list">
<li><span id="LabErr"></span></li> <li><span id="LabErr"></span></li>
<b>章节名</b> <b>章节名</b>
<li><input type="text" id="bookIndex" name="bookIndex" class="s_input"></li> <li><input type="text" id="bookIndex" name="bookIndex" class="s_input" ></li>
<b>章节内容</b> <b>章节内容</b><li id="contentLi">
<li id="contentLi" style="width: 500px"> <textarea name="bookContent" rows="30" cols="80" id="bookContent"
<div class="editor-container"> class="textarea"></textarea></li><br/>
<div class="ai-toolbar">
<a class="ai-link expand" data-type="stream/expand">AI扩写</a>
<a class="ai-link condense" data-type="stream/condense">AI缩写</a>
<a class="ai-link continue" data-type="stream/continue">AI续写</a>
<a class="ai-link polish" data-type="stream/polish">AI润色</a>
</div>
<textarea id="bookContent" name="bookContent"
placeholder="请输入文本内容..."></textarea>
</div>
<b>是否收费</b> <b>是否收费</b>
<li><input type="radio" name="isVip" value="0" checked>免费 <li><input type="radio" name="isVip" value="0" checked >免费
<input type="radio" name="isVip" value="1">收费 <input type="radio" name="isVip" value="1" >收费</li>
</li>
<li style="margin-top: 10px"><input type="button" onclick="addBookContent()" <li style="margin-top: 10px"><input type="button" onclick="addBookContent()" name="btnRegister" value="提交"
name="btnRegister" value="提交"
id="btnRegister" class="btn_red"> id="btnRegister" class="btn_red">
@ -197,10 +113,9 @@
var lock = false; var lock = false;
function addBookContent() { function addBookContent() {
if (lock) { if(lock){
return; return;
} }
lock = true; lock = true;
@ -210,14 +125,14 @@
var indexName = $("#bookIndex").val(); var indexName = $("#bookIndex").val();
if (!indexName) { if(!indexName){
$("#LabErr").html("章节名不能为空!"); $("#LabErr").html("章节名不能为空!");
lock = false; lock = false;
return; return;
} }
var content = $("#bookContent").val(); var content = $("#bookContent").val();
if (!content) { if(!content){
$("#LabErr").html("章节内容不能为空!"); $("#LabErr").html("章节内容不能为空!");
lock = false; lock = false;
return; return;
@ -227,15 +142,17 @@
var isVip = $("input:checked[name=isVip]").val(); var isVip = $("input:checked[name=isVip]").val();
$.ajax({ $.ajax({
type: "POST", type: "POST",
url: "/author/addBookContent", url: "/author/addBookContent",
data: {'bookId': bookId, 'indexName': indexName, 'content': content, 'isVip': isVip}, data: {'bookId':bookId,'indexName':indexName,'content':content,'isVip':isVip},
dataType: "json", dataType: "json",
success: function (data) { success: function (data) {
if (data.code == 200) { if (data.code == 200) {
window.location.href = '/author/index_list.html?bookId=' + bookId; window.location.href = '/author/index_list.html?bookId='+bookId;
} else { } else {
@ -252,116 +169,5 @@
} }
// 打字机效果函数
function typeWriter(textarea, text, speed = 50) {
let i = 0;
const timer = setInterval(() => {
if (i < text.length) {
textarea.val(textarea.val() + text.charAt(i));
i++;
// 滚动到底部
textarea.scrollTop(textarea[0].scrollHeight);
} else {
clearInterval(timer);
}
}, speed);
}
$('.ai-toolbar .ai-link').click(function (e) {
e.preventDefault(); // 阻止默认链接行为
const type = $(this).data('type');
const textarea = $('#bookContent');
const selectedText = textarea.val().substring(textarea[0].selectionStart, textarea[0].selectionEnd);
// 检查是否选中文本
if (!selectedText) {
layer.msg('请先选中要处理的文本');
return;
}
const loading = layer.load(1, {shade: 0.3});
// 参数配置
let params = {text: selectedText};
if (type === 'stream/expand' || type === 'stream/condense') {
layer.prompt({
title: '请输入比例',
value: 2,
btn: ['确定', '取消'],
btn2: function () {
layer.close(loading);
},
cancel: function () {
layer.close(loading);
}
}, function (value, index) {
if (isNaN(Number(value)) || isNaN(parseFloat(value))) {
layer.msg('请输入正确的比例');
return;
}
if (type === 'stream/expand' && value <= 1) {
layer.msg('请输入正确的比例');
return;
}
if (type === 'stream/condense' && (value <= 0 || value >= 1)) {
layer.msg('请输入正确的比例');
return;
}
params.ratio = parseFloat(value) * 100;
layer.close(index);
sendRequest(type, params, loading, textarea);
});
return;
} else if (type === 'stream/continue') {
layer.prompt({
title: '请输入续写长度字数',
value: 200,
btn: ['确定', '取消'],
btn2: function () {
layer.close(loading);
},
cancel: function () {
layer.close(loading);
}
}, function (value, index) {
if (!Number.isInteger(Number(value)) || value <= 0) {
layer.msg('请输入正确的长度');
return;
}
params.length = parseInt(value);
layer.close(index);
sendRequest(type, params, loading, textarea);
});
return;
}
sendRequest(type, params, loading, textarea);
});
function sendRequest(type, params, loading, textarea) {
const url = `/author/ai/${type}?text=${encodeURIComponent(params.text)}&ratio=${params.ratio}&length=${params.length}`;
const eventSource = new EventSource(url);
// 监听消息事件
eventSource.onmessage = function (event) {
layer.close(loading);
const data = event.data;
console.log('Received data:', data);
textarea.val(textarea.val() + data);
// 滚动到底部
textarea.scrollTop(textarea[0].scrollHeight);
};
// 监听错误事件
eventSource.onerror = function (error) {
layer.close(loading);
console.error('EventSource failed:', error);
eventSource.close(); // 关闭连接
};
}
</script> </script>
</html> </html>

View File

@ -142,12 +142,10 @@
<script src="/javascript/common.js" type="text/javascript"></script> <script src="/javascript/common.js" type="text/javascript"></script>
<script language="javascript" type="text/javascript"> <script language="javascript" type="text/javascript">
var coverUpdateInterval;
search(1, 5); search(1, 5);
function search(curr, limit) { function search(curr, limit) {
clearInterval(coverUpdateInterval);
$.ajax({ $.ajax({
type: "get", type: "get",
url: "/author/listBookByPage", url: "/author/listBookByPage",
@ -157,25 +155,6 @@
if (data.code == 200) { if (data.code == 200) {
var bookList = data.data.list; var bookList = data.data.list;
if (bookList.length > 0) { if (bookList.length > 0) {
if(curr == 1 && bookList[0].picUrl == '/images/default.gif'){
coverUpdateInterval = setInterval(function(){
$.ajax({
type: "get",
url: "/author/queryAiGenPic",
data: {'bookId': bookList[0].id},
dataType: "json",
success: function (data) {
if(data.code == 200 && data.data){
$("#cover"+bookList[0].id).attr("src", data.data);
clearInterval(coverUpdateInterval);
}
}
});
}, 3000);
setTimeout(() => {
clearInterval(coverUpdateInterval);
}, 10000);
}
$("#hasContentDiv").css("display", "block"); $("#hasContentDiv").css("display", "block");
$("#noContentDiv").css("display", "none"); $("#noContentDiv").css("display", "none");
var bookListHtml = ""; var bookListHtml = "";
@ -187,12 +166,15 @@
" </td>\n" +*/ " </td>\n" +*/
" <td style=\"position: relative\" class=\"goread\">\n" + " <td style=\"position: relative\" class=\"goread\">\n" +
"<input class=\"opacity\" onchange=\"picChange('" + book.id + "')\"\n" + "<input class=\"opacity\" onchange=\"picChange('" + book.id + "'," + i + ")\"\n" +
" type=\"file\" id=\"file" + book.id + "\" name=\"file\"\n" + " type=\"file\" id=\"file" + i + "\" name=\"file\"\n" +
" title=\"点击上传图片\"\n" + " title=\"点击上传图片\"\n" +
" style=\"z-index: 100;cursor: pointer;left: 30px; top: 0px; width: 60px; height: 80px; opacity: 0; position: absolute; \"\n" + " style=\"z-index: 100;cursor: pointer;left: 30px; top: 0px; width: 60px; height: 80px; opacity: 0; position: absolute; \"\n" +
" />" + " />" +
"<img id=\"cover" + book.id + "\" width='50' height='70' src='" + book.picUrl + "'/><br/>" + " " + book.bookName + "</td>\n" + "<img width='50' height='70' src='" + book.picUrl + "'/><br/>" +
" " + book.bookName + "</td>\n" +
" <td class=\"goread\" >" " <td class=\"goread\" >"
+ book.catName + "</td>\n" + + book.catName + "</td>\n" +
@ -295,20 +277,19 @@
} }
function picChange(bookId) { function picChange(bookId, i) {
var file = $("#file" + bookId).val(); //文件名称 var file = $("#file" + i).val(); //文件名称
if (file != "") { if (file != "") {
if (checkPicUpload($("#file" + bookId)[0])) { if (checkPicUpload($("#file" + i)[0])) {
$.ajaxFileUpload({ $.ajaxFileUpload({
url: "/file/picUpload", //用于文件上传的服务器端请求地址 url: "/file/picUpload", //用于文件上传的服务器端请求地址
secureuri: false, //是否需要安全协议一般设置为false secureuri: false, //是否需要安全协议一般设置为false
fileElementId: "file" + bookId, //文件上传域的ID fileElementId: "file" + i, //文件上传域的ID
dataType: "json", //返回值类型 一般设置为json dataType: "json", //返回值类型 一般设置为json
type: "post", type: "post",
success: function (data) { //服务器成功响应处理函数 success: function (data) { //服务器成功响应处理函数
if (data.code == 200) { if (data.code == 200) {
let picUrl = data.data;
$.ajax({ $.ajax({
type: "POST", type: "POST",
url: "/author/updateBookPic", url: "/author/updateBookPic",
@ -316,13 +297,17 @@
dataType: "json", dataType: "json",
success: function (data) { success: function (data) {
if (data.code == 200) { if (data.code == 200) {
$("#cover"+bookId).attr("src", picUrl);
location.reload();
} else { } else {
lock = false;
layer.alert(data.msg); layer.alert(data.msg);
} }
}, },
error: function () { error: function () {
lock = false;
layer.alert('网络异常'); layer.alert('网络异常');
} }
}) })

View File

@ -6,8 +6,8 @@
<meta name="keywords" th:content="${application.website.name}+',小说,小说网,言情小说,都市小说,玄幻小说,穿越小说,青春小说,总裁豪门小说,网络小说,免费小说,全本小说,原创网络文学'"/> <meta name="keywords" th:content="${application.website.name}+',小说,小说网,言情小说,都市小说,玄幻小说,穿越小说,青春小说,总裁豪门小说,网络小说,免费小说,全本小说,原创网络文学'"/>
<meta name="description" <meta name="description"
th:content="${application.website.name}+'每日更新小说连载,小说排行榜,提供言情小说,都市小说,玄幻小说,穿越小说,青春小说,总裁豪门小说,网络小说,免费小说,全本小说,首发小说,最新章节免费小说阅读,精彩尽在'+${application.website.name}+'。'"/> th:content="${application.website.name}+'每日更新小说连载,小说排行榜,提供言情小说,都市小说,玄幻小说,穿越小说,青春小说,总裁豪门小说,网络小说,免费小说,全本小说,首发小说,最新章节免费小说阅读,精彩尽在'+${application.website.name}+'。'"/>
<link href="/favicon.ico" type="image/x-icon" rel="shortcut icon"/> <link href="favicon.ico" type="image/x-icon" rel="shortcut icon"/>
<link href="/favicon.ico" type="image/x-icon" rel="Bookmark"/> <link href="favicon.ico" type="image/x-icon" rel="Bookmark"/>
<link rel="stylesheet" href="/css/main.css"/> <link rel="stylesheet" href="/css/main.css"/>
</head> </head>
<body> <body>

View File

@ -7,8 +7,8 @@
<meta name="keywords" th:content="${application.website.keyword}"/> <meta name="keywords" th:content="${application.website.keyword}"/>
<meta name="description" <meta name="description"
th:content="${application.website.description}"/> th:content="${application.website.description}"/>
<link href="/favicon.ico" type="image/x-icon" rel="shortcut icon"/> <link href="favicon.ico" type="image/x-icon" rel="shortcut icon"/>
<link href="/favicon.ico" type="image/x-icon" rel="Bookmark"/> <link href="favicon.ico" type="image/x-icon" rel="Bookmark"/>
<link rel="stylesheet" href="/css/main.css"/> <link rel="stylesheet" href="/css/main.css"/>
</head> </head>
<body> <body>

Some files were not shown because too many files have changed in this diff Show More