diff --git a/CLAUDE.md b/CLAUDE.md index d4040f9..ec52241 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,33 +29,59 @@ Each module has its own `CLAUDE.md` with module-specific architecture and conven ## Build & Run Commands +### Database Setup + +```bash +mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS weather_data_system DEFAULT CHARSET utf8mb4" +mysql -u root -p weather_data_system < system-admin/db/weather_data_system.sql +mysql -u root -p weather_data_system < system-admin/db/init_system_params.sql +``` + ### Backend ```bash # Build everything (skip tests) mvn clean package -DskipTests +# Build everything with tests +mvn clean package + # Build only system-admin module cd system-admin && mvn package -DskipTests -# Run with dev profile +# Run a single test class +cd system-admin && mvn test -Dtest=RedisTest -DfailIfNoTests=false + +# Run with dev profile (most common for local dev) cd system-admin && mvn spring-boot:run -Dspring-boot.run.profiles=dev + +# Run with other profiles +cd system-admin && mvn spring-boot:run -Dspring-boot.run.profiles=test +cd system-admin && mvn spring-boot:run -Dspring-boot.run.profiles=prod ``` -Main entry: `system-admin/src/main/java/com/weather/AdminApplication.java` +Main entry: `system-admin/src/main/java/com/weather/AdminApplication.java`. Default account: `admin` / `admin`. + +Backend is served at `http://localhost:8080/system-admin`. API docs at `http://localhost:8080/system-admin/doc.html`. Druid monitoring at `http://localhost:8080/system-admin/druid/`. ### Frontend ```bash cd weather-data-ui npm install -npm run dev # dev server on port 8001 -npm run build # production build +npm run dev # dev server on port 8001, proxies /system-admin to backend +npm run build # production build, outputs to dist/ npm run lint # ESLint on src/**/*.{vue,ts} +npm run serve # build + vite preview ``` +The Vite dev server proxies `/system-admin` requests to the backend. Edit `weather-data-ui/.env.development` and set `VITE_APP_API` to change the backend URL. + ## Key Configuration -- **Dev profile**: `system-admin/src/main/resources/application-dev.yml` -- local MySQL (weather_data_system), Redis 127.0.0.1:6379 -- **Prod profile**: `application-prod.yml` -- production configuration with external DB/Redis -- **project-options.redis.open**: toggles Redis caching globally (set via sys_params) +- **Dev profile** (`application-dev.yml`): local MySQL (weather_data_system), Redis 127.0.0.1:6379 +- **Test profile** (`application-test.yml`): test server MySQL (192.168.2.186), Redis 192.168.2.186:6379 +- **Prod profile** (`application-prod.yml`): production configuration with external DB/Redis +- **project-options.redis.open**: toggles Redis caching globally (set via sys_params table) +- **Tests are skipped by default**: `maven-surefire-plugin` has `true`. Tests use JUnit 4 with `SpringRunner` (`@SpringBootTest`). Override with `-DskipTests=false` or run individual tests via `-Dtest=`. +- **JDBC URL** uses `rewriteBatchedStatements=true` for MySQL batch INSERT optimization. diff --git a/README.md b/README.md index 1e557c4..cfe377e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Weather Data Management System -A full-stack meteorological data management and analysis platform built with Spring Boot 3.5 and Vue 3. +A full-stack meteorological data management and analysis platform built with Spring Boot 3.5 and Vue 3. Supports multi-datasource environments with real-time file monitoring, statistical analysis, and role-based access control. > Developer documentation: [CLAUDE.md](CLAUDE.md) | Module docs: [system-admin](system-admin/CLAUDE.md) | [system-common](system-common/CLAUDE.md) | [system-dynamic-datasource](system-dynamic-datasource/CLAUDE.md) | [weather-data-ui](weather-data-ui/CLAUDE.md) > @@ -33,18 +33,18 @@ weather-data-ui <-- independent frontend, communicates with system-admin | Domain | Package | Description | |--------|---------|-------------| -| Weather daily data | `modules/weather/dailydata/` | Excel batch import, CRUD, statistical summaries with Redis caching | +| Weather daily data | `modules/weather/dailydata/` | Excel batch import with async two-pass processing, CRUD, statistical summaries with Redis caching | | Weather stations | `modules/weather/station/` | Station registry, dept association for data scoping | -| File scanning | `modules/weather/filescan/` | WatchService-based directory monitoring, auto-import of meteorological files | -| Region management | `modules/region/` | Geographic region tree (province/city/county) | +| File scanning | `modules/weather/filescan/` | WatchService-based directory monitoring, parallel file scanning with MD5 deduplication, auto-import of meteorological files | +| Region management | `modules/region/` | Geographic region tree (province/city/county), CSV data loading | | System management | `modules/sys/` | Users, roles, menus, departments, dictionaries, parameters | -| Alerts / notifications | `modules/sys/alert/` | SSE real-time push, Spring event-driven broadcast | -| Security | `modules/security/` | Shiro + token-based auth, OAuth2 filter, password hashing | -| Job scheduling | `modules/job/` | Quartz dynamic job management, online start/stop | +| Alerts / notifications | `modules/sys/alert/` | SSE real-time push, Spring event-driven broadcast, polling-based external alert collection | +| Security | `modules/security/` | Shiro + token-based auth, OAuth2 filter, BCrypt password hashing, session persistence | +| Job scheduling | `modules/job/` | Quartz dynamic job management, online start/stop/configure | | Audit logs | `modules/log/` | Operation log, login log, error log | | Cloud storage | `modules/oss/` | Alibaba Cloud / Qiniu / Tencent Cloud file storage | -`weather-data-ui` is a Vue 3 SPA that communicates with `system-admin` via REST API. It features dynamic routing (routes loaded from server menus on login), tab-based navigation, ECharts visualization, and real-time SSE alert streaming. +`weather-data-ui` is a Vue 3 SPA that communicates with `system-admin` via REST API. It features dynamic routing (routes loaded from server menus on login), tab-based navigation, ECharts visualization, SSE real-time alert streaming, and client-side Excel/PDF export. ## Tech Stack @@ -68,6 +68,7 @@ weather-data-ui <-- independent frontend, communicates with system-admin ```bash mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS weather_data_system DEFAULT CHARSET utf8mb4" mysql -u root -p weather_data_system < system-admin/db/weather_data_system.sql +mysql -u root -p weather_data_system < system-admin/db/init_system_params.sql ``` ### 2. Configure Datasource @@ -123,17 +124,13 @@ Edit `weather-data-ui/.env.development` and set `VITE_APP_API` to your backend U 4. Configure `application-prod.yml` with production DB/Redis settings 5. Start backend: `java -jar system-admin.jar --spring.profiles.active=prod` -### Windows Server - -See `deploy/` directory for one-click deployment scripts (`deploy.bat` / `deploy.ps1`). Requires Windows 10 1803+ or Windows Server 2019+, JDK 17+ pre-installed. Run as Administrator. - ## Key Features -- **Weather data import**: Excel batch import with async two-pass processing, progress tracking, multi-row INSERT optimization -- **Statistical analysis**: Same-date-across-years summarization with Redis caching, multi-dimensional queries -- **File monitoring**: Automatic directory watching (WatchService), MD5 deduplication, file lifecycle management (receive -> display -> archive) -- **Real-time notifications**: SSE-based alert streaming, Spring event-driven broadcast -- **Data scoping**: Dept-level row-level security via MyBatis-Plus interceptor with `@DataFilter` annotation -- **Dynamic datasource**: Runtime datasource switching via `@DataSource` annotation with ThreadLocal context -- **Job scheduling**: Quartz-based dynamic job management with online start/stop/configure -- **Cloud storage**: Alibaba Cloud OSS, Qiniu, Tencent Cloud COS integration \ No newline at end of file +- **Weather data import**: Excel batch import with async two-pass processing, real-time progress tracking, multi-row INSERT optimization, automatic station-to-department mapping +- **Statistical analysis**: Same-date-across-years summarization with Redis caching, automatic cache-to-database fallback, multi-dimensional queries +- **File monitoring**: WatchService-based automatic directory watching, parallel file scanning on startup, MD5 deduplication, file lifecycle management (receive -> display -> archive) +- **Real-time notifications**: SSE-based alert streaming to frontend marquee, Spring event-driven broadcast with create/withdraw/delete lifecycle, polling-based external alert source collection +- **Data scoping**: Department-level row-level security via MyBatis-Plus interceptor with `@DataFilter` annotation, SQL WHERE clause injection +- **Dynamic datasource**: Runtime datasource switching via `@DataSource` annotation with ThreadLocal context propagation +- **Job scheduling**: Quartz-based dynamic job management with online create/start/stop/pause, Cron expression editing, execution history +- **Cloud storage**: Alibaba Cloud OSS, Qiniu, Tencent Cloud COS integration with unified upload interface diff --git a/README.zh-CN.md b/README.zh-CN.md index f5af2e0..df5f1e7 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,6 +1,6 @@ # 气象数据平台 -基于 Spring Boot 3.5 与 Vue 3 的全栈气象数据平台。 +基于 Spring Boot 3.5 与 Vue 3 的全栈气象数据管理与分析平台。支持多数据源环境，提供实时文件监控、统计分析与基于角色的访问控制。 > 开发者文档：[CLAUDE.md](CLAUDE.md) | 模块文档：[system-admin](system-admin/CLAUDE.md) | [system-common](system-common/CLAUDE.md) | [system-dynamic-datasource](system-dynamic-datasource/CLAUDE.md) | [weather-data-ui](weather-data-ui/CLAUDE.md) > @@ -33,18 +33,18 @@ weather-data-ui <-- 独立前端，通过 REST API 与 system-admin 通 | 业务域 | 包路径 | 功能说明 | |--------|--------|----------| -| 逐日气象数据 | [modules/weather/dailydata/](system-admin/src/main/java/com/weather/modules/weather/dailydata/) | Excel 批量导入、CRUD、Redis 缓存统计汇总 | +| 逐日气象数据 | [modules/weather/dailydata/](system-admin/src/main/java/com/weather/modules/weather/dailydata/) | Excel 异步双遍扫描批量导入、CRUD、Redis 缓存统计汇总 | | 气象站点 | [modules/weather/station/](system-admin/src/main/java/com/weather/modules/weather/station/) | 站点注册，关联部门实现数据权限隔离 | -| 文件扫描 | [modules/weather/filescan/](system-admin/src/main/java/com/weather/modules/weather/filescan/) | WatchService 目录监控，气象资料自动入库 | -| 区域管理 | [modules/region/](system-admin/src/main/java/com/weather/modules/region/) | 行政区域树（省/市/县） | +| 文件扫描 | [modules/weather/filescan/](system-admin/src/main/java/com/weather/modules/weather/filescan/) | WatchService 目录监控，并行文件扫描，MD5 去重，气象资料自动入库 | +| 区域管理 | [modules/region/](system-admin/src/main/java/com/weather/modules/region/) | 行政区域树（省/市/县），CSV 数据加载 | | 系统管理 | [modules/sys/](system-admin/src/main/java/com/weather/modules/sys/) | 用户、角色、菜单、部门、字典、参数 | -| 通知消息 | [modules/sys/alert/](system-admin/src/main/java/com/weather/modules/sys/alert/) | SSE 实时推送，Spring 事件驱动广播 | -| 安全认证 | [modules/security/](system-admin/src/main/java/com/weather/modules/security/) | Shiro + Token 认证，OAuth2 过滤器，密码加密 | -| 定时任务 | [modules/job/](system-admin/src/main/java/com/weather/modules/job/) | Quartz 动态任务管理，在线启停 | +| 通知消息 | [modules/sys/alert/](system-admin/src/main/java/com/weather/modules/sys/alert/) | SSE 实时推送，Spring 事件驱动广播，外部告警源轮询采集 | +| 安全认证 | [modules/security/](system-admin/src/main/java/com/weather/modules/security/) | Shiro + Token 认证，OAuth2 过滤器，BCrypt 密码加密，登录状态持久化 | +| 定时任务 | [modules/job/](system-admin/src/main/java/com/weather/modules/job/) | Quartz 动态任务管理，在线创建、启停、修改 Cron 表达式 | | 审计日志 | [modules/log/](system-admin/src/main/java/com/weather/modules/log/) | 操作日志、登录日志、异常日志 | | 云存储 | [modules/oss/](system-admin/src/main/java/com/weather/modules/oss/) | 阿里云 OSS / 七牛云 / 腾讯云 COS 文件存储 | -**[weather-data-ui](weather-data-ui/)** 是 Vue 3 单页应用，通过 REST API 与 system-admin 通信。特性包括：动态路由（登录时从服务端菜单加载）、标签页导航、ECharts 图表可视化、SSE 实时通知滚动。 +**[weather-data-ui](weather-data-ui/)** 是 Vue 3 单页应用，通过 REST API 与 system-admin 通信。特性包括：动态路由（登录时从服务端菜单加载）、标签页导航、ECharts 图表可视化、SSE 实时通知滚动、客户端 Excel/PDF 导出。 ## 技术栈 @@ -68,9 +68,10 @@ weather-data-ui <-- 独立前端，通过 REST API 与 system-admin 通 ```bash mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS weather_data_system DEFAULT CHARSET utf8mb4" mysql -u root -p weather_data_system < system-admin/db/weather_data_system.sql +mysql -u root -p weather_data_system < system-admin/db/init_system_params.sql ``` -数据库脚本：[system-admin/db/weather_data_system.sql](system-admin/db/weather_data_system.sql) +数据库脚本：[system-admin/db/weather_data_system.sql](system-admin/db/weather_data_system.sql) | [system-admin/db/init_system_params.sql](system-admin/db/init_system_params.sql) ### 2. 配置数据源 @@ -125,17 +126,13 @@ npm run dev 4. 配置 `application-prod.yml` 中的生产环境数据库和 Redis 连接信息 5. 启动后端：`java -jar system-admin.jar --spring.profiles.active=prod` -### Windows Server - -参见 `deploy/` 目录下的一键部署脚本（[deploy.bat](deploy/deploy.bat) / [deploy.ps1](deploy/deploy.ps1)）。要求 Windows 10 1803+ 或 Windows Server 2019+，需预装 JDK 17+。以管理员身份运行。 - ## 核心功能 -- **气象数据导入**：Excel 异步双遍扫描批量导入，进度可查，多行 INSERT 优化。入口：[WeatherDataImportManager](system-admin/src/main/java/com/weather/modules/weather/dailydata/WeatherDataImportManager.java) -- **统计汇总**：历年同月同日数据汇总，Redis 缓存加速查询，缓存命中自动降级至数据库。缓存任务：[WeatherSummarizeCacheTask](system-admin/src/main/java/com/weather/modules/weather/dailydata/task/WeatherSummarizeCacheTask.java) -- **文件监控**：基于 WatchService 自动监控资料目录，MD5 去重，文件生命周期管理（接收 -> 展示 -> 归档）。服务：[FileWatchServiceManager](system-admin/src/main/java/com/weather/modules/weather/filescan/FileWatchServiceManager.java) -- **实时通知**：基于 SSE 的服务端推送，Spring 事件驱动广播。服务：[SseAlertService](system-admin/src/main/java/com/weather/modules/sys/alert/SseAlertService.java) -- **数据权限**：基于 MyBatis-Plus 拦截器的部门级行级安全，通过 `@DataFilter` 注解启用。拦截器：[DataFilterInterceptor](system-admin/src/main/java/com/weather/common/interceptor/DataFilterInterceptor.java) +- **气象数据导入**：Excel 异步双遍扫描批量导入，实时进度可查，多行 INSERT 优化，自动站点-部门映射。入口：[WeatherDataImportManager](system-admin/src/main/java/com/weather/modules/weather/dailydata/WeatherDataImportManager.java) +- **统计汇总**：历年同月同日数据汇总，Redis 缓存加速查询，缓存未命中时自动降级至数据库。缓存任务：[WeatherSummarizeCacheTask](system-admin/src/main/java/com/weather/modules/weather/dailydata/task/WeatherSummarizeCacheTask.java) +- **文件监控**：基于 WatchService 自动监控资料目录，启动时并行扫描存量文件，MD5 去重，文件生命周期管理（接收 -> 展示 -> 归档）。服务：[FileWatchServiceManager](system-admin/src/main/java/com/weather/modules/weather/filescan/FileWatchServiceManager.java) +- **实时通知**：基于 SSE 的服务端推送，Spring 事件驱动广播（创建/撤回/删除），支持外部告警源轮询采集。服务：[SseAlertService](system-admin/src/main/java/com/weather/modules/sys/alert/SseAlertService.java) +- **数据权限**：基于 MyBatis-Plus 拦截器的部门级行级安全，通过 `@DataFilter` 注解启用，SQL WHERE 子句注入。拦截器：[DataFilterInterceptor](system-admin/src/main/java/com/weather/common/interceptor/DataFilterInterceptor.java) - **动态数据源**：运行时通过 `@DataSource` 注解切换数据源，ThreadLocal 上下文传递。实现：[DynamicDataSource](system-dynamic-datasource/src/main/java/com/weather/commons/dynamic/datasource/config/DynamicDataSource.java) -- **定时任务**：Quartz 动态任务管理，支持在线创建、启停、修改 Cron 表达式。配置：[ScheduleConfig](system-admin/src/main/java/com/weather/modules/job/config/ScheduleConfig.java) -- **云存储**：支持阿里云 OSS、七牛云、腾讯云 COS 文件上传与管理。工厂：[OSSFactory](system-admin/src/main/java/com/weather/modules/oss/cloud/OSSFactory.java) +- **定时任务**：Quartz 动态任务管理，支持在线创建、启停、暂停、修改 Cron 表达式，执行历史可查。配置：[ScheduleConfig](system-admin/src/main/java/com/weather/modules/job/config/ScheduleConfig.java) +- **云存储**：支持阿里云 OSS、七牛云、腾讯云 COS 文件上传与管理，统一上传接口。工厂：[OSSFactory](system-admin/src/main/java/com/weather/modules/oss/cloud/OSSFactory.java)