137 lines
6.7 KiB
Markdown
137 lines
6.7 KiB
Markdown
# Weather Data Management System
|
|
|
|
A full-stack meteorological data management and analysis platform built with Spring Boot 3.5 and Vue 3. Supports multi-datasource environments with real-time file monitoring, statistical analysis, and role-based access control.
|
|
|
|
> Developer documentation: [CLAUDE.md](CLAUDE.md) | Module docs: [system-admin](system-admin/CLAUDE.md) | [system-common](system-common/CLAUDE.md) | [system-dynamic-datasource](system-dynamic-datasource/CLAUDE.md) | [weather-data-ui](weather-data-ui/CLAUDE.md)
|
|
>
|
|
> Chinese version: [README.zh-CN.md](README.zh-CN.md)
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
weather-data/
|
|
├── system-common/ Shared library (base classes, Redis, validation, utilities)
|
|
├── system-dynamic-datasource/ Multi-datasource support (AbstractRoutingDataSource)
|
|
├── system-admin/ Main application (REST API, security, jobs, file scanning)
|
|
└── weather-data-ui/ Vue 3 frontend (Element Plus, ECharts, Pinia)
|
|
```
|
|
|
|
### Module Relationships
|
|
|
|
```
|
|
system-common <-- depended on by all backend modules
|
|
system-dynamic-datasource <-- depended on by system-admin
|
|
system-admin <-- runnable Spring Boot app, depends on both above
|
|
weather-data-ui <-- independent frontend, communicates with system-admin via REST API
|
|
```
|
|
|
|
`system-common` provides the base class hierarchy (`BaseEntity`, `CrudService`, `CrudServiceImpl`), Redis utilities, validation framework, XSS protection, and shared utilities. All backend modules consume it.
|
|
|
|
`system-dynamic-datasource` provides the `@DataSource` annotation and `DynamicDataSource` (extends `AbstractRoutingDataSource`) for switching between multiple databases at runtime. Used by `system-admin` when queries need to target different data sources.
|
|
|
|
`system-admin` is the main runnable application. It contains all business logic organized under `modules/`:
|
|
|
|
| Domain | Package | Description |
|
|
|--------|---------|-------------|
|
|
| Weather daily data | `modules/weather/dailydata/` | Excel batch import with async two-pass processing, CRUD, statistical summaries with Redis caching |
|
|
| Weather stations | `modules/weather/station/` | Station registry, dept association for data scoping |
|
|
| File scanning | `modules/weather/filescan/` | WatchService-based directory monitoring, parallel file scanning with MD5 deduplication, auto-import of meteorological files |
|
|
| Region management | `modules/region/` | Geographic region tree (province/city/county), CSV data loading |
|
|
| System management | `modules/sys/` | Users, roles, menus, departments, dictionaries, parameters |
|
|
| Alerts / notifications | `modules/sys/alert/` | SSE real-time push, Spring event-driven broadcast, polling-based external alert collection |
|
|
| Security | `modules/security/` | Shiro + token-based auth, OAuth2 filter, BCrypt password hashing, session persistence |
|
|
| Job scheduling | `modules/job/` | Quartz dynamic job management, online start/stop/configure |
|
|
| Audit logs | `modules/log/` | Operation log, login log, error log |
|
|
| Cloud storage | `modules/oss/` | Alibaba Cloud / Qiniu / Tencent Cloud file storage |
|
|
|
|
`weather-data-ui` is a Vue 3 SPA that communicates with `system-admin` via REST API. It features dynamic routing (routes loaded from server menus on login), tab-based navigation, ECharts visualization, SSE real-time alert streaming, and client-side Excel/PDF export.
|
|
|
|
## Tech Stack
|
|
|
|
| Layer | Technology |
|
|
|-------|-----------|
|
|
| Backend framework | Spring Boot 3.5.11, MyBatis-Plus 3.5.8, Apache Shiro 1.12 |
|
|
| Database | MySQL 8.0 (also supports Oracle, SQL Server, PostgreSQL, Dameng) |
|
|
| Connection pool | Druid 1.2 |
|
|
| Cache | Redis 7 (Lettuce client, optional via `project-options.redis.open`) |
|
|
| Job scheduling | Quartz 2.3 |
|
|
| API documentation | Knife4j 4.5 (Swagger) |
|
|
| Excel processing | EasyExcel 3.2 |
|
|
| Frontend | Vue 3.5, TypeScript 5.7, Vite 5.4, Element Plus 2.10, Pinia 2.3, ECharts 5 |
|
|
|
|
**Requirements**: JDK 17+, Maven 3.6+, Node.js 18+, MySQL 8.0+
|
|
|
|
## Quick Start (Local Development)
|
|
|
|
### 1. Initialize Database
|
|
|
|
```bash
|
|
mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS weather_data_system DEFAULT CHARSET utf8mb4"
|
|
mysql -u root -p weather_data_system < system-admin/db/weather_data_system.sql
|
|
mysql -u root -p weather_data_system < system-admin/db/init_system_params.sql
|
|
```
|
|
|
|
### 2. Configure Datasource
|
|
|
|
Edit `system-admin/src/main/resources/application-dev.yml`, update MySQL and Redis connection settings:
|
|
|
|
```yaml
|
|
spring:
|
|
datasource:
|
|
druid:
|
|
url: jdbc:mysql://localhost:3306/weather_data_system?...
|
|
username: root
|
|
password: your_password
|
|
data:
|
|
redis:
|
|
host: 127.0.0.1
|
|
port: 6379
|
|
password:
|
|
```
|
|
|
|
### 3. Start Backend
|
|
|
|
```bash
|
|
# Full build
|
|
mvn clean install -DskipTests
|
|
|
|
# Run from IDE: AdminApplication.java
|
|
# Or from CLI:
|
|
cd system-admin && mvn spring-boot:run -Dspring-boot.run.profiles=dev
|
|
```
|
|
|
|
Admin backend: http://localhost:8080/system-admin
|
|
API docs: http://localhost:8080/system-admin/doc.html
|
|
Default account: `admin` / `admin`
|
|
|
|
### 4. Start Frontend
|
|
|
|
```bash
|
|
cd weather-data-ui
|
|
npm install
|
|
npm run dev
|
|
```
|
|
|
|
Frontend dev server: http://localhost:8001
|
|
|
|
Edit `weather-data-ui/.env.development` and set `VITE_APP_API` to your backend URL if needed.
|
|
|
|
## Deployment
|
|
|
|
1. Build backend: `mvn clean package -DskipTests -f system-admin`
|
|
2. Build frontend: `cd weather-data-ui && npm run build`
|
|
3. Deploy `system-admin/target/system-admin.jar` and `weather-data-ui/dist/` to server
|
|
4. Configure `application-prod.yml` with production DB/Redis settings
|
|
5. Start backend: `java -jar system-admin.jar --spring.profiles.active=prod`
|
|
|
|
## Key Features
|
|
|
|
- **Weather data import**: Excel batch import with async two-pass processing, real-time progress tracking, multi-row INSERT optimization, automatic station-to-department mapping
|
|
- **Statistical analysis**: Same-date-across-years summarization with Redis caching, automatic cache-to-database fallback, multi-dimensional queries
|
|
- **File monitoring**: WatchService-based automatic directory watching, parallel file scanning on startup, MD5 deduplication, file lifecycle management (receive -> display -> archive)
|
|
- **Real-time notifications**: SSE-based alert streaming to frontend marquee, Spring event-driven broadcast with create/withdraw/delete lifecycle, polling-based external alert source collection
|
|
- **Data scoping**: Department-level row-level security via MyBatis-Plus interceptor with `@DataFilter` annotation, SQL WHERE clause injection
|
|
- **Dynamic datasource**: Runtime datasource switching via `@DataSource` annotation with ThreadLocal context propagation
|
|
- **Job scheduling**: Quartz-based dynamic job management with online create/start/stop/pause, Cron expression editing, execution history
|
|
- **Cloud storage**: Alibaba Cloud OSS, Qiniu, Tencent Cloud COS integration with unified upload interface
|