weather-data/README.md

# Weather Data Management System

A full-stack meteorological data management and analysis platform built with Spring Boot 3.5 and Vue 3. Supports multi-datasource environments with real-time file monitoring, statistical analysis, and role-based access control.

> Developer documentation: [CLAUDE.md](CLAUDE.md) | Module docs: [system-admin](system-admin/CLAUDE.md) | [system-common](system-common/CLAUDE.md) | [system-dynamic-datasource](system-dynamic-datasource/CLAUDE.md) | [weather-data-ui](weather-data-ui/CLAUDE.md)
>
> Chinese version: [README.zh-CN.md](README.zh-CN.md)

## Project Structure

```
weather-data/
├── system-common/              Shared library (base classes, Redis, validation, utilities)
├── system-dynamic-datasource/  Multi-datasource support (AbstractRoutingDataSource)
├── system-admin/               Main application (REST API, security, jobs, file scanning)
└── weather-data-ui/            Vue 3 frontend (Element Plus, ECharts, Pinia)
```

### Module Relationships

```
system-common          <--  depended on by all backend modules
system-dynamic-datasource  <--  depended on by system-admin
system-admin           <--  runnable Spring Boot app, depends on both above
weather-data-ui        <--  independent frontend, communicates with system-admin via REST API
```

`system-common` provides the base class hierarchy (`BaseEntity`, `CrudService`, `CrudServiceImpl`), Redis utilities, validation framework, XSS protection, and shared utilities. All backend modules consume it.

`system-dynamic-datasource` provides the `@DataSource` annotation and `DynamicDataSource` (extends `AbstractRoutingDataSource`) for switching between multiple databases at runtime. Used by `system-admin` when queries need to target different data sources.

`system-admin` is the main runnable application. It contains all business logic organized under `modules/`:

| Domain | Package | Description |
|--------|---------|-------------|
| Weather daily data | `modules/weather/dailydata/` | Excel batch import with async two-pass processing, CRUD, statistical summaries with Redis caching |
| Weather stations | `modules/weather/station/` | Station registry, dept association for data scoping |
| File scanning | `modules/weather/filescan/` | WatchService-based directory monitoring, parallel file scanning with MD5 deduplication, auto-import of meteorological files |
| Region management | `modules/region/` | Geographic region tree (province/city/county), CSV data loading |
| System management | `modules/sys/` | Users, roles, menus, departments, dictionaries, parameters |
| Alerts / notifications | `modules/sys/alert/` | SSE real-time push, Spring event-driven broadcast, polling-based external alert collection |
| Security | `modules/security/` | Shiro + token-based auth, OAuth2 filter, BCrypt password hashing, session persistence |
| Job scheduling | `modules/job/` | Quartz dynamic job management, online start/stop/configure |
| Audit logs | `modules/log/` | Operation log, login log, error log |
| Cloud storage | `modules/oss/` | Alibaba Cloud / Qiniu / Tencent Cloud file storage |

`weather-data-ui` is a Vue 3 SPA that communicates with `system-admin` via REST API. It features dynamic routing (routes loaded from server menus on login), tab-based navigation, ECharts visualization, SSE real-time alert streaming, and client-side Excel/PDF export.

## Tech Stack

| Layer | Technology |
|-------|-----------|
| Backend framework | Spring Boot 3.5.11, MyBatis-Plus 3.5.8, Apache Shiro 1.12 |
| Database | MySQL 8.0 (also supports Oracle, SQL Server, PostgreSQL, Dameng) |
| Connection pool | Druid 1.2 |
| Cache | Redis 7 (Lettuce client, optional via `project-options.redis.open`) |
| Job scheduling | Quartz 2.3 |
| API documentation | Knife4j 4.5 (Swagger) |
| Excel processing | EasyExcel 3.2 |
| Frontend | Vue 3.5, TypeScript 5.7, Vite 5.4, Element Plus 2.10, Pinia 2.3, ECharts 5 |

**Requirements**: JDK 17+, Maven 3.6+, Node.js 18+, MySQL 8.0+

## Quick Start (Local Development)

### 1. Initialize Database

```bash
mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS weather_data_system DEFAULT CHARSET utf8mb4"
mysql -u root -p weather_data_system < system-admin/db/weather_data_system.sql
mysql -u root -p weather_data_system < system-admin/db/init_system_params.sql
```

### 2. Configure Datasource

Edit `system-admin/src/main/resources/application-dev.yml`, update MySQL and Redis connection settings:

```yaml
spring:
  datasource:
    druid:
      url: jdbc:mysql://localhost:3306/weather_data_system?...
      username: root
      password: your_password
  data:
    redis:
      host: 127.0.0.1
      port: 6379
      password:
```

### 3. Start Backend

```bash
# Full build
mvn clean install -DskipTests

# Run from IDE: AdminApplication.java
# Or from CLI:
cd system-admin && mvn spring-boot:run -Dspring-boot.run.profiles=dev
```

Admin backend: http://localhost:8080/system-admin
API docs: http://localhost:8080/system-admin/doc.html
Default account: `admin` / `admin`

### 4. Start Frontend

```bash
cd weather-data-ui
npm install
npm run dev
```

Frontend dev server: http://localhost:8001

Edit `weather-data-ui/.env.development` and set `VITE_APP_API` to your backend URL if needed.

## Deployment

1. Build backend: `mvn clean package -DskipTests -f system-admin`
2. Build frontend: `cd weather-data-ui && npm run build`
3. Deploy `system-admin/target/system-admin.jar` and `weather-data-ui/dist/` to server
4. Configure `application-prod.yml` with production DB/Redis settings
5. Start backend: `java -jar system-admin.jar --spring.profiles.active=prod`

## Key Features

- **Weather data import**: Excel batch import with async two-pass processing, real-time progress tracking, multi-row INSERT optimization, automatic station-to-department mapping
- **Statistical analysis**: Same-date-across-years summarization with Redis caching, automatic cache-to-database fallback, multi-dimensional queries
- **File monitoring**: WatchService-based automatic directory watching, parallel file scanning on startup, MD5 deduplication, file lifecycle management (receive -> display -> archive)
- **Real-time notifications**: SSE-based alert streaming to frontend marquee, Spring event-driven broadcast with create/withdraw/delete lifecycle, polling-based external alert source collection
- **Data scoping**: Department-level row-level security via MyBatis-Plus interceptor with `@DataFilter` annotation, SQL WHERE clause injection
- **Dynamic datasource**: Runtime datasource switching via `@DataSource` annotation with ThreadLocal context propagation
- **Job scheduling**: Quartz-based dynamic job management with online create/start/stop/pause, Cron expression editing, execution history
- **Cloud storage**: Alibaba Cloud OSS, Qiniu, Tencent Cloud COS integration with unified upload interface