sansenhoshi/weather-data
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

项目优化，构建文件Lombok异常问题修复

Browse Source
This commit is contained in:
sansenhoshi
2026-06-29 13:55:42 +08:00
parent 53b3b2c989
commit 0799f9cb4d
47 changed files with 1034 additions and 865 deletions
Download Patch File Download Diff File Expand all files Collapse all files
system-admin/CLAUDE.md
+111 -109
View File
@@ -1,144 +1,146 @@
# CLAUDE.md — system-admin
# CLAUDE.md - system-admin
Backend module: Spring Boot 3.5 admin application (port 8080, context `/system-admin`).
This file provides guidance to Claude Code when working in the system-admin module.
---
## Purpose
## Build & Run
Main Spring Boot application module. Entry point: `com.weather.AdminApplication`. Depends on `system-common` and `system-dynamic-datasource`.
```bash
mvn clean install -DskipTests # full build (tests skipped by default)
mvn clean install -DskipTests=false # build with tests
mvn -pl system-admin -DskipTests=false -Dtest=YourTestClass test # single test
## Module Structure
```
com.weather/
├── AdminApplication.java # @SpringBootApplication, @EnableAsync
├── common/ # Module-level shared code
│ ├── annotation/ # @DataFilter, @LogOperation
│ ├── aspect/ # DataFilterAspect, LogOperationAspect
│ ├── config/ # JacksonConfig, MybatisPlusConfig, SwaggerConfig, FileServerConfig
│ ├── exception/ # CustomExceptionHandler
│ ├── handler/ # FieldMetaObjectHandler (MyBatis-Plus auto-fill)
│ ├── interceptor/ # DataFilterInterceptor, DataScope
│ ├── utils/ # ExcelUtils, TimeUtils
│ └── validator/group/ # Cloud storage validator groups
└── modules/
├── job/ # Quartz scheduling
├── log/ # Audit logs (operation, login, error)
├── oss/ # Cloud file storage (Alibaba/Qiniu/Tencent)
├── region/ # Geographic region tree
├── security/ # Shiro auth, OAuth2 filter, login
├── sys/ # System management (users, roles, menus, depts, dicts, params, alerts)
└── weather/ # Weather domain
├── dailydata/ # Daily weather data CRUD, import, caching
├── filescan/ # WatchService file scanning pipeline
└── station/ # Weather station management
```
Tests use **JUnit 4** (`@RunWith(SpringRunner.class)`), not JUnit 5. Only 2 test files exist (Redis, dynamic datasource). No Maven wrapper (`mvnw`). Maven remote repository: Aliyun mirror.
## Service Layer Pattern
Launch from IntelliJ:
- `AdminApplication` (`system-admin/`) → port 8080, context `/system-admin`
- `GeneratorApplication` (`renren-generator/`) — commented out of build
All business modules follow this structure under `modules/<domain>/`:
## Service layer pattern
Two base classes in `system-common`:
| Base | Purpose |
|---|---|
| `CrudService<Dao, Entity, DTO>` | Generic CRUD: `page()`, `get()`, `save()`, `update()`, `delete()` |
| `BaseService<Dao>` | Lighter base without DTO generic |
Module convention:
```
modules/<name>/
├── controller/ → @RestController, returns Result
├── dao/ → extends BaseMapper<Entity> (MyBatis-Plus)
├── dto/ → request/query DTOs (often extends BaseEntity)
├── entity/ → @TableName JPA entity
├── service/ → interface extends CrudService/BaseService
│ └── impl/ → @Service implementation
├── excel/ → EasyExcel VO classes (optional)
└── vo/ → response VO classes (optional)
controller/ → @RestController, delegates to service interface
service/ → Interface extends CrudService<Entity, Dto>
service/impl/ → Extends CrudServiceImpl<Dao, Entity, Dto>, implements getWrapper()
dao/ → Interface extends BaseDao / BaseMapper
entity/ → Extends BaseEntity, maps to DB table
dto/ → Request/response DTOs
excel/ → @ExcelProperty-annotated beans for EasyExcel import/export
```
Mapper XMLs: `src/main/resources/mapper/<domain>/**/*.xml`
Entity-to-DTO conversion uses `ConvertUtils.sourceToTarget()`.
### Conventions
## Security / Auth
- Lombok used throughout: `@Data`, `@AllArgsConstructor`, `@Slf4j` are standard on entity/service classes.
- DTO/Entity/VO separation per module — request DTOs often extend `BaseEntity`.
- **Framework**: Apache Shiro with Jakarta-compatible artifacts (classifier: `jakarta`)
- **Token flow**: Client sends `token` header/param -> `Oauth2Filter` extracts it -> `Oauth2Realm` validates against `sys_user_token` table -> returns `UserDetail` (user + dept scope + permissions)
- **Public paths** (no auth): `/login`, `/captcha`, `/druid/**`, `/doc.html`, `/swagger/**`, `/v3/api-docs/**`, `/favicon.ico`
- **Password hashing**: Custom `BCryptPasswordEncoder` with `PasswordUtils` (configurable rounds via sys_params)
- **Async context**: `UserContextHolder` (ThreadLocal) preserves user context across async task boundaries (used in file import, scheduled tasks). `SecurityUser.getUser()` falls back to this when Shiro Subject is unavailable.
## PK & Auth
## Data Scoping (Dept-Based Row-Level Security)
- PK: `ASSIGN_ID` (Snowflake) via `IdUtil.getSnowflakeNextId()`. All entities extend `BaseEntity`. Exception: `WeatherStationEntity` uses `AUTO_INCREMENT`.
- Auth: Apache Shiro 1.12 (**Jakarta classifier**) + OAuth2 token. Login → `token` header.
- **Do not introduce Spring Security** — the project uses Shiro exclusively.
`DataFilterInterceptor` is a MyBatis-Plus `InnerInterceptor`. When a service method is annotated with `@DataFilter`, the `DataFilterAspect` injects a `DataScope` object containing a SQL filter clause into the query parameters. The interceptor then appends this WHERE clause to the SQL, restricting results to the user's authorized departments.
## Key cross-cutting mechanisms
Example service usage:
```java
@DataFilter
public PageData<Dto> page(Map<String, Object> params) {
// SQL automatically filtered by dept scope
}
```
| Mechanism | How |
|---|---|
| **Data permissions** | `@DataFilter` on controller → `DataFilterAspect` → MyBatis interceptor injects dept-based SQL |
| **Auto-fill** | `FieldMetaObjectHandler` fills creator/date via MyBatis-Plus. **Only works with `insert()`/`updateById()`** — batch inserts (e.g. `insertBatchMultiRow`) bypass auto-fill; fields must be set manually. |
| **Scheduled jobs** | Quartz. `schedule_job` table, implements `ITask`, `@Component("beanName")`. Jobs auto-register at startup via `JobCommandLineRunner`. Seed data: `testTask` (paused, every 30 min), `fileScanTask` (paused, every 5 min), `weatherSummarizeCacheTask` (**active**, daily 1 AM). |
| **File scanning** | `WatchService` (primary, background thread) + Quartz fallback (`FileScanTask`) + startup runner (`FileScanStartupRunner`). Files identified by MD5 hash. |
| **Excel import** | EasyExcel + async dual-pass via `WeatherDataImportManager`. Progress tracked in-memory (`ConcurrentHashMap`), lost on restart. |
| **API responses** | Always wrapped in `Result`. Frontend expects `code === 0` for success. |
| **Validation** | Hibernate Validator. XSS filter via `XssFilter`. i18n messages in `system-common/src/main/resources/i18n/validation.properties`. |
## File Scanning Pipeline
## Exception handling
`FileWatchServiceManager` (`modules/weather/filescan/`) monitors `{FILE_SCAN_ROOT_PATH}/receive/` using `java.nio.file.WatchService`:
Single `@RestControllerAdvice` handler in the admin module:
1. **Directory layout**: `receive/<deptName>/`, `display/<deptName>/`, `archive/<deptName>/`
2. **Detection**: WatchService detects `ENTRY_CREATE` and `ENTRY_MODIFY` events
3. **Wait**: `waitForFileReady()` polls file size stability + file lock to ensure write completion
4. **Dedup**: MD5 hash check against `weather_file_scan_record` table
5. **Parse**: `FileNameParser` extracts region, category, period from filename
6. **Store**: Record inserted; file copied to `display/`; old version moved to `archive/`
7. **Root files**: Files in `receive/` root (no dept subdir) are classified as "model forecast" with `deptId = null`
| Handler | Catches | Persists errors? |
|---|---|---|
| `CustomExceptionHandler` | `CommonException`, `DuplicateKeyException`, `UnauthorizedException`, generic `Exception` | **Yes** — saves to `SysLogErrorService` (IP, user-agent, URI, params, stack trace) |
`FileScanStartupRunner` (ApplicationRunner) triggers full directory scan + WatchService registration on startup.
Both return a generic error for caught `Exception` (not the exception message). `CommonException` uses i18n message lookup via `MessageUtils.getMessage(code)`. Error codes follow `int` scheme: 5 digits, first 2 = module, last 3 = business (e.g. `10001`-`10029`).
## Weather Daily Data
## Logging
Core table: `weather_daily_data` (entity: `WeatherDailyDataEntity`). Fields: stationId, observeDate, avgTemp, maxTemp/minTemp (with time), rainfall (20-20 and 08-08), relativeHumidity, atmospheres, wind (avg/max/extreme speed + direction + time), deptId.
- Logback-spring config in `system-admin/src/main/resources/logback-spring.xml`. Logger names use `io.renren` (fork legacy), **not** `com.weather`.
- Admin dev profile enables MyBatis SQL stdout logging (`StdOutImpl`).
- When adding `@Slf4j` to `com.weather.*` classes, add a `com.weather` level override or change the existing `io.renren` logger scope.
### Import Flow
## Redis & Docs
`WeatherDataImportManager` handles Excel uploads:
1. Save MultipartFile to temp file
2. Submit async task (`CompletableFuture`), returns taskId immediately
3. First pass: `EasyExcel.read()` with count listener to get total rows
4. Second pass: `WeatherDataListener` reads rows in batches, calls `insertBatch()`
5. `insertBatch()` resolves station-to-dept mapping, fills audit fields, uses `insertBatchMultiRow()` (custom MySQL multi-row INSERT)
6. Clears weather summarize Redis cache on completion
7. Progress tracked in-memory (`ConcurrentHashMap<String, ImportProgress>`)
- Redis: `project-options.redis.open: true` in dev YAML controls whether `RedisAspect` intercepts Redis calls (default `false` — Redis operations silently skipped when disabled).
- Knife4j: disabled by default (`knife4j.enable: false`). Docs at `/doc.html` when enabled.
- `RedisAspect` wraps `@RedisCache` annotations with channel publish for cache invalidation.
### Summarize Caching
## MyBatis-Plus gotchas
`WeatherSummarizeCacheTask` (Quartz job) precomputes "same month-day across years" summary data per station into Redis. Cache key: `weather:summarize:{month}:{day}`. On cache hit, `getCachedSummarize()` filters by station + year range. On miss, falls back to `selectSummarizeByMonthDay()` DB query.
- **Batch inserts bypass auto-fill** — `FieldMetaObjectHandler` only fires on `insert()`/`updateById()`. Custom batch methods must manually set `creator`, `createDate`, `updater`, `updateDate`, `deptId`.
- Column names with special characters (e.g. `rain_20_20`) require explicit `@TableField` annotations — MyBatis-Plus cannot auto-map them from camelCase.
- `typeAliasesPackage: com.weather.modules.*.entity` — all entity classes must reside under a `modules` sub-package.
## Alert / Notification System
## Weather domain
- **Storage**: `SysAlertEntity` in `sys_alert` table
- **Real-time push**: `SseAlertService` manages active SSE connections (`CopyOnWriteArraySet<SseEmitter>`), broadcasts on Spring events:
- `AlertCreatedEvent` -> SSE event `alert`
- `AlertWithdrawnEvent` -> SSE event `alert-withdrawn`
- `AlertDeletedEvent` -> SSE event `alert-deleted`
- **External sources**: `AlertSourcePollingTask` (Quartz) polls external services; `AlertSourceCollector` gathers results
Three sub-modules under `system-admin/.../modules/weather/`:
## Job Scheduling (Quartz)
| Module | Purpose |
|---|---|
| `dailydata/` | Daily observations, Excel batch import (async dual-pass), EasyExcel listener, summary export |
| `station/` | Weather station CRUD, linked to dept via `dept_id` |
| `filescan/` | File monitoring + serving. Format: `<地区>地区-<指标>.png` / `<地区>地区631信息.txt` |
Tables: `schedule_job`, `schedule_job_log`, plus standard `qrtz_*` tables.
### Weather data import flow
- `ScheduleConfig`: Quartz `SchedulerFactoryBean` configuration
- `ScheduleJob` entity: bean class, cron expression, params, status (PAUSE/NORMAL)
- `ScheduleUtils`: Create/update/delete/pause/resume Quartz triggers
- `JobCommandLineRunner`: On startup, restores all NORMAL-status jobs from DB
- `ITask` interface: `run(String params)` method -- all job classes implement this
- Concurrency control via `@DisallowConcurrentExecution`
1. **First pass**: `AnalysisEventListener` counts total rows.
2. **Second pass**: `WeatherDataListener` processes with batch insert (2000 records/batch).
3. Progress tracked in-memory via `ConcurrentHashMap<String, ImportProgress>` (`volatile` fields + `AtomicInteger`).
4. Runs on `CompletableFuture` with manual `UserContextHolder` propagation for security context. (Note: `UserContextHolder` is in `system-admin/.../security/user/`, not the empty `system-common/.../holder/` package.)
5. On completion, clears Redis summary cache (`weather:summarize:*`).
## Database
### Weather summarize cache
Schema: `system-admin/db/weather_data_system.sql`
`WeatherSummarizeCacheTask` (Quartz job) pre-computes daily historical summaries into Redis. Uses **MySQL-specific** SQL functions (`MONTH()`, `DAY()` on `observe_date`). Cache key: `weather:summarize:{month}:{day}`, non-expiring. Service checks cache first for queries spanning same month/day across years.
### Core Tables
### Station priority ordering
`WeatherDailyDataServiceImpl.page()` uses custom `CASE WHEN` SQL to order stations belonging to the user's department + sub-departments first.
## System alert module
CRUD + polling notification system under `modules/sys/`:
| Layer | File |
|---|---|
| Entity | `entity/SysAlertEntity.java` — `sys_alert` table, Snowflake PK |
| DAO | `dao/SysAlertDao.java` + `mapper/sys/SysAlertDao.xml` — `selectActiveAlerts`, `selectActiveAlertsSince`, `countBySource` |
| DTO | `dto/SysAlertDTO.java` — `publishTime`/`createDate`/`updateDate` read-only |
| VO | `vo/SysAlertVO.java` — `id` as String (mapped from Long) |
| Service | `service/SysAlertService.java` + `impl/` — `publishIfNotExists` (dedup by source), `withdraw` (soft), CRUD |
| Controller | `controller/SysAlertController.java` — polling: `GET active`, `GET active/since` (no auth); CRUD: `page`/`get`/`save`/`update`/`delete`/`withdraw` (Shiro permissions) |
| Enums | `enums/AlertLevelEnum.java` (info/warning/danger), `enums/AlertSourceTypeEnum.java` (manual/import/schedule/datasource) |
### Data source extension
| File | Role |
|---|---|
| `alert/AlertSource.java` | SPI interface: `getName()`, `check()` returning `AlertMessage` records |
| `alert/AlertSourceCollector.java` | `@Autowired(required=false) List<AlertSource>` auto-discovery, calls `publishIfNotExists` for each |
| `alert/task/AlertSourcePollingTask.java` | Quartz job (`alertSourcePollingTask`), default paused in `schedule_job` |
To add a new alert source: implement `AlertSource`, register as `@Component`, enable the polling task.
| Table | Purpose |
|-------|---------|
| `sys_user`, `sys_role`, `sys_menu`, `sys_dept` | RBAC |
| `sys_role_user`, `sys_role_menu`, `sys_role_data_scope` | RBAC associations |
| `sys_user_token` | Shiro auth tokens |
| `sys_dict_type`, `sys_dict_data` | Dictionary system |
| `sys_params` | Key-value system parameters |
| `sys_alert` | Alert/notification records |
| `weather_station` | Weather station registry |
| `weather_daily_data` | Daily meteorological observations |
| `weather_file_scan_record` | File tracking (receive -> display -> archive) |
| `sys_region` | Geographic region tree (province/city/county) |
| `schedule_job`, `schedule_job_log` | Quartz job definitions and execution history |
| `qrtz_*` | Quartz internal scheduler tables |
| `sys_log_operation`, `sys_log_login`, `sys_log_error` | Audit logs |
| `sys_oss` | Cloud storage object records |