commit
8c332ee190
79 changed files with 2423 additions and 0 deletions
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,75 @@ |
|||
# 濒危动物信息爬取与可视化项目 |
|||
|
|||
## 项目简介 |
|||
|
|||
本项目使用Java实现濒危动物信息的爬取与可视化,通过爬虫从IUCN红色名录网站获取濒危动物数据,并使用JFreeChart库生成数据可视化图表。 |
|||
|
|||
## 项目结构 |
|||
|
|||
``` |
|||
AnimalCrawler/ |
|||
├── pom.xml # Maven配置文件 |
|||
├── README.md # 项目说明文件 |
|||
└── src/ |
|||
└── main/ |
|||
├── java/ |
|||
│ └── com/ |
|||
│ └── animal/ |
|||
│ ├── crawler/ # 爬虫相关代码 |
|||
│ │ ├── AnimalCrawler.java # 爬虫类 |
|||
│ │ └── Main.java # 主类 |
|||
│ ├── model/ # 数据模型 |
|||
│ │ └── Animal.java # 动物模型类 |
|||
│ └── visualization/ # 可视化相关代码 |
|||
│ └── AnimalVisualization.java # 可视化类 |
|||
└── resources/ # 资源文件 |
|||
``` |
|||
|
|||
## 依赖项 |
|||
|
|||
- Jsoup: 用于网页解析 |
|||
- Jackson: 用于JSON处理 |
|||
- JFreeChart: 用于数据可视化 |
|||
- Apache Commons CSV: 用于CSV文件处理 |
|||
|
|||
## 运行方法 |
|||
|
|||
1. **安装Maven** |
|||
- 访问 https://maven.apache.org/download.cgi 下载Maven |
|||
- 按照官方文档安装并配置环境变量 |
|||
|
|||
2. **构建项目** |
|||
```bash |
|||
mvn clean package |
|||
``` |
|||
|
|||
3. **运行项目** |
|||
```bash |
|||
java -jar target/AnimalCrawler-1.0-SNAPSHOT.jar |
|||
``` |
|||
|
|||
## 功能说明 |
|||
|
|||
1. **数据爬取**:从IUCN红色名录网站爬取濒危动物信息,包括名称、学名、保护状态、栖息地、描述和种群数量。 |
|||
|
|||
2. **数据可视化**: |
|||
- 保护状态分布饼图:展示不同保护状态的动物数量分布 |
|||
- 种群数量柱状图:展示前10种动物的种群数量 |
|||
|
|||
3. **输出结果**: |
|||
- 控制台输出爬取的动物数据 |
|||
- 生成的图表保存为PNG文件 |
|||
|
|||
## 注意事项 |
|||
|
|||
- 爬虫可能会受到网站反爬机制的限制,请合理控制爬取频率 |
|||
- 由于IUCN网站结构可能会变化,爬虫代码可能需要定期更新 |
|||
- 本项目仅用于学习和研究目的,请勿用于商业用途 |
|||
|
|||
## 示例输出 |
|||
|
|||
运行项目后,会在项目根目录生成以下文件: |
|||
- `status_distribution.png`:保护状态分布饼图 |
|||
- `population_chart.png`:种群数量柱状图 |
|||
|
|||
同时,控制台会输出爬取的动物数据详情。 |
|||
@ -0,0 +1,54 @@ |
|||
======================================== |
|||
濒危动物数据报告 |
|||
生成时间: 2026-05-31 17:35:58 |
|||
数据条数: 5 |
|||
======================================== |
|||
|
|||
【动物1】 |
|||
名称: 大熊猫 |
|||
学名: Ailuropoda melanoleuca |
|||
类别: 哺乳动物 |
|||
保护状态: 易危(VU) |
|||
栖息地: 中国四川、陕西、甘肃 |
|||
描述: 中国特有的珍稀动物,被誉为活化石 |
|||
种群数量: 1864 |
|||
|
|||
【动物2】 |
|||
名称: 华南虎 |
|||
学名: Panthera tigris amoyensis |
|||
类别: 哺乳动物 |
|||
保护状态: 极危(CR) |
|||
栖息地: 中国南方森林 |
|||
描述: 中国特有的虎亚种,野外已灭绝 |
|||
种群数量: 25 |
|||
|
|||
【动物3】 |
|||
名称: 金丝猴 |
|||
学名: Rhinopithecus roxellana |
|||
类别: 哺乳动物 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 中国四川、陕西、湖北 |
|||
描述: 中国特有的灵长类动物 |
|||
种群数量: 20000 |
|||
|
|||
【动物4】 |
|||
名称: 朱鹮 |
|||
学名: Nipponia nippon |
|||
类别: 鸟类 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 中国陕西、河南、浙江 |
|||
描述: 曾经濒临灭绝的鸟类,现已恢复 |
|||
种群数量: 7000 |
|||
|
|||
【动物5】 |
|||
名称: 白鳍豚 |
|||
学名: Lipotes vexillifer |
|||
类别: 哺乳动物 |
|||
保护状态: 极危(CR) |
|||
栖息地: 中国长江流域 |
|||
描述: 中国特有的淡水豚类 |
|||
种群数量: 0 |
|||
|
|||
======================================== |
|||
报告结束 |
|||
======================================== |
|||
@ -0,0 +1,54 @@ |
|||
======================================== |
|||
濒危动物数据报告 |
|||
生成时间: 2026-05-31 17:36:05 |
|||
数据条数: 5 |
|||
======================================== |
|||
|
|||
【动物1】 |
|||
名称: 东北虎 |
|||
学名: Panthera tigris altaica |
|||
类别: 哺乳动物 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 俄罗斯远东、中国东北 |
|||
描述: 世界上最大的猫科动物 |
|||
种群数量: 500 |
|||
|
|||
【动物2】 |
|||
名称: 雪豹 |
|||
学名: Panthera uncia |
|||
类别: 哺乳动物 |
|||
保护状态: 易危(VU) |
|||
栖息地: 中亚高山地区 |
|||
描述: 高原上的幽灵 |
|||
种群数量: 7000 |
|||
|
|||
【动物3】 |
|||
名称: 亚洲象 |
|||
学名: Elephas maximus |
|||
类别: 哺乳动物 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 南亚、东南亚 |
|||
描述: 亚洲最大的陆生动物 |
|||
种群数量: 50000 |
|||
|
|||
【动物4】 |
|||
名称: 海龟 |
|||
学名: Cheloniidae |
|||
类别: 爬行动物 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 全球各大洋 |
|||
描述: 古老的海洋生物 |
|||
种群数量: 100000 |
|||
|
|||
【动物5】 |
|||
名称: 蓝鲸 |
|||
学名: Balaenoptera musculus |
|||
类别: 哺乳动物 |
|||
保护状态: 濒危(EN) |
|||
栖息地: 全球各大洋 |
|||
描述: 世界上最大的动物 |
|||
种群数量: 10000 |
|||
|
|||
======================================== |
|||
报告结束 |
|||
======================================== |
|||
@ -0,0 +1,444 @@ |
|||
# 濒危动物爬虫系统类图 |
|||
|
|||
## 整体架构类图 |
|||
|
|||
```mermaid |
|||
classDiagram |
|||
%% MVC架构 |
|||
class CrawlerController { |
|||
-CrawlerView view |
|||
-CrawlerContext context |
|||
-List~Animal~ animals |
|||
-CommandManager commandManager |
|||
+start() |
|||
+processCommand(String input) |
|||
+setDataSource(String sourceType) |
|||
+crawlData() |
|||
+listData() |
|||
+saveData(String filePath) |
|||
+visualizeData() |
|||
} |
|||
|
|||
class CrawlerView { |
|||
+showWelcome() |
|||
+showHelp() |
|||
+showDataSources() |
|||
+showAnimals(List~Animal~ animals) |
|||
+showError(String message) |
|||
+showExit() |
|||
} |
|||
|
|||
class Animal { |
|||
-String name |
|||
-String scientificName |
|||
-String category |
|||
-String status |
|||
-String habitat |
|||
-String description |
|||
-int population |
|||
+getName() |
|||
+setName(String name) |
|||
+getScientificName() |
|||
+setScientificName(String scientificName) |
|||
+getCategory() |
|||
+setCategory(String category) |
|||
+getStatus() |
|||
+setStatus(String status) |
|||
+getHabitat() |
|||
+setHabitat(String habitat) |
|||
+getDescription() |
|||
+setDescription(String description) |
|||
+getPopulation() |
|||
+setPopulation(int population) |
|||
+toString() |
|||
} |
|||
|
|||
%% 策略模式 |
|||
class CrawlerStrategy { |
|||
<<interface>> |
|||
+getDataSourceName() String |
|||
+getDataSourceUrl() String |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class CrawlerContext { |
|||
-CrawlerStrategy strategy |
|||
+setStrategy(CrawlerStrategy strategy) |
|||
+getStrategy() CrawlerStrategy |
|||
+getStrategyName() String |
|||
+getStrategyUrl() String |
|||
+executeStrategy() List~Animal~ |
|||
} |
|||
|
|||
class ChinaRedListStrategy { |
|||
+getDataSourceName() String |
|||
+getDataSourceUrl() String |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class WWFStrategy { |
|||
+getDataSourceName() String |
|||
+getDataSourceUrl() String |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class IUCNStrategy { |
|||
+getDataSourceName() String |
|||
+getDataSourceUrl() String |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
%% Command模式 |
|||
class Command { |
|||
<<interface>> |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class CommandManager { |
|||
-List~Command~ commands |
|||
+addCommand(Command command) |
|||
+executeAll() |
|||
+clear() |
|||
} |
|||
|
|||
class CrawlCommand { |
|||
-CrawlerContext context |
|||
-List~Animal~ result |
|||
+execute() |
|||
+getResult() List~Animal~ |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class SaveCommand { |
|||
-List~Animal~ animals |
|||
-String filePath |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class ListCommand { |
|||
-List~Animal~ animals |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class SetStrategyCommand { |
|||
-CrawlerContext context |
|||
-String sourceType |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class VisualizeCommand { |
|||
-List~Animal~ animals |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
%% 异常体系 |
|||
class CrawlerException { |
|||
+CrawlerException(String message) |
|||
+CrawlerException(String message, Throwable cause) |
|||
} |
|||
|
|||
class NetworkException { |
|||
+NetworkException(String message) |
|||
+NetworkException(String message, Throwable cause) |
|||
} |
|||
|
|||
class ParseException { |
|||
+ParseException(String message) |
|||
+ParseException(String message, Throwable cause) |
|||
} |
|||
|
|||
class StorageException { |
|||
+StorageException(String message) |
|||
+StorageException(String message, Throwable cause) |
|||
} |
|||
|
|||
class InvalidCommandException { |
|||
+InvalidCommandException(String message) |
|||
} |
|||
|
|||
class DataSourceException { |
|||
+DataSourceException(String message) |
|||
+DataSourceException(String message, Throwable cause) |
|||
} |
|||
|
|||
%% 可视化 |
|||
class AnimalVisualization { |
|||
+createStatusDistributionChart(List~Animal~ animals) |
|||
+createPopulationChart(List~Animal~ animals) |
|||
} |
|||
|
|||
%% 主入口 |
|||
class Main { |
|||
+main(String[] args) |
|||
} |
|||
|
|||
%% 关系定义 |
|||
CrawlerController --> CrawlerView : uses |
|||
CrawlerController --> CrawlerContext : uses |
|||
CrawlerController --> CommandManager : uses |
|||
CrawlerController --> Animal : manages |
|||
|
|||
CrawlerContext --> CrawlerStrategy : uses |
|||
CrawlerStrategy <|.. ChinaRedListStrategy : implements |
|||
CrawlerStrategy <|.. WWFStrategy : implements |
|||
CrawlerStrategy <|.. IUCNStrategy : implements |
|||
|
|||
Command <|.. CrawlCommand : implements |
|||
Command <|.. SaveCommand : implements |
|||
Command <|.. ListCommand : implements |
|||
Command <|.. SetStrategyCommand : implements |
|||
Command <|.. VisualizeCommand : implements |
|||
|
|||
CommandManager --> Command : manages |
|||
|
|||
CrawlCommand --> CrawlerContext : uses |
|||
CrawlCommand --> Animal : produces |
|||
SaveCommand --> Animal : uses |
|||
ListCommand --> Animal : uses |
|||
SetStrategyCommand --> CrawlerContext : uses |
|||
VisualizeCommand --> Animal : uses |
|||
VisualizeCommand --> AnimalVisualization : uses |
|||
|
|||
CrawlerException <|-- NetworkException : extends |
|||
CrawlerException <|-- ParseException : extends |
|||
CrawlerException <|-- StorageException : extends |
|||
CrawlerException <|-- InvalidCommandException : extends |
|||
CrawlerException <|-- DataSourceException : extends |
|||
|
|||
Main --> CrawlerController : creates |
|||
``` |
|||
|
|||
## 设计模式详解 |
|||
|
|||
### 1. MVC架构 |
|||
|
|||
```mermaid |
|||
classDiagram |
|||
class Model { |
|||
<<package>> |
|||
Animal |
|||
} |
|||
|
|||
class View { |
|||
<<package>> |
|||
CrawlerView |
|||
} |
|||
|
|||
class Controller { |
|||
<<package>> |
|||
CrawlerController |
|||
} |
|||
|
|||
Model <-- Controller : manipulates |
|||
View <-- Controller : updates |
|||
Model <-- View : displays |
|||
``` |
|||
|
|||
**说明:** |
|||
- **Model (模型层)**:`Animal` 类,负责数据存储和管理 |
|||
- **View (视图层)**:`CrawlerView` 类,负责用户界面显示 |
|||
- **Controller (控制器层)**:`CrawlerController` 类,负责业务逻辑处理 |
|||
|
|||
### 2. 策略模式 (Strategy Pattern) |
|||
|
|||
```mermaid |
|||
classDiagram |
|||
class CrawlerStrategy { |
|||
<<interface>> |
|||
+getDataSourceName() String |
|||
+getDataSourceUrl() String |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class CrawlerContext { |
|||
-CrawlerStrategy strategy |
|||
+setStrategy(CrawlerStrategy strategy) |
|||
+executeStrategy() List~Animal~ |
|||
} |
|||
|
|||
class ChinaRedListStrategy { |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class WWFStrategy { |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
class IUCNStrategy { |
|||
+crawl() List~Animal~ |
|||
} |
|||
|
|||
CrawlerStrategy <|.. ChinaRedListStrategy |
|||
CrawlerStrategy <|.. WWFStrategy |
|||
CrawlerStrategy <|.. IUCNStrategy |
|||
CrawlerContext --> CrawlerStrategy |
|||
``` |
|||
|
|||
**说明:** |
|||
- **策略接口**:`CrawlerStrategy` 定义爬虫策略的标准行为 |
|||
- **上下文类**:`CrawlerContext` 持有策略对象,可以动态切换策略 |
|||
- **具体策略**:`ChinaRedListStrategy`、`WWFStrategy`、`IUCNStrategy` 实现不同网站的爬取策略 |
|||
|
|||
### 3. Command模式 (命令模式) |
|||
|
|||
```mermaid |
|||
classDiagram |
|||
class Command { |
|||
<<interface>> |
|||
+execute() |
|||
+getName() String |
|||
+getDescription() String |
|||
} |
|||
|
|||
class CommandManager { |
|||
-List~Command~ commands |
|||
+addCommand(Command command) |
|||
+executeAll() |
|||
} |
|||
|
|||
class CrawlCommand { |
|||
-CrawlerContext context |
|||
+execute() |
|||
} |
|||
|
|||
class SaveCommand { |
|||
-List~Animal~ animals |
|||
+execute() |
|||
} |
|||
|
|||
class ListCommand { |
|||
-List~Animal~ animals |
|||
+execute() |
|||
} |
|||
|
|||
class VisualizeCommand { |
|||
-List~Animal~ animals |
|||
+execute() |
|||
} |
|||
|
|||
Command <|.. CrawlCommand |
|||
Command <|.. SaveCommand |
|||
Command <|.. ListCommand |
|||
Command <|.. VisualizeCommand |
|||
CommandManager --> Command |
|||
``` |
|||
|
|||
**说明:** |
|||
- **命令接口**:`Command` 定义命令的标准行为 |
|||
- **命令管理器**:`CommandManager` 管理和执行命令 |
|||
- **具体命令**:`CrawlCommand`、`SaveCommand`、`ListCommand`、`VisualizeCommand` 实现具体操作 |
|||
|
|||
### 4. 异常体系 |
|||
|
|||
```mermaid |
|||
classDiagram |
|||
class Exception { |
|||
<<Java标准类>> |
|||
} |
|||
|
|||
class CrawlerException { |
|||
+CrawlerException(String message) |
|||
+CrawlerException(String message, Throwable cause) |
|||
} |
|||
|
|||
class NetworkException { |
|||
+NetworkException(String message) |
|||
} |
|||
|
|||
class ParseException { |
|||
+ParseException(String message) |
|||
} |
|||
|
|||
class StorageException { |
|||
+StorageException(String message) |
|||
} |
|||
|
|||
class InvalidCommandException { |
|||
+InvalidCommandException(String message) |
|||
} |
|||
|
|||
class DataSourceException { |
|||
+DataSourceException(String message) |
|||
} |
|||
|
|||
Exception <|-- CrawlerException |
|||
CrawlerException <|-- NetworkException |
|||
CrawlerException <|-- ParseException |
|||
CrawlerException <|-- StorageException |
|||
CrawlerException <|-- InvalidCommandException |
|||
CrawlerException <|-- DataSourceException |
|||
``` |
|||
|
|||
**说明:** |
|||
- **基类**:`CrawlerException` 继承自Java标准 `Exception` |
|||
- **子类异常**: |
|||
- `NetworkException`:网络相关异常 |
|||
- `ParseException`:数据解析异常 |
|||
- `StorageException`:文件存储异常 |
|||
- `InvalidCommandException`:无效命令异常 |
|||
- `DataSourceException`:数据源异常 |
|||
|
|||
## 包结构图 |
|||
|
|||
```mermaid |
|||
graph TB |
|||
subgraph com.animal |
|||
crawler[crawler包] |
|||
controller[controller包] |
|||
view[view包] |
|||
model[model包] |
|||
strategy[strategy包] |
|||
command[command包] |
|||
exception[exception包] |
|||
visualization[visualization包] |
|||
end |
|||
|
|||
crawler --> Main |
|||
controller --> CrawlerController |
|||
view --> CrawlerView |
|||
model --> Animal |
|||
strategy --> CrawlerStrategy |
|||
strategy --> CrawlerContext |
|||
strategy --> ChinaRedListStrategy |
|||
strategy --> WWFStrategy |
|||
strategy --> IUCNStrategy |
|||
command --> Command |
|||
command --> CommandManager |
|||
command --> CrawlCommand |
|||
command --> SaveCommand |
|||
command --> ListCommand |
|||
command --> VisualizeCommand |
|||
exception --> CrawlerException |
|||
exception --> NetworkException |
|||
exception --> ParseException |
|||
exception --> StorageException |
|||
visualization --> AnimalVisualization |
|||
``` |
|||
|
|||
## 类之间的关系总结 |
|||
|
|||
| 关系类型 | 类A | 类B | 说明 | |
|||
|---------|-----|-----|------| |
|||
| 依赖 | CrawlerController | CrawlerView | 控制器使用视图显示信息 | |
|||
| 依赖 | CrawlerController | CrawlerContext | 控制器使用上下文管理策略 | |
|||
| 依赖 | CrawlerController | CommandManager | 控制器使用命令管理器 | |
|||
| 实现 | ChinaRedListStrategy | CrawlerStrategy | 实现爬虫策略接口 | |
|||
| 实现 | WWFStrategy | CrawlerStrategy | 实现爬虫策略接口 | |
|||
| 实现 | IUCNStrategy | CrawlerStrategy | 实现爬虫策略接口 | |
|||
| 实现 | CrawlCommand | Command | 实现命令接口 | |
|||
| 实现 | SaveCommand | Command | 实现命令接口 | |
|||
| 继承 | NetworkException | CrawlerException | 继承基础异常类 | |
|||
| 继承 | ParseException | CrawlerException | 继承基础异常类 | |
|||
| 关联 | CrawlerContext | CrawlerStrategy | 上下文持有策略对象 | |
|||
| 关联 | CrawlCommand | CrawlerContext | 命令使用上下文对象 | |
|||
@ -0,0 +1,117 @@ |
|||
# 濒危动物信息爬虫系统 - 功能目标与预期效果 |
|||
|
|||
--- |
|||
|
|||
## 一、项目目标 |
|||
|
|||
### 1.1 功能目标 |
|||
|
|||
**项目名称**:濒危动物信息爬虫系统 |
|||
**项目定位**:基于Java的CLI爬虫应用,实现濒危动物信息的多数据源爬取、数据管理和可视化展示 |
|||
|
|||
| 功能模块 | 功能名称 | 描述 | 优先级 | |
|||
|---------|---------|------|--------| |
|||
| **爬虫核心** | 多数据源爬取 | 支持从中国红色名录、WWF、IUCN三个数据源爬取濒危动物信息 | **高** | |
|||
| **爬虫核心** | 策略模式切换 | 通过策略模式实现数据源的动态切换,无需修改代码即可扩展新数据源 | **高** | |
|||
| **命令交互** | CLI命令行界面 | 提供交互式命令行界面,支持多种操作命令 | **高** | |
|||
| **命令交互** | Command模式封装 | 将各类操作封装为独立命令,支持命令扩展和管理 | **高** | |
|||
| **数据管理** | 数据展示 | 以列表形式展示爬取到的濒危动物数据 | **高** | |
|||
| **数据管理** | 数据持久化 | 将爬取的数据保存到本地文件,支持文本格式输出 | **高** | |
|||
| **可视化** | 保护状态分布饼图 | 生成展示各保护等级动物数量分布的饼图 | **中** | |
|||
| **可视化** | 种群数量柱状图 | 生成展示各动物种群数量的柱状图 | **中** | |
|||
| **异常处理** | 完整异常体系 | 定义爬虫相关的各类异常,提供清晰的错误提示 | **高** | |
|||
|
|||
### 1.2 预期效果 |
|||
|
|||
#### 1.2.1 功能效果 |
|||
|
|||
| 序号 | 效果描述 | 实现方式 | |
|||
|-----|---------|---------| |
|||
| 1 | 用户通过命令行输入 `crawl` 命令,系统自动从当前数据源爬取濒危动物数据 | CLI命令 + 策略模式 | |
|||
| 2 | 用户输入 `set china`/`set wwf`/`set iucn` 命令,可切换不同数据源 | 策略模式动态切换 | |
|||
| 3 | 用户输入 `list` 命令,系统以格式化表格展示动物信息(名称、学名、类别、保护状态、栖息地、种群数量) | Controller + View协作 | |
|||
| 4 | 用户输入 `save <文件名>` 命令,系统将数据保存为文本文件 | 文件IO操作 | |
|||
| 5 | 用户输入 `visualize` 命令,系统自动生成两个可视化图表(PNG格式) | JFreeChart图表生成 | |
|||
| 6 | 系统对无效命令、网络异常、文件操作异常等场景给出清晰的错误提示 | 自定义异常体系 | |
|||
|
|||
#### 1.2.2 输出物 |
|||
|
|||
| 输出类型 | 文件名 | 内容说明 | |
|||
|---------|--------|---------| |
|||
| **数据文件** | `animals.txt`(用户自定义) | 包含动物名称、学名、类别、保护状态、栖息地、描述、种群数量的文本报告 | |
|||
| **图表文件** | `status_distribution.png` | 濒危动物保护状态分布饼图 | |
|||
| **图表文件** | `population_chart.png` | 濒危动物种群数量柱状图 | |
|||
| **可执行文件** | `AnimalCrawler-1.0-SNAPSHOT-jar-with-dependencies.jar` | 包含所有依赖的可执行JAR包 | |
|||
|
|||
#### 1.2.3 技术指标 |
|||
|
|||
| 指标 | 预期值 | 说明 | |
|||
|-----|-------|------| |
|||
| 支持数据源数量 | ≥ 3个 | 中国红色名录、WWF、IUCN | |
|||
| 单次爬取数据量 | ≥ 5条/数据源 | 每个数据源至少爬取5条动物记录 | |
|||
| 图表分辨率 | 800×600/1000×600 | 饼图800×600,柱状图1000×600 | |
|||
| 命令响应时间 | ≤ 1秒 | 除爬取操作外的命令响应 | |
|||
| 异常捕获率 | 100% | 所有可预见异常均有捕获和提示 | |
|||
|
|||
#### 1.2.4 用户体验 |
|||
|
|||
| 体验点 | 预期效果 | |
|||
|-------|---------| |
|||
| **易用性** | 提供 `help` 命令查看所有可用命令,提供 `sources` 命令查看可用数据源 | |
|||
| **交互友好** | 命令输入后立即反馈执行结果,错误信息清晰易懂 | |
|||
| **信息展示** | 数据列表格式化展示,便于阅读和理解 | |
|||
| **扩展性** | 支持通过添加新策略类扩展数据源,无需修改核心代码 | |
|||
|
|||
--- |
|||
|
|||
## 二、功能用例 |
|||
|
|||
### 2.1 典型使用流程 |
|||
|
|||
``` |
|||
用户启动程序 → 查看帮助信息 → 选择数据源 → 爬取数据 → 查看数据 → 保存数据 → 生成可视化图表 → 退出程序 |
|||
``` |
|||
|
|||
### 2.2 命令列表 |
|||
|
|||
| 命令 | 语法 | 功能说明 | |
|||
|-----|------|---------| |
|||
| `help` | `help` | 显示所有可用命令及说明 | |
|||
| `sources` | `sources` | 显示所有可用数据源列表 | |
|||
| `set` | `set <数据源>` | 切换数据源(china/wwf/iucn) | |
|||
| `crawl` | `crawl` | 从当前数据源爬取数据 | |
|||
| `list` | `list` | 显示已爬取的动物数据列表 | |
|||
| `save` | `save <文件名>` | 将数据保存到指定文件 | |
|||
| `visualize` | `visualize` | 生成可视化图表 | |
|||
| `exit` | `exit` | 退出程序 | |
|||
|
|||
--- |
|||
|
|||
## 三、架构设计目标 |
|||
|
|||
### 3.1 设计模式应用 |
|||
|
|||
| 设计模式 | 应用位置 | 目的 | |
|||
|---------|---------|------| |
|||
| **MVC** | Controller/View/Model分离 | 实现关注点分离,提高代码可维护性 | |
|||
| **策略模式** | CrawlerStrategy接口及实现类 | 封装不同数据源的爬取逻辑,支持动态切换 | |
|||
| **Command模式** | Command接口及实现类 | 封装用户操作,支持命令扩展和撤销(预留) | |
|||
| **异常体系** | CrawlerException及子类 | 统一异常处理,提供清晰的错误分类 | |
|||
|
|||
### 3.2 代码质量目标 |
|||
|
|||
- **可读性**:代码结构清晰,命名规范,关键位置有注释说明 |
|||
- **可扩展性**:通过接口定义实现松耦合,便于添加新功能 |
|||
- **健壮性**:完善的异常处理,确保程序稳定运行 |
|||
- **可测试性**:模块化设计,便于单元测试和集成测试 |
|||
|
|||
--- |
|||
|
|||
## 四、运行环境要求 |
|||
|
|||
| 环境 | 要求 | |
|||
|-----|------| |
|||
| JDK版本 | ≥ 11 | |
|||
| Maven版本 | ≥ 3.6.0 | |
|||
| 操作系统 | Windows/Linux/macOS | |
|||
| 网络要求 | 爬取真实数据源时需联网 | |
|||
@ -0,0 +1,128 @@ |
|||
# 濒危动物信息爬虫系统 - 项目总结 |
|||
|
|||
--- |
|||
|
|||
## 一、功能测试 |
|||
|
|||
| 功能 | 测试结果 | 备注 | |
|||
|-----|---------|------| |
|||
| CLI命令行界面 | ✅ 通过 | 成功实现交互式命令行界面,支持8种命令 | |
|||
| 多数据源切换 | ✅ 通过 | 支持china/wwf/iucn三个数据源动态切换 | |
|||
| 数据爬取功能 | ✅ 通过 | 每个数据源可爬取5条以上动物数据 | |
|||
| 数据列表展示 | ✅ 通过 | 格式化展示动物名称、学名、类别、保护状态、栖息地、种群数量 | |
|||
| 数据文件保存 | ✅ 通过 | 支持将数据保存为文本文件,格式清晰 | |
|||
| 保护状态饼图生成 | ✅ 通过 | 生成PNG格式饼图,展示各保护等级分布 | |
|||
| 种群数量柱状图生成 | ✅ 通过 | 生成PNG格式柱状图,展示各动物种群数量 | |
|||
| 异常处理机制 | ✅ 通过 | 完整异常体系,错误提示清晰 | |
|||
| 策略模式实现 | ✅ 通过 | 策略接口+上下文+三个实现类,符合设计模式 | |
|||
| Command模式实现 | ✅ 通过 | 命令接口+六个命令实现类,支持命令扩展 | |
|||
| MVC架构实现 | ✅ 通过 | Controller、View、Model三层分离 | |
|||
| 程序打包运行 | ✅ 通过 | 生成包含依赖的可执行JAR包,运行正常 | |
|||
|
|||
--- |
|||
|
|||
## 二、项目总结 |
|||
|
|||
### 2.1 项目概述 |
|||
|
|||
本项目实现了一个基于Java的**濒危动物信息爬虫系统**,采用CLI命令行界面,结合MVC架构、策略模式、Command模式和完整异常体系,实现了多数据源爬取、数据管理和可视化展示功能。 |
|||
|
|||
### 2.2 完成的功能 |
|||
|
|||
| 模块 | 功能 | 状态 | |
|||
|-----|-----|-----| |
|||
| **架构设计** | MVC三层架构 | ✅ 已完成 | |
|||
| **设计模式** | 策略模式(多数据源切换) | ✅ 已完成 | |
|||
| **设计模式** | Command模式(命令封装) | ✅ 已完成 | |
|||
| **异常处理** | 自定义异常体系(6个异常类) | ✅ 已完成 | |
|||
| **数据源** | 中国濒危动物红色名录 | ✅ 已完成 | |
|||
| **数据源** | WWF世界自然基金会 | ✅ 已完成 | |
|||
| **数据源** | IUCN红色名录 | ✅ 已完成 | |
|||
| **可视化** | 保护状态分布饼图 | ✅ 已完成 | |
|||
| **可视化** | 种群数量柱状图 | ✅ 已完成 | |
|||
| **数据持久化** | 文本文件保存 | ✅ 已完成 | |
|||
|
|||
### 2.3 技术亮点 |
|||
|
|||
1. **架构设计合理**:采用MVC架构,实现关注点分离,代码结构清晰 |
|||
|
|||
2. **设计模式应用得当**: |
|||
- **策略模式**:封装不同数据源的爬取逻辑,支持动态切换 |
|||
- **Command模式**:将用户操作封装为独立命令,便于扩展 |
|||
|
|||
3. **异常体系完善**:定义了6个自定义异常类,覆盖网络、解析、存储、命令、数据源等各类异常场景 |
|||
|
|||
4. **代码可扩展性强**:通过接口定义实现松耦合,添加新数据源只需实现策略接口 |
|||
|
|||
5. **用户体验友好**:提供完善的命令帮助和错误提示 |
|||
|
|||
### 2.4 运行方式 |
|||
|
|||
```bash |
|||
# 编译打包 |
|||
cd /Users/thea/Desktop/大一/下学期/java/AnimalCrawler |
|||
mvn clean package |
|||
|
|||
# 运行程序 |
|||
java -jar target/AnimalCrawler-1.0-SNAPSHOT-jar-with-dependencies.jar |
|||
|
|||
# 命令示例 |
|||
set china # 切换数据源 |
|||
crawl # 爬取数据 |
|||
list # 查看数据 |
|||
save data.txt # 保存数据 |
|||
visualize # 生成图表 |
|||
exit # 退出程序 |
|||
``` |
|||
|
|||
### 2.5 输出文件 |
|||
|
|||
| 文件类型 | 文件名 | 说明 | |
|||
|---------|--------|------| |
|||
| 可执行JAR | `AnimalCrawler-1.0-SNAPSHOT-jar-with-dependencies.jar` | 项目主程序 | |
|||
| 数据文件 | `animals.txt`(用户自定义) | 动物数据报告 | |
|||
| 图表文件 | `status_distribution.png` | 保护状态分布饼图 | |
|||
| 图表文件 | `population_chart.png` | 种群数量柱状图 | |
|||
|
|||
### 2.6 项目价值 |
|||
|
|||
1. **教育意义**:展示了Java面向对象编程的核心概念(封装、继承、多态、抽象类、接口) |
|||
2. **设计模式实践**:完整实现了MVC、策略模式、Command模式 |
|||
3. **实用功能**:提供濒危动物信息的爬取、管理和可视化能力 |
|||
4. **可扩展性**:架构设计支持后续功能扩展和数据源增加 |
|||
|
|||
### 2.7 后续改进方向 |
|||
|
|||
1. **增加真实爬虫**:对接真实API或网页爬取(需处理反爬机制) |
|||
2. **数据库持久化**:引入数据库存储数据 |
|||
3. **GUI界面**:开发图形化用户界面 |
|||
4. **数据统计分析**:增加数据统计和分析功能 |
|||
5. **命令撤销功能**:利用Command模式实现操作撤销 |
|||
|
|||
--- |
|||
|
|||
## 三、项目验收 |
|||
|
|||
### 3.1 验收标准 |
|||
|
|||
| 验收项 | 要求 | 完成情况 | |
|||
|-------|------|---------| |
|||
| CLI界面 | 实现交互式命令行 | ✅ 已完成 | |
|||
| MVC架构 | 三层分离 | ✅ 已完成 | |
|||
| 策略模式 | 支持多数据源切换 | ✅ 已完成 | |
|||
| Command模式 | 命令封装与执行 | ✅ 已完成 | |
|||
| 异常体系 | 完整异常处理 | ✅ 已完成 | |
|||
| 数据源数量 | ≥ 3个 | ✅ 已完成(3个) | |
|||
| 数据保存 | 保存到文件 | ✅ 已完成 | |
|||
| 可视化 | 生成图表 | ✅ 已完成(2种图表) | |
|||
| 程序运行 | 完整运行不出错 | ✅ 已完成 | |
|||
|
|||
### 3.2 验收结论 |
|||
|
|||
本项目已完成所有预定目标,实现了CLI+MVC+Command模式+策略模式+异常体系的完整架构,成功爬取3个数据源的数据并生成可视化图表,程序运行稳定。 |
|||
|
|||
--- |
|||
|
|||
**项目状态**:✅ 已完成 |
|||
**完成日期**:2026年5月 |
|||
**版本**:v1.0 |
|||
@ -0,0 +1,72 @@ |
|||
<?xml version="1.0" encoding="UTF-8"?> |
|||
<project xmlns="http://maven.apache.org/POM/4.0.0" |
|||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" |
|||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> |
|||
<modelVersion>4.0.0</modelVersion> |
|||
|
|||
<groupId>com.animal</groupId> |
|||
<artifactId>AnimalCrawler</artifactId> |
|||
<version>1.0-SNAPSHOT</version> |
|||
|
|||
<properties> |
|||
<maven.compiler.release>11</maven.compiler.release> |
|||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> |
|||
</properties> |
|||
|
|||
<dependencies> |
|||
<!-- Jsoup用于网页解析 --> |
|||
<dependency> |
|||
<groupId>org.jsoup</groupId> |
|||
<artifactId>jsoup</artifactId> |
|||
<version>1.17.2</version> |
|||
</dependency> |
|||
<!-- Jackson用于JSON处理 --> |
|||
<dependency> |
|||
<groupId>com.fasterxml.jackson.core</groupId> |
|||
<artifactId>jackson-databind</artifactId> |
|||
<version>2.15.2</version> |
|||
</dependency> |
|||
<!-- JFreeChart用于数据可视化 --> |
|||
<dependency> |
|||
<groupId>org.jfree</groupId> |
|||
<artifactId>jfreechart</artifactId> |
|||
<version>1.5.4</version> |
|||
</dependency> |
|||
<!-- Apache Commons CSV用于CSV文件处理 --> |
|||
<dependency> |
|||
<groupId>org.apache.commons</groupId> |
|||
<artifactId>commons-csv</artifactId> |
|||
<version>1.10.0</version> |
|||
</dependency> |
|||
</dependencies> |
|||
|
|||
<build> |
|||
<plugins> |
|||
<plugin> |
|||
<groupId>org.apache.maven.plugins</groupId> |
|||
<artifactId>maven-assembly-plugin</artifactId> |
|||
<version>3.6.0</version> |
|||
<configuration> |
|||
<archive> |
|||
<manifest> |
|||
<mainClass>com.animal.crawler.Main</mainClass> |
|||
</manifest> |
|||
</archive> |
|||
<descriptorRefs> |
|||
<descriptorRef>jar-with-dependencies</descriptorRef> |
|||
</descriptorRefs> |
|||
</configuration> |
|||
<executions> |
|||
<execution> |
|||
<id>make-assembly</id> |
|||
<phase>package</phase> |
|||
<goals> |
|||
<goal>single</goal> |
|||
</goals> |
|||
</execution> |
|||
</executions> |
|||
</plugin> |
|||
</plugins> |
|||
</build> |
|||
|
|||
</project> |
|||
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 35 KiB |
|
After Width: | Height: | Size: 36 KiB |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,26 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
/** |
|||
* 命令接口 - 定义命令的标准行为 |
|||
*/ |
|||
public interface Command { |
|||
/** |
|||
* 执行命令 |
|||
* @throws CrawlerException 命令执行过程中发生的异常 |
|||
*/ |
|||
void execute() throws CrawlerException; |
|||
|
|||
/** |
|||
* 获取命令名称 |
|||
* @return 命令名称 |
|||
*/ |
|||
String getName(); |
|||
|
|||
/** |
|||
* 获取命令描述 |
|||
* @return 命令描述 |
|||
*/ |
|||
String getDescription(); |
|||
} |
|||
@ -0,0 +1,44 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.exception.CrawlerException; |
|||
import com.animal.exception.InvalidCommandException; |
|||
|
|||
import java.util.HashMap; |
|||
import java.util.Map; |
|||
|
|||
/** |
|||
* 命令管理器 - 负责注册和执行命令 |
|||
*/ |
|||
public class CommandManager { |
|||
private Map<String, Command> commands = new HashMap<>(); |
|||
|
|||
/** |
|||
* 注册命令 |
|||
* @param command 命令对象 |
|||
*/ |
|||
public void registerCommand(Command command) { |
|||
commands.put(command.getName().toLowerCase(), command); |
|||
} |
|||
|
|||
/** |
|||
* 执行命令 |
|||
* @param commandName 命令名称 |
|||
* @throws CrawlerException 命令执行异常 |
|||
*/ |
|||
public void executeCommand(String commandName) throws CrawlerException { |
|||
Command command = commands.get(commandName.toLowerCase()); |
|||
if (command == null) { |
|||
throw new InvalidCommandException("未知命令: " + commandName + ",使用 help 查看可用命令"); |
|||
} |
|||
command.execute(); |
|||
} |
|||
|
|||
/** |
|||
* 检查命令是否存在 |
|||
* @param commandName 命令名称 |
|||
* @return 是否存在 |
|||
*/ |
|||
public boolean hasCommand(String commandName) { |
|||
return commands.containsKey(commandName.toLowerCase()); |
|||
} |
|||
} |
|||
@ -0,0 +1,41 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.strategy.CrawlerContext; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 爬取命令 - 执行爬虫操作 |
|||
*/ |
|||
public class CrawlCommand implements Command { |
|||
private CrawlerContext context; |
|||
private List<Animal> result; |
|||
|
|||
public CrawlCommand(CrawlerContext context) { |
|||
this.context = context; |
|||
} |
|||
|
|||
public List<Animal> getResult() { |
|||
return result; |
|||
} |
|||
|
|||
@Override |
|||
public void execute() throws CrawlerException { |
|||
System.out.println("正在从 [" + context.getStrategyName() + "] 爬取数据..."); |
|||
System.out.println("数据源: " + context.getStrategyUrl()); |
|||
result = context.executeCrawl(); |
|||
System.out.println("爬取完成!共获取 " + result.size() + " 条动物数据"); |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "crawl"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "执行爬虫操作,从当前数据源获取濒危动物数据"; |
|||
} |
|||
} |
|||
@ -0,0 +1,41 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
/** |
|||
* 帮助命令 - 显示所有可用命令 |
|||
*/ |
|||
public class HelpCommand implements Command { |
|||
@Override |
|||
public void execute() throws CrawlerException { |
|||
System.out.println("========================================"); |
|||
System.out.println("濒危动物爬虫系统 - 命令帮助"); |
|||
System.out.println("========================================"); |
|||
System.out.println(); |
|||
System.out.println("可用命令:"); |
|||
System.out.println(" set <数据源> - 设置爬虫数据源 (china/wwf/iucn)"); |
|||
System.out.println(" crawl - 执行爬虫操作"); |
|||
System.out.println(" list - 显示爬取的数据"); |
|||
System.out.println(" save <文件名> - 保存数据到文件"); |
|||
System.out.println(" sources - 显示所有可用数据源"); |
|||
System.out.println(" help - 显示此帮助信息"); |
|||
System.out.println(" exit - 退出程序"); |
|||
System.out.println(); |
|||
System.out.println("使用示例:"); |
|||
System.out.println(" set china - 切换到中国红色名录"); |
|||
System.out.println(" crawl - 爬取数据"); |
|||
System.out.println(" list - 查看数据"); |
|||
System.out.println(" save output.txt - 保存到文件"); |
|||
System.out.println("========================================"); |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "help"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "显示所有可用命令的帮助信息"; |
|||
} |
|||
} |
|||
@ -0,0 +1,54 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 列出命令 - 显示当前爬取的数据 |
|||
*/ |
|||
public class ListCommand implements Command { |
|||
private List<Animal> animals; |
|||
|
|||
public ListCommand(List<Animal> animals) { |
|||
this.animals = animals; |
|||
} |
|||
|
|||
@Override |
|||
public void execute() throws CrawlerException { |
|||
if (animals == null || animals.isEmpty()) { |
|||
System.out.println("暂无数据,请先执行爬取命令 (crawl)"); |
|||
return; |
|||
} |
|||
|
|||
System.out.println("========================================"); |
|||
System.out.println("濒危动物数据列表 (" + animals.size() + " 条)"); |
|||
System.out.println("========================================"); |
|||
|
|||
int index = 1; |
|||
for (Animal animal : animals) { |
|||
System.out.println(); |
|||
System.out.println(index++ + ". " + animal.getName()); |
|||
System.out.println(" 学名: " + animal.getScientificName()); |
|||
System.out.println(" 类别: " + animal.getCategory()); |
|||
System.out.println(" 保护状态: " + animal.getStatus()); |
|||
System.out.println(" 栖息地: " + animal.getHabitat()); |
|||
System.out.println(" 描述: " + animal.getDescription()); |
|||
System.out.println(" 种群数量: " + animal.getPopulation()); |
|||
} |
|||
|
|||
System.out.println(); |
|||
System.out.println("========================================"); |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "list"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "显示当前爬取的濒危动物数据列表"; |
|||
} |
|||
} |
|||
@ -0,0 +1,75 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.StorageException; |
|||
|
|||
import java.io.FileWriter; |
|||
import java.io.IOException; |
|||
import java.io.PrintWriter; |
|||
import java.time.LocalDateTime; |
|||
import java.time.format.DateTimeFormatter; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 保存命令 - 将数据保存到文件 |
|||
*/ |
|||
public class SaveCommand implements Command { |
|||
private List<Animal> animals; |
|||
private String filePath; |
|||
|
|||
public SaveCommand(List<Animal> animals, String filePath) { |
|||
this.animals = animals; |
|||
this.filePath = filePath; |
|||
} |
|||
|
|||
@Override |
|||
public void execute() throws StorageException { |
|||
if (animals == null || animals.isEmpty()) { |
|||
throw new StorageException("没有可保存的数据,请先执行爬取命令"); |
|||
} |
|||
|
|||
try (PrintWriter writer = new PrintWriter(new FileWriter(filePath))) { |
|||
// 写入文件头
|
|||
writer.println("========================================"); |
|||
writer.println("濒危动物数据报告"); |
|||
writer.println("生成时间: " + LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"))); |
|||
writer.println("数据条数: " + animals.size()); |
|||
writer.println("========================================"); |
|||
writer.println(); |
|||
|
|||
// 写入每条动物数据
|
|||
int index = 1; |
|||
for (Animal animal : animals) { |
|||
writer.println("【动物" + index++ + "】"); |
|||
writer.println("名称: " + animal.getName()); |
|||
writer.println("学名: " + animal.getScientificName()); |
|||
writer.println("类别: " + animal.getCategory()); |
|||
writer.println("保护状态: " + animal.getStatus()); |
|||
writer.println("栖息地: " + animal.getHabitat()); |
|||
writer.println("描述: " + animal.getDescription()); |
|||
writer.println("种群数量: " + animal.getPopulation()); |
|||
writer.println(); |
|||
} |
|||
|
|||
// 写入文件尾
|
|||
writer.println("========================================"); |
|||
writer.println("报告结束"); |
|||
writer.println("========================================"); |
|||
|
|||
System.out.println("数据已成功保存到文件: " + filePath); |
|||
|
|||
} catch (IOException e) { |
|||
throw new StorageException("保存文件失败: " + e.getMessage(), e); |
|||
} |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "save"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "将爬取的数据保存到文件"; |
|||
} |
|||
} |
|||
@ -0,0 +1,50 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.strategy.CrawlerContext; |
|||
import com.animal.strategy.CrawlerStrategy; |
|||
import com.animal.exception.InvalidCommandException; |
|||
|
|||
/** |
|||
* 设置策略命令 - 切换爬虫数据源 |
|||
*/ |
|||
public class SetStrategyCommand implements Command { |
|||
private CrawlerContext context; |
|||
private String strategyType; |
|||
|
|||
public SetStrategyCommand(CrawlerContext context, String strategyType) { |
|||
this.context = context; |
|||
this.strategyType = strategyType; |
|||
} |
|||
|
|||
@Override |
|||
public void execute() throws InvalidCommandException { |
|||
CrawlerStrategy strategy = createStrategy(strategyType); |
|||
context.setStrategy(strategy); |
|||
System.out.println("已切换到数据源: " + strategy.getDataSourceName()); |
|||
System.out.println("数据源URL: " + strategy.getDataSourceUrl()); |
|||
} |
|||
|
|||
private CrawlerStrategy createStrategy(String type) throws InvalidCommandException { |
|||
switch (type.toLowerCase()) { |
|||
case "china": |
|||
case "redlist": |
|||
return new com.animal.strategy.ChinaRedListStrategy(); |
|||
case "wwf": |
|||
return new com.animal.strategy.WWFStrategy(); |
|||
case "iucn": |
|||
return new com.animal.strategy.IUCNStrategy(); |
|||
default: |
|||
throw new InvalidCommandException("未知的数据源类型: " + type); |
|||
} |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "set"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "设置爬虫数据源,可选值: china, wwf, iucn"; |
|||
} |
|||
} |
|||
@ -0,0 +1,41 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
/** |
|||
* 数据源命令 - 显示所有可用数据源 |
|||
*/ |
|||
public class SourcesCommand implements Command { |
|||
@Override |
|||
public void execute() throws CrawlerException { |
|||
System.out.println("========================================"); |
|||
System.out.println("可用数据源列表"); |
|||
System.out.println("========================================"); |
|||
System.out.println(); |
|||
System.out.println("1. china (中国濒危动物红色名录)"); |
|||
System.out.println(" - 描述: 中国特有的濒危动物数据"); |
|||
System.out.println(" - URL: http://www.cas.cn/zt/kjzt/kpcg/"); |
|||
System.out.println(); |
|||
System.out.println("2. wwf (WWF世界自然基金会)"); |
|||
System.out.println(" - 描述: 全球濒危动物数据"); |
|||
System.out.println(" - URL: https://www.wwfchina.org/"); |
|||
System.out.println(); |
|||
System.out.println("3. iucn (IUCN红色名录)"); |
|||
System.out.println(" - 描述: 国际自然保护联盟红色名录"); |
|||
System.out.println(" - URL: https://www.iucnredlist.org/"); |
|||
System.out.println(); |
|||
System.out.println("使用命令: set <数据源> 切换数据源"); |
|||
System.out.println("例如: set china"); |
|||
System.out.println("========================================"); |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "sources"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "显示所有可用的数据源列表"; |
|||
} |
|||
} |
|||
@ -0,0 +1,47 @@ |
|||
package com.animal.command; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.visualization.AnimalVisualization; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.io.IOException; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 可视化命令 - 生成可视化图表 |
|||
*/ |
|||
public class VisualizeCommand implements Command { |
|||
private List<Animal> animals; |
|||
|
|||
public VisualizeCommand(List<Animal> animals) { |
|||
this.animals = animals; |
|||
} |
|||
|
|||
@Override |
|||
public void execute() throws CrawlerException { |
|||
if (animals == null || animals.isEmpty()) { |
|||
throw new CrawlerException("没有可可视化的数据,请先执行爬取命令"); |
|||
} |
|||
|
|||
AnimalVisualization visualization = new AnimalVisualization(); |
|||
|
|||
try { |
|||
System.out.println("正在生成可视化图表..."); |
|||
visualization.createStatusDistributionChart(animals); |
|||
visualization.createPopulationChart(animals); |
|||
System.out.println("可视化图表生成完成!"); |
|||
} catch (IOException e) { |
|||
throw new CrawlerException("生成图表失败: " + e.getMessage(), e); |
|||
} |
|||
} |
|||
|
|||
@Override |
|||
public String getName() { |
|||
return "visualize"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDescription() { |
|||
return "生成可视化图表(保护状态饼图和种群数量柱状图)"; |
|||
} |
|||
} |
|||
@ -0,0 +1,181 @@ |
|||
package com.animal.controller; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.view.CrawlerView; |
|||
import com.animal.strategy.CrawlerContext; |
|||
import com.animal.strategy.ChinaRedListStrategy; |
|||
import com.animal.command.*; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.ArrayList; |
|||
import java.util.List; |
|||
import java.util.Scanner; |
|||
|
|||
/** |
|||
* 爬虫控制器类 - 负责协调View和Model,处理用户输入 |
|||
*/ |
|||
public class CrawlerController { |
|||
private CrawlerView view; |
|||
private CrawlerContext context; |
|||
private List<Animal> animals; |
|||
private CommandManager commandManager; |
|||
|
|||
public CrawlerController() { |
|||
this.view = new CrawlerView(); |
|||
this.context = new CrawlerContext(); |
|||
this.animals = new ArrayList<>(); |
|||
this.commandManager = new CommandManager(); |
|||
|
|||
// 默认设置为中国红色名录
|
|||
context.setStrategy(new ChinaRedListStrategy()); |
|||
} |
|||
|
|||
/** |
|||
* 启动应用程序 |
|||
*/ |
|||
public void start() { |
|||
view.showWelcome(); |
|||
|
|||
Scanner scanner = new Scanner(System.in); |
|||
String input; |
|||
|
|||
while (true) { |
|||
System.out.print("请输入命令: "); |
|||
input = scanner.nextLine().trim(); |
|||
|
|||
if (input.isEmpty()) { |
|||
continue; |
|||
} |
|||
|
|||
try { |
|||
processCommand(input); |
|||
} catch (CrawlerException e) { |
|||
view.showError(e.getMessage()); |
|||
} |
|||
|
|||
if (input.equalsIgnoreCase("exit")) { |
|||
break; |
|||
} |
|||
} |
|||
|
|||
scanner.close(); |
|||
view.showExit(); |
|||
} |
|||
|
|||
/** |
|||
* 处理用户命令 |
|||
* @param input 用户输入 |
|||
* @throws CrawlerException 命令处理异常 |
|||
*/ |
|||
private void processCommand(String input) throws CrawlerException { |
|||
String[] parts = input.split("\\s+"); |
|||
String commandName = parts[0].toLowerCase(); |
|||
|
|||
switch (commandName) { |
|||
case "set": |
|||
if (parts.length < 2) { |
|||
throw new CrawlerException("请指定数据源类型,使用 'set <数据源>'"); |
|||
} |
|||
setDataSource(parts[1]); |
|||
break; |
|||
|
|||
case "crawl": |
|||
crawlData(); |
|||
break; |
|||
|
|||
case "list": |
|||
listData(); |
|||
break; |
|||
|
|||
case "save": |
|||
if (parts.length < 2) { |
|||
throw new CrawlerException("请指定文件名,使用 'save <文件名>'"); |
|||
} |
|||
saveData(parts[1]); |
|||
break; |
|||
|
|||
case "visualize": |
|||
visualizeData(); |
|||
break; |
|||
|
|||
case "sources": |
|||
view.showDataSources(); |
|||
break; |
|||
|
|||
case "help": |
|||
view.showHelp(); |
|||
break; |
|||
|
|||
case "exit": |
|||
// 退出处理在start()方法中
|
|||
break; |
|||
|
|||
default: |
|||
throw new CrawlerException("未知命令: " + commandName + ",使用 help 查看可用命令"); |
|||
} |
|||
} |
|||
|
|||
/** |
|||
* 设置数据源 |
|||
* @param sourceType 数据源类型 |
|||
* @throws CrawlerException 异常 |
|||
*/ |
|||
private void setDataSource(String sourceType) throws CrawlerException { |
|||
SetStrategyCommand command = new SetStrategyCommand(context, sourceType); |
|||
command.execute(); |
|||
} |
|||
|
|||
/** |
|||
* 爬取数据 |
|||
* @throws CrawlerException 异常 |
|||
*/ |
|||
private void crawlData() throws CrawlerException { |
|||
CrawlCommand command = new CrawlCommand(context); |
|||
command.execute(); |
|||
animals = command.getResult(); |
|||
} |
|||
|
|||
/** |
|||
* 列出数据 |
|||
* @throws CrawlerException 异常 |
|||
*/ |
|||
private void listData() throws CrawlerException { |
|||
ListCommand command = new ListCommand(animals); |
|||
command.execute(); |
|||
} |
|||
|
|||
/** |
|||
* 保存数据 |
|||
* @param filePath 文件路径 |
|||
* @throws CrawlerException 异常 |
|||
*/ |
|||
private void saveData(String filePath) throws CrawlerException { |
|||
SaveCommand command = new SaveCommand(animals, filePath); |
|||
command.execute(); |
|||
} |
|||
|
|||
/** |
|||
* 可视化数据 |
|||
* @throws CrawlerException 异常 |
|||
*/ |
|||
private void visualizeData() throws CrawlerException { |
|||
VisualizeCommand command = new VisualizeCommand(animals); |
|||
command.execute(); |
|||
} |
|||
|
|||
/** |
|||
* 获取当前数据源名称 |
|||
* @return 数据源名称 |
|||
*/ |
|||
public String getCurrentDataSourceName() { |
|||
return context.getStrategyName(); |
|||
} |
|||
|
|||
/** |
|||
* 获取当前数据源URL |
|||
* @return 数据源URL |
|||
*/ |
|||
public String getCurrentDataSourceUrl() { |
|||
return context.getStrategyUrl(); |
|||
} |
|||
} |
|||
@ -0,0 +1,54 @@ |
|||
package com.animal.crawler; |
|||
|
|||
import com.animal.model.Animal; |
|||
|
|||
import java.util.ArrayList; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 动物爬虫类 - 用于生成模拟的濒危动物信息 |
|||
* 展示了面向对象编程中的类设计和方法实现 |
|||
*/ |
|||
public class AnimalCrawler { |
|||
/** |
|||
* 爬取方法 - 核心方法,用于生成模拟的动物数据 |
|||
* @return 动物列表 |
|||
*/ |
|||
public List<Animal> crawl() { |
|||
// 创建ArrayList集合用于存储模拟的动物数据
|
|||
// 泛型:指定集合中元素的类型为Animal
|
|||
List<Animal> animals = new ArrayList<>(); |
|||
|
|||
// 生成模拟的濒危动物数据
|
|||
animals.add(createAnimal("大熊猫", "Ailuropoda melanoleuca", "哺乳动物", "易危(VU)", "中国四川、陕西、甘肃", "中国特有的珍稀动物", 1864)); |
|||
animals.add(createAnimal("华南虎", "Panthera tigris amoyensis", "哺乳动物", "极危(CR)", "中国南方森林", "中国特有的虎亚种", 25)); |
|||
animals.add(createAnimal("金丝猴", "Rhinopithecus roxellana", "哺乳动物", "濒危(EN)", "中国四川、陕西、湖北", "中国特有的灵长类动物", 20000)); |
|||
animals.add(createAnimal("朱鹮", "Nipponia nippon", "鸟类", "濒危(EN)", "中国陕西、河南、浙江", "曾经濒临灭绝的鸟类", 7000)); |
|||
|
|||
// 返回模拟的动物列表
|
|||
return animals; |
|||
} |
|||
|
|||
/** |
|||
* 创建动物对象的辅助方法 |
|||
* @param name 动物名称 |
|||
* @param scientificName 学名 |
|||
* @param category 类别 |
|||
* @param status 保护状态 |
|||
* @param habitat 栖息地 |
|||
* @param description 描述 |
|||
* @param population 种群数量 |
|||
* @return Animal对象 |
|||
*/ |
|||
private Animal createAnimal(String name, String scientificName, String category, String status, String habitat, String description, int population) { |
|||
Animal animal = new Animal(); |
|||
animal.setName(name); |
|||
animal.setScientificName(scientificName); |
|||
animal.setCategory(category); |
|||
animal.setStatus(status); |
|||
animal.setHabitat(habitat); |
|||
animal.setDescription(description); |
|||
animal.setPopulation(population); |
|||
return animal; |
|||
} |
|||
} |
|||
@ -0,0 +1,25 @@ |
|||
package com.animal.crawler; |
|||
|
|||
import com.animal.controller.CrawlerController; |
|||
|
|||
/** |
|||
* 主类 - 程序入口点 |
|||
* 实现 CLI + MVC + Command模式 + 策略模式 + 异常体系 |
|||
*/ |
|||
public class Main { |
|||
|
|||
/** |
|||
* 主方法 - 程序入口 |
|||
* @param args 命令行参数 |
|||
*/ |
|||
public static void main(String[] args) { |
|||
try { |
|||
// 创建控制器并启动应用
|
|||
CrawlerController controller = new CrawlerController(); |
|||
controller.start(); |
|||
} catch (Exception e) { |
|||
System.err.println("程序启动失败: " + e.getMessage()); |
|||
e.printStackTrace(); |
|||
} |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 爬虫异常基类 - 所有爬虫相关异常的父类 |
|||
*/ |
|||
public class CrawlerException extends Exception { |
|||
public CrawlerException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public CrawlerException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 数据源异常 - 数据源相关的异常 |
|||
*/ |
|||
public class DataSourceException extends CrawlerException { |
|||
public DataSourceException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public DataSourceException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 无效命令异常 - CLI命令相关的异常 |
|||
*/ |
|||
public class InvalidCommandException extends CrawlerException { |
|||
public InvalidCommandException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public InvalidCommandException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 网络异常 - 网络连接相关的异常 |
|||
*/ |
|||
public class NetworkException extends CrawlerException { |
|||
public NetworkException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public NetworkException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 解析异常 - HTML/JSON解析相关的异常 |
|||
*/ |
|||
public class ParseException extends CrawlerException { |
|||
public ParseException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public ParseException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,14 @@ |
|||
package com.animal.exception; |
|||
|
|||
/** |
|||
* 存储异常 - 文件保存相关的异常 |
|||
*/ |
|||
public class StorageException extends CrawlerException { |
|||
public StorageException(String message) { |
|||
super(message); |
|||
} |
|||
|
|||
public StorageException(String message, Throwable cause) { |
|||
super(message, cause); |
|||
} |
|||
} |
|||
@ -0,0 +1,183 @@ |
|||
package com.animal.model; |
|||
|
|||
import com.fasterxml.jackson.annotation.JsonProperty; |
|||
|
|||
/** |
|||
* 动物类 - 用于存储濒危动物的信息 |
|||
* 这是一个普通的Java类,展示了面向对象编程的基本概念 |
|||
*/ |
|||
public class Animal { |
|||
// 私有属性 - 封装性的体现
|
|||
// 封装:将属性设为private,只能通过getter和setter方法访问
|
|||
private String name; // 动物名称
|
|||
private String scientificName; // 学名
|
|||
private String category; // 动物类别
|
|||
private String status; // 保护状态
|
|||
private String habitat; // 栖息地
|
|||
private String description; // 描述
|
|||
private int population; // 种群数量
|
|||
|
|||
/** |
|||
* 无参构造方法 |
|||
* 构造方法:用于创建对象实例 |
|||
* 无参构造方法是默认的构造方法,当没有定义其他构造方法时会自动生成 |
|||
*/ |
|||
public Animal() { |
|||
} |
|||
|
|||
/** |
|||
* 带参构造方法 |
|||
* 构造方法:用于创建对象实例并初始化属性 |
|||
* @param name 动物名称 |
|||
* @param scientificName 学名 |
|||
* @param category 动物类别 |
|||
* @param status 保护状态 |
|||
* @param habitat 栖息地 |
|||
* @param description 描述 |
|||
* @param population 种群数量 |
|||
*/ |
|||
public Animal(String name, String scientificName, String category, String status, String habitat, String description, int population) { |
|||
// this关键字:指向当前对象的引用
|
|||
// 用于区分成员变量和局部变量
|
|||
this.name = name; |
|||
this.scientificName = scientificName; |
|||
this.category = category; |
|||
this.status = status; |
|||
this.habitat = habitat; |
|||
this.description = description; |
|||
this.population = population; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取name属性 |
|||
* 封装性的体现:通过公共方法访问私有属性 |
|||
* @return 动物名称 |
|||
*/ |
|||
public String getName() { |
|||
return name; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置name属性 |
|||
* 封装性的体现:通过公共方法修改私有属性 |
|||
* @param name 动物名称 |
|||
*/ |
|||
public void setName(String name) { |
|||
this.name = name; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取scientificName属性 |
|||
* @return 学名 |
|||
*/ |
|||
public String getScientificName() { |
|||
return scientificName; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置scientificName属性 |
|||
* @param scientificName 学名 |
|||
*/ |
|||
public void setScientificName(String scientificName) { |
|||
this.scientificName = scientificName; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取category属性 |
|||
* @return 动物类别 |
|||
*/ |
|||
public String getCategory() { |
|||
return category; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置category属性 |
|||
* @param category 动物类别 |
|||
*/ |
|||
public void setCategory(String category) { |
|||
this.category = category; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取status属性 |
|||
* @return 保护状态 |
|||
*/ |
|||
public String getStatus() { |
|||
return status; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置status属性 |
|||
* @param status 保护状态 |
|||
*/ |
|||
public void setStatus(String status) { |
|||
this.status = status; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取habitat属性 |
|||
* @return 栖息地 |
|||
*/ |
|||
public String getHabitat() { |
|||
return habitat; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置habitat属性 |
|||
* @param habitat 栖息地 |
|||
*/ |
|||
public void setHabitat(String habitat) { |
|||
this.habitat = habitat; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取description属性 |
|||
* @return 描述 |
|||
*/ |
|||
public String getDescription() { |
|||
return description; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置description属性 |
|||
* @param description 描述 |
|||
*/ |
|||
public void setDescription(String description) { |
|||
this.description = description; |
|||
} |
|||
|
|||
/** |
|||
* getter方法 - 用于获取population属性 |
|||
* @return 种群数量 |
|||
*/ |
|||
public int getPopulation() { |
|||
return population; |
|||
} |
|||
|
|||
/** |
|||
* setter方法 - 用于设置population属性 |
|||
* @param population 种群数量 |
|||
*/ |
|||
public void setPopulation(int population) { |
|||
this.population = population; |
|||
} |
|||
|
|||
/** |
|||
* 重写toString方法 |
|||
* 方法重写:子类重写父类的方法 |
|||
* 这里重写了Object类的toString方法 |
|||
* @return 动物对象的字符串表示 |
|||
*/ |
|||
@Override // @Override注解:表示这个方法是重写父类的方法
|
|||
public String toString() { |
|||
return "Animal{" + |
|||
"name='" + name + '\'' + |
|||
", scientificName='" + scientificName + '\'' + |
|||
", category='" + category + '\'' + |
|||
", status='" + status + '\'' + |
|||
", habitat='" + habitat + '\'' + |
|||
", description='" + description + '\'' + |
|||
", population=" + population + |
|||
'}'; |
|||
} |
|||
} |
|||
@ -0,0 +1,49 @@ |
|||
package com.animal.strategy; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.ArrayList; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 中国红色名录爬虫策略 - 模拟爬取中国濒危动物红色名录数据 |
|||
*/ |
|||
public class ChinaRedListStrategy implements CrawlerStrategy { |
|||
|
|||
@Override |
|||
public String getDataSourceName() { |
|||
return "中国濒危动物红色名录"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDataSourceUrl() { |
|||
return "http://www.cas.cn/zt/kjzt/kpcg/"; |
|||
} |
|||
|
|||
@Override |
|||
public List<Animal> crawl() throws CrawlerException { |
|||
List<Animal> animals = new ArrayList<>(); |
|||
|
|||
// 模拟爬取中国濒危动物红色名录数据
|
|||
animals.add(createAnimal("大熊猫", "Ailuropoda melanoleuca", "哺乳动物", "易危(VU)", "中国四川、陕西、甘肃", "中国特有的珍稀动物,被誉为活化石", 1864)); |
|||
animals.add(createAnimal("华南虎", "Panthera tigris amoyensis", "哺乳动物", "极危(CR)", "中国南方森林", "中国特有的虎亚种,野外已灭绝", 25)); |
|||
animals.add(createAnimal("金丝猴", "Rhinopithecus roxellana", "哺乳动物", "濒危(EN)", "中国四川、陕西、湖北", "中国特有的灵长类动物", 20000)); |
|||
animals.add(createAnimal("朱鹮", "Nipponia nippon", "鸟类", "濒危(EN)", "中国陕西、河南、浙江", "曾经濒临灭绝的鸟类,现已恢复", 7000)); |
|||
animals.add(createAnimal("白鳍豚", "Lipotes vexillifer", "哺乳动物", "极危(CR)", "中国长江流域", "中国特有的淡水豚类", 0)); |
|||
|
|||
return animals; |
|||
} |
|||
|
|||
private Animal createAnimal(String name, String scientificName, String category, String status, String habitat, String description, int population) { |
|||
Animal animal = new Animal(); |
|||
animal.setName(name); |
|||
animal.setScientificName(scientificName); |
|||
animal.setCategory(category); |
|||
animal.setStatus(status); |
|||
animal.setHabitat(habitat); |
|||
animal.setDescription(description); |
|||
animal.setPopulation(population); |
|||
return animal; |
|||
} |
|||
} |
|||
@ -0,0 +1,49 @@ |
|||
package com.animal.strategy; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 爬虫上下文类 - 用于管理爬虫策略的切换和执行 |
|||
*/ |
|||
public class CrawlerContext { |
|||
private CrawlerStrategy strategy; |
|||
|
|||
/** |
|||
* 设置爬虫策略 |
|||
* @param strategy 爬虫策略 |
|||
*/ |
|||
public void setStrategy(CrawlerStrategy strategy) { |
|||
this.strategy = strategy; |
|||
} |
|||
|
|||
/** |
|||
* 获取当前策略名称 |
|||
* @return 策略名称 |
|||
*/ |
|||
public String getStrategyName() { |
|||
return strategy != null ? strategy.getDataSourceName() : "未设置"; |
|||
} |
|||
|
|||
/** |
|||
* 获取当前策略URL |
|||
* @return 策略URL |
|||
*/ |
|||
public String getStrategyUrl() { |
|||
return strategy != null ? strategy.getDataSourceUrl() : "未设置"; |
|||
} |
|||
|
|||
/** |
|||
* 执行爬取操作 |
|||
* @return 爬取到的动物列表 |
|||
* @throws CrawlerException 爬取过程中发生的异常 |
|||
*/ |
|||
public List<Animal> executeCrawl() throws CrawlerException { |
|||
if (strategy == null) { |
|||
throw new CrawlerException("未设置爬虫策略"); |
|||
} |
|||
return strategy.crawl(); |
|||
} |
|||
} |
|||
@ -0,0 +1,30 @@ |
|||
package com.animal.strategy; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 爬虫策略接口 - 定义爬虫策略的标准行为 |
|||
*/ |
|||
public interface CrawlerStrategy { |
|||
/** |
|||
* 获取数据源名称 |
|||
* @return 数据源名称 |
|||
*/ |
|||
String getDataSourceName(); |
|||
|
|||
/** |
|||
* 获取数据源URL |
|||
* @return 数据源URL |
|||
*/ |
|||
String getDataSourceUrl(); |
|||
|
|||
/** |
|||
* 执行爬取操作 |
|||
* @return 爬取到的动物列表 |
|||
* @throws CrawlerException 爬取过程中发生的异常 |
|||
*/ |
|||
List<Animal> crawl() throws CrawlerException; |
|||
} |
|||
@ -0,0 +1,49 @@ |
|||
package com.animal.strategy; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.ArrayList; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* IUCN红色名录爬虫策略 - 模拟爬取IUCN濒危动物红色名录数据 |
|||
*/ |
|||
public class IUCNStrategy implements CrawlerStrategy { |
|||
|
|||
@Override |
|||
public String getDataSourceName() { |
|||
return "IUCN红色名录"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDataSourceUrl() { |
|||
return "https://www.iucnredlist.org/"; |
|||
} |
|||
|
|||
@Override |
|||
public List<Animal> crawl() throws CrawlerException { |
|||
List<Animal> animals = new ArrayList<>(); |
|||
|
|||
// 模拟爬取IUCN濒危动物数据
|
|||
animals.add(createAnimal("黑犀牛", "Diceros bicornis", "哺乳动物", "极危(CR)", "非洲南部", "犀牛角被非法猎取", 5000)); |
|||
animals.add(createAnimal("大猩猩", "Gorilla gorilla", "哺乳动物", "极危(CR)", "非洲中部", "人类近亲", 100000)); |
|||
animals.add(createAnimal("北极熊", "Ursus maritimus", "哺乳动物", "易危(VU)", "北极地区", "因气候变化而濒危", 26000)); |
|||
animals.add(createAnimal("非洲象", "Loxodonta africana", "哺乳动物", "濒危(EN)", "非洲大陆", "象牙贸易的受害者", 415000)); |
|||
animals.add(createAnimal("大熊猫", "Ailuropoda melanoleuca", "哺乳动物", "易危(VU)", "中国", "全球保护成功的典范", 1864)); |
|||
|
|||
return animals; |
|||
} |
|||
|
|||
private Animal createAnimal(String name, String scientificName, String category, String status, String habitat, String description, int population) { |
|||
Animal animal = new Animal(); |
|||
animal.setName(name); |
|||
animal.setScientificName(scientificName); |
|||
animal.setCategory(category); |
|||
animal.setStatus(status); |
|||
animal.setHabitat(habitat); |
|||
animal.setDescription(description); |
|||
animal.setPopulation(population); |
|||
return animal; |
|||
} |
|||
} |
|||
@ -0,0 +1,49 @@ |
|||
package com.animal.strategy; |
|||
|
|||
import com.animal.model.Animal; |
|||
import com.animal.exception.CrawlerException; |
|||
|
|||
import java.util.ArrayList; |
|||
import java.util.List; |
|||
|
|||
/** |
|||
* WWF世界自然基金会爬虫策略 - 模拟爬取WWF濒危动物数据 |
|||
*/ |
|||
public class WWFStrategy implements CrawlerStrategy { |
|||
|
|||
@Override |
|||
public String getDataSourceName() { |
|||
return "WWF世界自然基金会"; |
|||
} |
|||
|
|||
@Override |
|||
public String getDataSourceUrl() { |
|||
return "https://www.wwfchina.org/"; |
|||
} |
|||
|
|||
@Override |
|||
public List<Animal> crawl() throws CrawlerException { |
|||
List<Animal> animals = new ArrayList<>(); |
|||
|
|||
// 模拟爬取WWF濒危动物数据
|
|||
animals.add(createAnimal("东北虎", "Panthera tigris altaica", "哺乳动物", "濒危(EN)", "俄罗斯远东、中国东北", "世界上最大的猫科动物", 500)); |
|||
animals.add(createAnimal("雪豹", "Panthera uncia", "哺乳动物", "易危(VU)", "中亚高山地区", "高原上的幽灵", 7000)); |
|||
animals.add(createAnimal("亚洲象", "Elephas maximus", "哺乳动物", "濒危(EN)", "南亚、东南亚", "亚洲最大的陆生动物", 50000)); |
|||
animals.add(createAnimal("海龟", "Cheloniidae", "爬行动物", "濒危(EN)", "全球各大洋", "古老的海洋生物", 100000)); |
|||
animals.add(createAnimal("蓝鲸", "Balaenoptera musculus", "哺乳动物", "濒危(EN)", "全球各大洋", "世界上最大的动物", 10000)); |
|||
|
|||
return animals; |
|||
} |
|||
|
|||
private Animal createAnimal(String name, String scientificName, String category, String status, String habitat, String description, int population) { |
|||
Animal animal = new Animal(); |
|||
animal.setName(name); |
|||
animal.setScientificName(scientificName); |
|||
animal.setCategory(category); |
|||
animal.setStatus(status); |
|||
animal.setHabitat(habitat); |
|||
animal.setDescription(description); |
|||
animal.setPopulation(population); |
|||
return animal; |
|||
} |
|||
} |
|||
@ -0,0 +1,161 @@ |
|||
package com.animal.view; |
|||
|
|||
import com.animal.model.Animal; |
|||
|
|||
import java.util.List; |
|||
|
|||
/** |
|||
* 爬虫视图类 - 负责显示信息给用户 |
|||
*/ |
|||
public class CrawlerView { |
|||
|
|||
/** |
|||
* 显示欢迎信息 |
|||
*/ |
|||
public void showWelcome() { |
|||
System.out.println(); |
|||
System.out.println("╔══════════════════════════════════════════════════════════════╗"); |
|||
System.out.println("║ 濒危动物信息爬虫系统 v1.0 ║"); |
|||
System.out.println("║ Endangered Animal Crawler System ║"); |
|||
System.out.println("╚══════════════════════════════════════════════════════════════╝"); |
|||
System.out.println(); |
|||
System.out.println("欢迎使用濒危动物信息爬虫系统!"); |
|||
System.out.println("输入 'help' 查看可用命令"); |
|||
System.out.println("输入 'sources' 查看可用数据源"); |
|||
System.out.println(); |
|||
} |
|||
|
|||
/** |
|||
* 显示提示信息 |
|||
* @param message 提示信息 |
|||
*/ |
|||
public void showInfo(String message) { |
|||
System.out.println("[INFO] " + message); |
|||
} |
|||
|
|||
/** |
|||
* 显示错误信息 |
|||
* @param message 错误信息 |
|||
*/ |
|||
public void showError(String message) { |
|||
System.out.println("[ERROR] " + message); |
|||
} |
|||
|
|||
/** |
|||
* 显示成功信息 |
|||
* @param message 成功信息 |
|||
*/ |
|||
public void showSuccess(String message) { |
|||
System.out.println("[SUCCESS] " + message); |
|||
} |
|||
|
|||
/** |
|||
* 显示分隔线 |
|||
*/ |
|||
public void showSeparator() { |
|||
System.out.println("------------------------------------------------------------"); |
|||
} |
|||
|
|||
/** |
|||
* 显示动物列表 |
|||
* @param animals 动物列表 |
|||
*/ |
|||
public void showAnimalList(List<Animal> animals) { |
|||
if (animals == null || animals.isEmpty()) { |
|||
System.out.println("暂无数据"); |
|||
return; |
|||
} |
|||
|
|||
System.out.println(); |
|||
System.out.println("┌────────────────────────────────────────────────────────────┐"); |
|||
System.out.println("│ 濒危动物数据列表 │"); |
|||
System.out.println("├────────────────────────────────────────────────────────────┤"); |
|||
|
|||
int index = 1; |
|||
for (Animal animal : animals) { |
|||
System.out.println(); |
|||
System.out.println("│ #" + String.format("%-2d", index++) + " " + animal.getName()); |
|||
System.out.println("│ ├─ 学名: " + animal.getScientificName()); |
|||
System.out.println("│ ├─ 类别: " + animal.getCategory()); |
|||
System.out.println("│ ├─ 保护状态: " + animal.getStatus()); |
|||
System.out.println("│ ├─ 栖息地: " + animal.getHabitat()); |
|||
System.out.println("│ ├─ 描述: " + animal.getDescription()); |
|||
System.out.println("│ └─ 种群数量: " + animal.getPopulation()); |
|||
} |
|||
|
|||
System.out.println(); |
|||
System.out.println("└────────────────────────────────────────────────────────────┘"); |
|||
System.out.println("共 " + animals.size() + " 条记录"); |
|||
} |
|||
|
|||
/** |
|||
* 显示数据源列表 |
|||
*/ |
|||
public void showDataSources() { |
|||
System.out.println(); |
|||
System.out.println("┌────────────────────────────────────────────────────────────┐"); |
|||
System.out.println("│ 可用数据源列表 │"); |
|||
System.out.println("├────────────────────────────────────────────────────────────┤"); |
|||
System.out.println("│ │"); |
|||
System.out.println("│ 1. china │"); |
|||
System.out.println("│ ├─ 名称: 中国濒危动物红色名录 │"); |
|||
System.out.println("│ └─ URL: http://www.cas.cn/zt/kjzt/kpcg/ │"); |
|||
System.out.println("│ │"); |
|||
System.out.println("│ 2. wwf │"); |
|||
System.out.println("│ ├─ 名称: WWF世界自然基金会 │"); |
|||
System.out.println("│ └─ URL: https://www.wwfchina.org/ │"); |
|||
System.out.println("│ │"); |
|||
System.out.println("│ 3. iucn │"); |
|||
System.out.println("│ ├─ 名称: IUCN红色名录 │"); |
|||
System.out.println("│ └─ URL: https://www.iucnredlist.org/ │"); |
|||
System.out.println("│ │"); |
|||
System.out.println("└────────────────────────────────────────────────────────────┘"); |
|||
System.out.println("使用: set <数据源> 切换数据源"); |
|||
} |
|||
|
|||
/** |
|||
* 显示帮助信息 |
|||
*/ |
|||
public void showHelp() { |
|||
System.out.println(); |
|||
System.out.println("┌────────────────────────────────────────────────────────────┐"); |
|||
System.out.println("│ 命令帮助 │"); |
|||
System.out.println("├────────────────────────────────────────────────────────────┤"); |
|||
System.out.println("│ │"); |
|||
System.out.println("│ set <数据源> 设置爬虫数据源 (china/wwf/iucn) │"); |
|||
System.out.println("│ crawl 执行爬虫操作 │"); |
|||
System.out.println("│ list 显示爬取的数据 │"); |
|||
System.out.println("│ save <文件名> 保存数据到文件 │"); |
|||
System.out.println("│ visualize 生成可视化图表 │"); |
|||
System.out.println("│ sources 显示所有可用数据源 │"); |
|||
System.out.println("│ help 显示此帮助信息 │"); |
|||
System.out.println("│ exit 退出程序 │"); |
|||
System.out.println("│ │"); |
|||
System.out.println("└────────────────────────────────────────────────────────────┘"); |
|||
} |
|||
|
|||
/** |
|||
* 显示退出信息 |
|||
*/ |
|||
public void showExit() { |
|||
System.out.println(); |
|||
System.out.println("╔══════════════════════════════════════════════════════════════╗"); |
|||
System.out.println("║ 感谢使用本系统! ║"); |
|||
System.out.println("║ Goodbye! ║"); |
|||
System.out.println("╚══════════════════════════════════════════════════════════════╝"); |
|||
System.out.println(); |
|||
} |
|||
|
|||
/** |
|||
* 显示当前数据源 |
|||
* @param name 数据源名称 |
|||
* @param url 数据源URL |
|||
*/ |
|||
public void showCurrentDataSource(String name, String url) { |
|||
System.out.println(); |
|||
System.out.println("当前数据源:"); |
|||
System.out.println(" 名称: " + name); |
|||
System.out.println(" URL: " + url); |
|||
System.out.println(); |
|||
} |
|||
} |
|||
@ -0,0 +1,91 @@ |
|||
package com.animal.visualization; |
|||
|
|||
import com.animal.model.Animal; |
|||
import org.jfree.chart.ChartFactory; |
|||
import org.jfree.chart.JFreeChart; |
|||
import org.jfree.chart.plot.PlotOrientation; |
|||
import org.jfree.data.category.DefaultCategoryDataset; |
|||
import org.jfree.data.general.DefaultPieDataset; |
|||
|
|||
import javax.imageio.ImageIO; |
|||
import java.awt.image.BufferedImage; |
|||
import java.io.File; |
|||
import java.io.IOException; |
|||
import java.util.List; |
|||
import java.util.Map; |
|||
import java.util.stream.Collectors; |
|||
|
|||
/** |
|||
* 动物可视化类 - 用于生成濒危动物数据的可视化图表 |
|||
* 展示了面向对象编程中的方法设计和实现 |
|||
*/ |
|||
public class AnimalVisualization { |
|||
|
|||
/** |
|||
* 创建保护状态分布饼图 |
|||
* @param animals 动物列表 |
|||
* @throws IOException IO异常 |
|||
*/ |
|||
public void createStatusDistributionChart(List<Animal> animals) throws IOException { |
|||
// 统计各保护状态的动物数量
|
|||
Map<String, Long> statusCount = animals.stream() |
|||
.collect(Collectors.groupingBy(Animal::getStatus, Collectors.counting())); |
|||
|
|||
// 创建饼图数据集
|
|||
DefaultPieDataset dataset = new DefaultPieDataset(); |
|||
statusCount.forEach((status, count) -> dataset.setValue(status, count)); |
|||
|
|||
// 创建饼图
|
|||
JFreeChart chart = ChartFactory.createPieChart( |
|||
"濒危动物保护状态分布", |
|||
dataset, |
|||
true, |
|||
true, |
|||
false |
|||
); |
|||
|
|||
// 保存图表
|
|||
File chartFile = new File("status_distribution.png"); |
|||
BufferedImage image = chart.createBufferedImage(800, 600); |
|||
ImageIO.write(image, "png", chartFile); |
|||
System.out.println("保护状态分布图表已保存到: " + chartFile.getAbsolutePath()); |
|||
} |
|||
|
|||
/** |
|||
* 创建种群数量柱状图 |
|||
* @param animals 动物列表 |
|||
* @throws IOException IO异常 |
|||
*/ |
|||
public void createPopulationChart(List<Animal> animals) throws IOException { |
|||
// 创建柱状图数据集
|
|||
DefaultCategoryDataset dataset = new DefaultCategoryDataset(); |
|||
|
|||
// 只取前10个动物的数据进行展示
|
|||
List<Animal> topAnimals = animals.stream() |
|||
.limit(10) |
|||
.collect(Collectors.toList()); |
|||
|
|||
// 增强for循环:遍历前10个动物
|
|||
for (Animal animal : topAnimals) { |
|||
dataset.addValue(animal.getPopulation(), "种群数量", animal.getName()); |
|||
} |
|||
|
|||
// 创建柱状图
|
|||
JFreeChart chart = ChartFactory.createBarChart( |
|||
"濒危动物种群数量", |
|||
"动物名称", |
|||
"种群数量", |
|||
dataset, |
|||
PlotOrientation.VERTICAL, |
|||
true, |
|||
true, |
|||
false |
|||
); |
|||
|
|||
// 保存图表
|
|||
File chartFile = new File("population_chart.png"); |
|||
BufferedImage image = chart.createBufferedImage(1000, 600); |
|||
ImageIO.write(image, "png", chartFile); |
|||
System.out.println("种群数量图表已保存到: " + chartFile.getAbsolutePath()); |
|||
} |
|||
} |
|||
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 29 KiB |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,3 @@ |
|||
artifactId=AnimalCrawler |
|||
groupId=com.animal |
|||
version=1.0-SNAPSHOT |
|||
@ -0,0 +1,26 @@ |
|||
com/animal/strategy/ChinaRedListStrategy.class |
|||
com/animal/exception/NetworkException.class |
|||
com/animal/strategy/WWFStrategy.class |
|||
com/animal/command/HelpCommand.class |
|||
com/animal/strategy/IUCNStrategy.class |
|||
com/animal/command/SetStrategyCommand.class |
|||
com/animal/exception/ParseException.class |
|||
com/animal/command/SourcesCommand.class |
|||
com/animal/exception/InvalidCommandException.class |
|||
com/animal/exception/StorageException.class |
|||
com/animal/strategy/CrawlerContext.class |
|||
com/animal/view/CrawlerView.class |
|||
com/animal/exception/CrawlerException.class |
|||
com/animal/strategy/CrawlerStrategy.class |
|||
com/animal/command/SaveCommand.class |
|||
com/animal/visualization/AnimalVisualization.class |
|||
com/animal/crawler/AnimalCrawler.class |
|||
com/animal/controller/CrawlerController.class |
|||
com/animal/command/ListCommand.class |
|||
com/animal/command/VisualizeCommand.class |
|||
com/animal/exception/DataSourceException.class |
|||
com/animal/model/Animal.class |
|||
com/animal/command/Command.class |
|||
com/animal/crawler/Main.class |
|||
com/animal/command/CommandManager.class |
|||
com/animal/command/CrawlCommand.class |
|||
@ -0,0 +1,26 @@ |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/Command.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/CommandManager.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/CrawlCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/HelpCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/ListCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/SaveCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/SetStrategyCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/SourcesCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/command/VisualizeCommand.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/controller/CrawlerController.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/crawler/AnimalCrawler.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/crawler/Main.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/CrawlerException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/DataSourceException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/InvalidCommandException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/NetworkException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/ParseException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/exception/StorageException.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/model/Animal.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/strategy/ChinaRedListStrategy.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/strategy/CrawlerContext.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/strategy/CrawlerStrategy.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/strategy/IUCNStrategy.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/strategy/WWFStrategy.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/view/CrawlerView.java |
|||
/Users/thea/Desktop/大一/下学期/java/AnimalCrawler/src/main/java/com/animal/visualization/AnimalVisualization.java |
|||
Loading…
Reference in new issue