ZhengJiayin
/
java
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
31 Commits
2 Branches
0 Tags
4.5 MiB
 
 
 
Branch: main
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'main'
${ noResults }
java/Crawler/logs/crawler.2026-05-25.log

172 lines
18 KiB
Raw Permalink Blame History

2026-05-25 21:53:47.585 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
2026-05-25 21:53:47.980 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
2026-05-25 21:53:48.622 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
2026-05-25 21:53:49.287 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
2026-05-25 21:53:49.995 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
2026-05-25 21:53:50.626 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
2026-05-25 21:53:51.180 [main] INFO CrawlerContext - Crawl completed, 210 items saved to dangdang_books.txt
2026-05-25 21:53:51.180 [main] INFO StrategyCrawlerMain -
=== Crawling EastMoney Fund (target: 200 items) ===
2026-05-25 21:53:51.775 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 1: 15 items
2026-05-25 21:53:52.446 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 2: 15 items
2026-05-25 21:53:53.185 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 3: 15 items
2026-05-25 21:53:53.849 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 4: 15 items
2026-05-25 21:53:54.461 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 5: 15 items
2026-05-25 21:53:55.075 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 6: 15 items
2026-05-25 21:53:55.735 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 7: 15 items
2026-05-25 21:53:56.393 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 8: 15 items
2026-05-25 21:53:56.990 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 9: 15 items
2026-05-25 21:53:57.643 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 10: 15 items
2026-05-25 21:53:58.268 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 11: 15 items
2026-05-25 21:53:58.927 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 12: 15 items
2026-05-25 21:53:59.563 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 13: 15 items
2026-05-25 21:54:00.230 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 14: 15 items
2026-05-25 21:54:00.741 [main] INFO CrawlerContext - Crawl completed, 210 items saved to eastmoney_funds.txt
2026-05-25 21:54:00.742 [main] INFO StrategyCrawlerMain -
=== Crawling Qidian (target: 200 items) ===
2026-05-25 21:54:00.982 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 1: 15 items
2026-05-25 21:54:01.593 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 2: 15 items
2026-05-25 21:54:02.198 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 3: 15 items
2026-05-25 21:54:02.806 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 4: 15 items
2026-05-25 21:54:03.413 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 5: 15 items
2026-05-25 21:54:04.010 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 6: 15 items
2026-05-25 21:54:04.621 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 7: 15 items
2026-05-25 21:54:05.241 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 8: 15 items
2026-05-25 21:54:05.846 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 9: 15 items
2026-05-25 21:54:06.444 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 10: 15 items
2026-05-25 21:54:07.055 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 11: 15 items
2026-05-25 21:54:07.675 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 12: 15 items
2026-05-25 21:54:08.509 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 13: 15 items
2026-05-25 21:54:09.107 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 14: 15 items
2026-05-25 21:54:09.621 [main] INFO CrawlerContext - Crawl completed, 210 items saved to qidian_novels.txt
2026-05-25 21:54:09.621 [main] INFO StrategyCrawlerMain -
=== All crawling tasks completed ===
2026-05-25 21:55:59.169 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
2026-05-25 21:55:59.524 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
2026-05-25 21:56:00.181 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
2026-05-25 21:56:00.850 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
2026-05-25 21:56:01.643 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
2026-05-25 21:56:02.264 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
2026-05-25 21:56:02.793 [main] INFO CrawlerContext - Crawl completed, 210 items saved to dangdang_books.txt
2026-05-25 21:56:02.793 [main] INFO StrategyCrawlerMain -
=== Crawling EastMoney Fund (target: 200 items) ===
2026-05-25 21:56:03.317 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 1: 15 items
2026-05-25 21:56:03.940 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 2: 15 items
2026-05-25 21:56:04.592 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 3: 15 items
2026-05-25 21:56:05.568 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 4: 15 items
2026-05-25 21:56:06.222 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 5: 15 items
2026-05-25 21:56:06.846 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 6: 15 items
2026-05-25 21:56:07.769 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 7: 15 items
2026-05-25 21:56:08.448 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 8: 15 items
2026-05-25 21:56:09.081 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 9: 15 items
2026-05-25 21:56:09.719 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 10: 15 items
2026-05-25 21:56:10.396 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 11: 15 items
2026-05-25 21:56:11.058 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 12: 15 items
2026-05-25 21:56:11.712 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 13: 15 items
2026-05-25 21:56:12.369 [main] INFO CrawlerContext - Crawling https://fund.eastmoney.com/ZTJJ/ Page 14: 15 items
2026-05-25 21:56:12.884 [main] INFO CrawlerContext - Crawl completed, 210 items saved to eastmoney_funds.txt
2026-05-25 21:56:12.884 [main] INFO StrategyCrawlerMain -
=== Crawling Qidian (target: 200 items) ===
2026-05-25 21:56:13.090 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 1: 15 items
2026-05-25 21:56:13.670 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 2: 15 items
2026-05-25 21:56:14.247 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 3: 15 items
2026-05-25 21:56:14.875 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 4: 15 items
2026-05-25 21:56:15.497 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 5: 15 items
2026-05-25 21:56:16.064 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 6: 15 items
2026-05-25 21:56:16.618 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 7: 15 items
2026-05-25 21:56:17.225 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 8: 15 items
2026-05-25 21:56:17.800 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 9: 15 items
2026-05-25 21:56:18.376 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 10: 15 items
2026-05-25 21:56:18.975 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 11: 15 items
2026-05-25 21:56:19.614 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 12: 15 items
2026-05-25 21:56:20.184 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 13: 15 items
2026-05-25 21:56:20.754 [main] INFO CrawlerContext - Crawling https://www.qidian.com/rank/yuepiao/ Page 14: 15 items
2026-05-25 21:56:21.260 [main] INFO CrawlerContext - Crawl completed, 210 items saved to qidian_novels.txt
2026-05-25 21:56:21.260 [main] INFO StrategyCrawlerMain -
=== All crawling tasks completed ===
2026-05-25 22:02:23.166 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
2026-05-25 22:02:23.551 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
2026-05-25 22:02:24.247 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
2026-05-25 22:02:24.880 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
2026-05-25 22:02:25.556 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
2026-05-25 22:02:26.224 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
2026-05-25 22:02:26.771 [main] INFO CrawlerContext - Crawl completed, 210 items saved to dangdang_books.txt
2026-05-25 22:02:26.772 [main] INFO StrategyCrawlerMain -
=== Crawling Weather China (target: 200 items) ===
2026-05-25 22:02:27.346 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 1: 10 items
2026-05-25 22:02:27.977 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 2: 10 items
2026-05-25 22:02:28.598 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 3: 10 items
2026-05-25 22:02:29.176 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 4: 10 items
2026-05-25 22:02:29.825 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 5: 10 items
2026-05-25 22:02:30.442 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 6: 10 items
2026-05-25 22:02:31.038 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 7: 10 items
2026-05-25 22:02:31.686 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 8: 10 items
2026-05-25 22:02:32.266 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 9: 10 items
2026-05-25 22:02:32.896 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 10: 10 items
2026-05-25 22:02:33.543 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 11: 10 items
2026-05-25 22:02:34.159 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 12: 10 items
2026-05-25 22:02:34.757 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 13: 10 items
2026-05-25 22:02:35.395 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 14: 10 items
2026-05-25 22:02:35.910 [main] INFO CrawlerContext - Crawl completed, 140 items saved to weather_china.txt
2026-05-25 22:02:35.910 [main] INFO StrategyCrawlerMain -
=== Crawling Douban Movie (target: 200 items) ===
2026-05-25 22:02:36.326 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 1: 15 items
2026-05-25 22:02:37.108 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 2: 15 items
2026-05-25 22:02:37.816 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 3: 15 items
2026-05-25 22:02:38.530 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 4: 15 items
2026-05-25 22:02:39.247 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 5: 15 items
2026-05-25 22:02:40.019 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 6: 15 items
2026-05-25 22:02:40.738 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 7: 15 items
2026-05-25 22:02:41.451 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 8: 15 items
2026-05-25 22:02:42.194 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 9: 15 items
2026-05-25 22:02:42.881 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 10: 15 items
2026-05-25 22:02:43.635 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 11: 15 items
2026-05-25 22:02:44.375 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 12: 15 items
2026-05-25 22:02:45.099 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 13: 15 items
2026-05-25 22:02:45.805 [main] INFO CrawlerContext - Crawling https://movie.douban.com/chart Page 14: 15 items
2026-05-25 22:02:46.315 [main] INFO CrawlerContext - Crawl completed, 210 items saved to douban_movies.txt
2026-05-25 22:02:46.315 [main] INFO StrategyCrawlerMain -
=== All crawling tasks completed ===
2026-05-25 22:03:48.409 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
2026-05-25 22:03:49.212 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
2026-05-25 22:03:49.873 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
2026-05-25 22:03:50.588 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
2026-05-25 22:03:51.228 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
2026-05-25 22:03:51.932 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
2026-05-25 22:03:52.464 [main] INFO CrawlerContext - Crawl completed, 210 items saved to dangdang_books.txt
2026-05-25 22:03:52.465 [main] INFO StrategyCrawlerMain -
=== Crawling Weather China (target: 200 items) ===
2026-05-25 22:03:52.916 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 1: 10 items
2026-05-25 22:03:53.576 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 2: 10 items
2026-05-25 22:03:54.185 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 3: 10 items
2026-05-25 22:03:54.782 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 4: 10 items
2026-05-25 22:03:55.414 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 5: 10 items
2026-05-25 22:03:56.040 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 6: 10 items
2026-05-25 22:03:56.666 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 7: 10 items
2026-05-25 22:03:57.240 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 8: 10 items
2026-05-25 22:03:57.898 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 9: 10 items
2026-05-25 22:03:58.496 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 10: 10 items
2026-05-25 22:03:59.086 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 11: 10 items
2026-05-25 22:03:59.724 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 12: 10 items
2026-05-25 22:04:00.329 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 13: 10 items
2026-05-25 22:04:00.911 [main] INFO CrawlerContext - Crawling http://www.weather.com.cn/ Page 14: 10 items
2026-05-25 22:04:01.428 [main] INFO CrawlerContext - Crawl completed, 140 items saved to weather_china.txt
2026-05-25 22:04:01.428 [main] INFO StrategyCrawlerMain -
=== Crawling Douban Movie (target: 200 items) ===
2026-05-25 22:04:01.745 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 1: 25 items
2026-05-25 22:04:02.401 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 2: 25 items
2026-05-25 22:04:03.067 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 3: 25 items
2026-05-25 22:04:03.689 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 4: 25 items
2026-05-25 22:04:04.386 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 5: 25 items
2026-05-25 22:04:05.040 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 6: 25 items
2026-05-25 22:04:05.683 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 7: 25 items
2026-05-25 22:04:06.302 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 8: 25 items
2026-05-25 22:04:06.979 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 9: 25 items
2026-05-25 22:04:07.627 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 10: 25 items
2026-05-25 22:04:08.227 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 11: 15 items
2026-05-25 22:04:08.843 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 12: 15 items
2026-05-25 22:04:09.458 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 13: 15 items
2026-05-25 22:04:10.085 [main] INFO CrawlerContext - Crawling https://movie.douban.com/top250?start=%d&filter= Page 14: 15 items
2026-05-25 22:04:10.604 [main] INFO CrawlerContext - Crawl completed, 310 items saved to douban_movies.txt
2026-05-25 22:04:10.604 [main] INFO StrategyCrawlerMain -
=== All crawling tasks completed ===