You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
159 lines
17 KiB
159 lines
17 KiB
2026-05-19 17:05:18.586 [main] INFO s.StrategyCrawlerMain - === Crawling DangDang ===
|
|
2026-05-19 17:05:19.280 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
|
|
2026-05-19 17:05:19.938 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
|
|
2026-05-19 17:05:20.807 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
|
|
2026-05-19 17:05:21.356 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 126 items saved to strategy_crawler/dangdang_books.txt
|
|
2026-05-19 17:05:21.356 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling MaoYan ===
|
|
2026-05-19 17:05:22.634 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 8 items
|
|
2026-05-19 17:05:23.148 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 8 items saved to strategy_crawler/maoyan_movies.txt
|
|
2026-05-19 17:05:23.148 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling JD ===
|
|
2026-05-19 17:05:23.410 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-19 17:05:24.086 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-19 17:05:24.773 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-19 17:05:25.293 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 45 items saved to strategy_crawler/jd_products.txt
|
|
2026-05-19 17:05:25.293 [main] INFO s.StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
2026-05-19 17:11:58.622 [main] INFO s.StrategyCrawlerMain - === Crawling DangDang ===
|
|
2026-05-19 17:11:59.066 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
|
|
2026-05-19 17:11:59.786 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
|
|
2026-05-19 17:12:00.458 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
|
|
2026-05-19 17:12:01.028 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 126 items saved to strategy_crawler/dangdang_books.txt
|
|
2026-05-19 17:12:01.028 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling MaoYan ===
|
|
2026-05-19 17:12:02.114 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 8 items
|
|
2026-05-19 17:12:02.638 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 8 items saved to strategy_crawler/maoyan_movies.txt
|
|
2026-05-19 17:12:02.643 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling JD ===
|
|
2026-05-19 17:12:02.902 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-19 17:12:03.608 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-19 17:12:04.330 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-19 17:12:04.845 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 45 items saved to strategy_crawler/jd_products.txt
|
|
2026-05-19 17:12:04.846 [main] INFO s.StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
2026-05-19 17:18:06.991 [main] INFO s.StrategyCrawlerMain - === Crawling DangDang ===
|
|
2026-05-19 17:18:07.413 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
|
|
2026-05-19 17:18:08.078 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
|
|
2026-05-19 17:18:08.720 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
|
|
2026-05-19 17:18:09.296 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 126 items saved to strategy_crawler/dangdang_books.txt
|
|
2026-05-19 17:18:09.296 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling MaoYan ===
|
|
2026-05-19 17:18:10.452 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 8 items
|
|
2026-05-19 17:18:10.972 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 8 items saved to strategy_crawler/maoyan_movies.txt
|
|
2026-05-19 17:18:10.972 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling JD ===
|
|
2026-05-19 17:18:11.307 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-19 17:18:12.011 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-19 17:18:12.698 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-19 17:18:13.208 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 45 items saved to strategy_crawler/jd_products.txt
|
|
2026-05-19 17:18:13.209 [main] INFO s.StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
2026-05-19 17:20:32.737 [main] INFO s.StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
|
|
2026-05-19 17:20:33.107 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
|
|
2026-05-19 17:20:33.866 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
|
|
2026-05-19 17:20:34.583 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
|
|
2026-05-19 17:20:35.297 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
|
|
2026-05-19 17:20:35.946 [main] INFO strategy_crawler.CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
|
|
2026-05-19 17:20:36.490 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 210 items saved to strategy_crawler/dangdang_books.txt
|
|
2026-05-19 17:20:36.491 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling MaoYan (target: 200 items) ===
|
|
2026-05-19 17:20:37.699 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 8 items
|
|
2026-05-19 17:20:38.584 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 2: 8 items
|
|
2026-05-19 17:20:39.345 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 3: 8 items
|
|
2026-05-19 17:20:40.123 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 4: 8 items
|
|
2026-05-19 17:20:40.937 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 5: 8 items
|
|
2026-05-19 17:20:42.083 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 6: 8 items
|
|
2026-05-19 17:20:42.880 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 7: 8 items
|
|
2026-05-19 17:20:43.661 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 8: 8 items
|
|
2026-05-19 17:20:44.425 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 9: 8 items
|
|
2026-05-19 17:20:45.196 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 10: 8 items
|
|
2026-05-19 17:20:45.949 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 11: 8 items
|
|
2026-05-19 17:20:46.998 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 12: 8 items
|
|
2026-05-19 17:20:47.749 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 13: 8 items
|
|
2026-05-19 17:20:48.668 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 14: 8 items
|
|
2026-05-19 17:20:49.403 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 15: 8 items
|
|
2026-05-19 17:20:50.309 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 16: 8 items
|
|
2026-05-19 17:20:51.101 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 17: 8 items
|
|
2026-05-19 17:20:52.188 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 18: 8 items
|
|
2026-05-19 17:20:52.953 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 19: 8 items
|
|
2026-05-19 17:20:53.748 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 20: 8 items
|
|
2026-05-19 17:20:54.481 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 21: 8 items
|
|
2026-05-19 17:20:55.293 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 22: 8 items
|
|
2026-05-19 17:20:56.075 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 23: 8 items
|
|
2026-05-19 17:20:56.977 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 24: 8 items
|
|
2026-05-19 17:20:57.744 [main] INFO strategy_crawler.CrawlerContext - Crawling https://www.maoyan.com/ Page 25: 8 items
|
|
2026-05-19 17:20:58.269 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 200 items saved to strategy_crawler/maoyan_movies.txt
|
|
2026-05-19 17:20:58.288 [main] INFO s.StrategyCrawlerMain -
|
|
=== Crawling JD (target: 200 items) ===
|
|
2026-05-19 17:20:58.555 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-19 17:20:59.244 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-19 17:21:00.027 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-19 17:21:00.727 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 4: 15 items
|
|
2026-05-19 17:21:01.498 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 5: 15 items
|
|
2026-05-19 17:21:02.171 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 6: 15 items
|
|
2026-05-19 17:21:02.764 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 7: 15 items
|
|
2026-05-19 17:21:03.498 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 8: 15 items
|
|
2026-05-19 17:21:04.230 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 9: 15 items
|
|
2026-05-19 17:21:04.947 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 10: 15 items
|
|
2026-05-19 17:21:05.707 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 11: 15 items
|
|
2026-05-19 17:21:06.396 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 12: 15 items
|
|
2026-05-19 17:21:07.095 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 13: 15 items
|
|
2026-05-19 17:21:07.820 [main] INFO strategy_crawler.CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 14: 15 items
|
|
2026-05-19 17:21:08.328 [main] INFO strategy_crawler.CrawlerContext - Crawl completed, 210 items saved to strategy_crawler/jd_products.txt
|
|
2026-05-19 17:21:08.329 [main] INFO s.StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
2026-05-19 17:28:37.850 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
|
|
2026-05-19 17:28:38.260 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 42 items
|
|
2026-05-19 17:28:38.922 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 42 items
|
|
2026-05-19 17:28:39.696 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 42 items
|
|
2026-05-19 17:28:40.371 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 42 items
|
|
2026-05-19 17:28:41.109 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 42 items
|
|
2026-05-19 17:28:41.662 [main] INFO CrawlerContext - Crawl completed, 210 items saved to dangdang_books.txt
|
|
2026-05-19 17:28:41.663 [main] INFO StrategyCrawlerMain -
|
|
=== Crawling MaoYan (target: 200 items) ===
|
|
2026-05-19 17:28:42.492 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 8 items
|
|
2026-05-19 17:28:43.318 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 2: 8 items
|
|
2026-05-19 17:28:44.171 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 3: 8 items
|
|
2026-05-19 17:28:45.015 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 4: 8 items
|
|
2026-05-19 17:28:45.768 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 5: 8 items
|
|
2026-05-19 17:28:46.531 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 6: 8 items
|
|
2026-05-19 17:28:47.298 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 7: 8 items
|
|
2026-05-19 17:28:48.077 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 8: 8 items
|
|
2026-05-19 17:28:48.904 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 9: 8 items
|
|
2026-05-19 17:28:49.684 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 10: 8 items
|
|
2026-05-19 17:28:50.537 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 11: 8 items
|
|
2026-05-19 17:28:51.348 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 12: 8 items
|
|
2026-05-19 17:28:52.125 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 13: 8 items
|
|
2026-05-19 17:28:52.891 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 14: 8 items
|
|
2026-05-19 17:28:53.625 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 15: 8 items
|
|
2026-05-19 17:28:54.704 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 16: 8 items
|
|
2026-05-19 17:28:55.452 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 17: 8 items
|
|
2026-05-19 17:28:56.172 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 18: 8 items
|
|
2026-05-19 17:28:56.896 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 19: 8 items
|
|
2026-05-19 17:28:57.657 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 20: 8 items
|
|
2026-05-19 17:28:58.457 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 21: 8 items
|
|
2026-05-19 17:28:59.309 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 22: 8 items
|
|
2026-05-19 17:29:00.057 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 23: 8 items
|
|
2026-05-19 17:29:00.825 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 24: 8 items
|
|
2026-05-19 17:29:01.742 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 25: 8 items
|
|
2026-05-19 17:29:02.264 [main] INFO CrawlerContext - Crawl completed, 200 items saved to maoyan_movies.txt
|
|
2026-05-19 17:29:02.264 [main] INFO StrategyCrawlerMain -
|
|
=== Crawling JD (target: 200 items) ===
|
|
2026-05-19 17:29:02.535 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-19 17:29:03.241 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-19 17:29:03.968 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-19 17:29:04.687 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 4: 15 items
|
|
2026-05-19 17:29:05.358 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 5: 15 items
|
|
2026-05-19 17:29:06.072 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 6: 15 items
|
|
2026-05-19 17:29:06.770 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 7: 15 items
|
|
2026-05-19 17:29:07.501 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 8: 15 items
|
|
2026-05-19 17:29:08.194 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 9: 15 items
|
|
2026-05-19 17:29:08.891 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 10: 15 items
|
|
2026-05-19 17:29:09.577 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 11: 15 items
|
|
2026-05-19 17:29:10.183 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 12: 15 items
|
|
2026-05-19 17:29:10.886 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 13: 15 items
|
|
2026-05-19 17:29:11.670 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 14: 15 items
|
|
2026-05-19 17:29:12.186 [main] INFO CrawlerContext - Crawl completed, 210 items saved to jd_products.txt
|
|
2026-05-19 17:29:12.186 [main] INFO StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
|