You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
95 lines
8.9 KiB
95 lines
8.9 KiB
2026-05-21 14:55:20.207 [main] INFO StrategyCrawlerMain - === Crawling DangDang (target: 200 items) ===
|
|
2026-05-21 14:55:20.582 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 1: 46 items
|
|
2026-05-21 14:55:21.240 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 2: 46 items
|
|
2026-05-21 14:55:21.900 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 3: 46 items
|
|
2026-05-21 14:55:22.567 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 4: 46 items
|
|
2026-05-21 14:55:23.224 [main] INFO CrawlerContext - Crawling http://bang.dangdang.com/books/bestsellers/%d Page 5: 46 items
|
|
2026-05-21 14:55:23.787 [main] INFO CrawlerContext - Crawl completed, 230 items saved to dangdang_books.txt
|
|
2026-05-21 14:55:23.788 [main] INFO StrategyCrawlerMain -
|
|
=== Crawling MaoYan (target: 200 items) ===
|
|
2026-05-21 14:55:24.611 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 1: 15 items
|
|
2026-05-21 14:55:25.569 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 2: 15 items
|
|
2026-05-21 14:55:26.498 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 3: 15 items
|
|
2026-05-21 14:55:27.470 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 4: 15 items
|
|
2026-05-21 14:55:28.302 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 5: 15 items
|
|
2026-05-21 14:55:29.087 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 6: 15 items
|
|
2026-05-21 14:55:29.853 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 7: 15 items
|
|
2026-05-21 14:55:30.631 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 8: 15 items
|
|
2026-05-21 14:55:31.428 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 9: 15 items
|
|
2026-05-21 14:55:32.223 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 10: 15 items
|
|
2026-05-21 14:55:33.021 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 11: 15 items
|
|
2026-05-21 14:55:33.820 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 12: 15 items
|
|
2026-05-21 14:55:34.601 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 13: 15 items
|
|
2026-05-21 14:55:35.385 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 14: 15 items
|
|
2026-05-21 14:55:36.193 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 15: 15 items
|
|
2026-05-21 14:55:37.190 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 16: 15 items
|
|
2026-05-21 14:55:37.989 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 17: 15 items
|
|
2026-05-21 14:55:38.910 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 18: 15 items
|
|
2026-05-21 14:55:39.703 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 19: 15 items
|
|
2026-05-21 14:55:40.480 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 20: 15 items
|
|
2026-05-21 14:55:41.276 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 21: 15 items
|
|
2026-05-21 14:55:42.057 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 22: 15 items
|
|
2026-05-21 14:55:42.833 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 23: 15 items
|
|
2026-05-21 14:55:43.607 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 24: 15 items
|
|
2026-05-21 14:55:44.395 [main] INFO CrawlerContext - Crawling https://www.maoyan.com/ Page 25: 15 items
|
|
2026-05-21 14:55:44.979 [main] INFO CrawlerContext - Crawl completed, 375 items saved to maoyan_movies.txt
|
|
2026-05-21 14:55:44.980 [main] INFO StrategyCrawlerMain -
|
|
=== Crawling JD (target: 200 items) ===
|
|
2026-05-21 14:55:45.277 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 1: 15 items
|
|
2026-05-21 14:55:45.987 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 2: 15 items
|
|
2026-05-21 14:55:46.679 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 3: 15 items
|
|
2026-05-21 14:55:47.377 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 4: 15 items
|
|
2026-05-21 14:55:48.048 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 5: 15 items
|
|
2026-05-21 14:55:48.756 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 6: 15 items
|
|
2026-05-21 14:55:49.437 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 7: 15 items
|
|
2026-05-21 14:55:50.103 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 8: 15 items
|
|
2026-05-21 14:55:50.691 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 9: 15 items
|
|
2026-05-21 14:55:51.280 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 10: 15 items
|
|
2026-05-21 14:55:51.937 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 11: 15 items
|
|
2026-05-21 14:55:52.628 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 12: 15 items
|
|
2026-05-21 14:55:53.299 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 13: 15 items
|
|
2026-05-21 14:55:54.002 [main] INFO CrawlerContext - Crawling https://list.jd.com/list.html?cat=1672,3272&page=%d Page 14: 15 items
|
|
2026-05-21 14:55:54.559 [main] INFO CrawlerContext - Crawl completed, 210 items saved to jd_products.txt
|
|
2026-05-21 14:55:54.559 [main] INFO StrategyCrawlerMain -
|
|
=== All crawling tasks completed ===
|
|
2026-05-21 15:37:36 [main] INFO view.CrawlerView - ����ϵͳ����
|
|
2026-05-21 15:37:36 [main] DEBUG view.CrawlerView - ��ʾ���˵�
|
|
2026-05-21 15:37:55 [main] INFO view.CrawlerView - ����ϵͳ����
|
|
2026-05-21 15:37:55 [main] DEBUG view.CrawlerView - ��ʾ���˵�
|
|
2026-05-21 15:37:55 [main] INFO view.CrawlerView - ��ʼ��ȡ: ��������
|
|
2026-05-21 15:37:56 [main] DEBUG view.CrawlerView - �������� �� 1 ҳ: 0 ������
|
|
2026-05-21 15:37:56 [main] DEBUG view.CrawlerView - �������� �� 2 ҳ: 0 ������
|
|
2026-05-21 15:37:57 [main] DEBUG view.CrawlerView - �������� �� 3 ҳ: 0 ������
|
|
2026-05-21 15:37:58 [main] DEBUG view.CrawlerView - �������� �� 4 ҳ: 0 ������
|
|
2026-05-21 15:37:58 [main] DEBUG view.CrawlerView - �������� �� 5 ҳ: 0 ������
|
|
2026-05-21 15:37:59 [main] DEBUG view.CrawlerView - �������� �� 6 ҳ: 0 ������
|
|
2026-05-21 15:38:00 [main] DEBUG view.CrawlerView - �������� �� 7 ҳ: 0 ������
|
|
2026-05-21 15:38:00 [main] DEBUG view.CrawlerView - �������� �� 8 ҳ: 0 ������
|
|
2026-05-21 15:38:01 [main] DEBUG view.CrawlerView - �������� �� 9 ҳ: 0 ������
|
|
2026-05-21 15:38:02 [main] DEBUG view.CrawlerView - �������� �� 10 ҳ: 0 ������
|
|
2026-05-21 15:38:03 [main] DEBUG view.CrawlerView - �������� �� 11 ҳ: 0 ������
|
|
2026-05-21 15:38:03 [main] DEBUG view.CrawlerView - �������� �� 12 ҳ: 0 ������
|
|
2026-05-21 15:38:04 [main] DEBUG view.CrawlerView - �������� �� 13 ҳ: 0 ������
|
|
2026-05-21 15:38:05 [main] DEBUG view.CrawlerView - �������� �� 14 ҳ: 0 ������
|
|
2026-05-21 15:38:05 [main] INFO repository.Repository - Successfully saved 210 items to CSV: jd_clothing.txt
|
|
2026-05-21 15:38:05 [main] INFO view.CrawlerView - �����ѱ��浽�ļ�: jd_clothing.txt
|
|
2026-05-21 15:38:05 [main] INFO repository.Repository - Successfully saved 210 items to JSON: jd_clothing.json
|
|
2026-05-21 15:38:05 [main] INFO view.CrawlerView - �����ѱ��浽�ļ�: jd_clothing.json
|
|
2026-05-21 15:38:05 [main] INFO view.CrawlerView - �������� ��ȡ���ɣ��� 210 ������
|
|
2026-05-21 15:38:05 [main] DEBUG view.CrawlerView - ��ʾ���˵�
|
|
2026-05-21 15:50:06 [main] INFO view.CrawlerView - ����ϵͳ����
|
|
2026-05-21 15:50:06 [main] DEBUG view.CrawlerView - ��ʾ���˵�
|
|
2026-05-21 15:50:06 [main] INFO view.CrawlerView - ��ʼ��ȡ: ��������
|
|
2026-05-21 15:50:09 [main] DEBUG view.CrawlerView - �������� �� 1 ҳ: 0 ������
|
|
2026-05-21 15:50:09 [main] ERROR view.CrawlerView - ��������: ҳ�� 1 ���ؿ�����
|
|
2026-05-21 15:50:11 [main] DEBUG view.CrawlerView - �������� �� 2 ҳ: 0 ������
|
|
2026-05-21 15:50:11 [main] ERROR view.CrawlerView - ��������: ҳ�� 2 ���ؿ�����
|
|
2026-05-21 15:50:16 [main] DEBUG view.CrawlerView - �������� �� 3 ҳ: 0 ������
|
|
2026-05-21 15:50:16 [main] ERROR view.CrawlerView - ��������: ҳ�� 3 ���ؿ�����
|
|
2026-05-21 15:50:16 [main] WARN view.CrawlerView - �������� ʵ����ȡ���ݲ��㣬����ģ������
|
|
2026-05-21 15:50:16 [main] INFO repository.Repository - Successfully saved 200 items to CSV: jd_clothing.txt
|
|
2026-05-21 15:50:16 [main] INFO view.CrawlerView - �����ѱ��浽�ļ�: jd_clothing.txt
|
|
2026-05-21 15:50:16 [main] INFO repository.Repository - Successfully saved 200 items to JSON: jd_clothing.json
|
|
2026-05-21 15:50:16 [main] INFO view.CrawlerView - �����ѱ��浽�ļ�: jd_clothing.json
|
|
2026-05-21 15:50:16 [main] INFO view.CrawlerView - �������� ��ȡ���ɣ��� 200 ������
|
|
2026-05-21 15:50:16 [main] DEBUG view.CrawlerView - ��ʾ���˵�
|
|
|