Udemyallzh

快速掌握Python分布式爬虫

Learn web scraping with Python! Master Scrapy, Requests, pygal & Jupyter. Extract data & visualize insights. Perfect for data enthusiasts.

4.0(69)
817students
20h 5m
Certificate
Song Hu
Song HuPython development engineer

Course Content

快速掌握Python分布式爬虫 is an all-levels Udemy course taught by Song Hu. It runs 20h 5m, is taught in Chinese, and includes a certificate of completion. Learners rate it 4.0 out of 5 from 69 reviews, with 817 students enrolled. It is a paid course, last updated July 2024.

Learn Web Scraping with Python & Scrapy: From Beginner to Advanced

This comprehensive course will take you from basic web scraping with Python scripts to building robust, scalable web crawlers using the powerful Scrapy framework. You'll gain practical skills and a deep understanding of web scraping techniques, ready to tackle real-world projects.

What You'll Get:

  • Video Tutorials & Source Code: Access both video lessons and complete, working code for each topic.
  • Detailed Q&A Support: I'm committed to providing thorough answers to all your questions. I've invested over a year developing this course, and I'll do my best to guide you.
  • Ongoing Content: New content is continuously being added, including advanced topics like distributed crawling and practical exercises.
  • Essential Knowledge: This course covers the fundamental concepts of web scraping, equipping you with the skills to quickly get started.
  • Bonus Insights: The Q&A section contains valuable, in-depth theoretical knowledge that isn't always covered in the video lessons.

Course Curriculum:

This course follows a structured learning path, progressing from simple scripts to complex Scrapy projects. Each video is concise and focused on practical application. You'll also receive PDF summaries to reinforce your learning.

Part 1: Python Scripting for Web Scraping

  • Development Environment Setup: Learn how to configure your environment for successful web scraping.
  • Mastering the requests Library: Efficiently fetch web pages using Python's requests library.
  • Data Visualization: Visualize your scraped data with charts and graphs using libraries like pygal.
  • Image Extraction & Downloading: Extract images from web pages using XPath, and automate their download and saving.
  • Anti-Ban Techniques: Implement user-agent rotation and IP proxy usage to avoid being blocked. Includes libraries: sqlalchemy, pygal, sqlite, requests, lxml, jupyter.

Part 2: Scrapy Framework

  • Scrapy Commands & Basic Crawling: Learn the essential Scrapy commands and build your first web crawler.
  • Scrapy Architecture & Callback Functions: Understand the Scrapy framework's architecture and how to use callback functions for data processing.
  • CrawlSpider vs. Spider: Learn when to use each type of spider for different crawling scenarios.
  • Scrapy Pipelines: Utilize Scrapy's built-in pipeline system for data cleaning, validation, and storage.
  • Multi-Web Crawling: Develop Scrapy projects that crawl multiple websites.
  • Understanding Anti-Scraping Measures: Learn about common anti-scraping techniques and how to overcome them.
  • User-Agent & Proxy Management: Dynamically switch user-agents and proxy IPs using middleware.
  • Scrapy Middleware: Explore and utilize various Scrapy middleware components to enhance your crawlers.

Why This Course?

Web scraping is a skill that requires hands-on practice. This course provides the knowledge and tools you need to succeed. Don't hesitate to ask questions in the Q&A section – I'm here to help you every step of the way. While no single course can cover everything about web scraping, this will provide a solid foundation and fast-track your learning journey.

$ whoami — the devs behind tutorialsearch.io

Need a web app built? Hire the team behind TutorialSearch.io

We ship fast, production-grade full-stack apps (Next.js, React, edge infra) — the same stack that powers this site.