Python Tools

urllib

This package contains modules for use with URLs. The urllib module is divided into several sub-modules. It can make requests and receive responses from the server. As a powerful and user-friendly tool, it has several additional features that Python does not such as:

  • Thread safety
  • Connection pooling
  • Client-side SSL/TLS verification
  • File uploads with multipart encoding
  • Helpers for retrying requests and dealing with HTTP redirects
  • Support for gzip, deflate, and brotli encoding
  • Proxy support for HTTP and SOCKS
  • 100% test coverage

Project Background

  • Project: urllib
  • Author:  Andrey Petrov
  • Initial Release: 2019
  • Type: Web Scraping Tool
  • License: MIT Licence
  • Contains: urllib.request, urllib.parse, urllib.error, urllib.robotparser
  • Language: Python
  • GitHub: urllib3/urllib3 with 2.8k stars and 252 contributors
  • Runs On: Windows, Linux, MacOS
  • Twitter: None

Applications

  • urllib.request is used to open and read URLs
  • urllib.parse is used to parse URLs.
  • urllib.error is used to handle exceptions.
  • For processing robot.txt files, use urllib.robotparser.
 
Scroll to Top