When you need to do some web scraping job in Python, an excellent choice is the Scrapy framework. Not only it takes care of most of the networking (HTTP, SSL, proxies, etc) but it also facilitates the process of extracting data from the web by providing things such as nifty xpath selectors.
Scrapy is built upon the Twisted networking engine. A limitation of its core component, the reactor, is that it cannot be restarted.