i trying download images url's change got error.
url_image="http://www.joblo.com/timthumb.php?src=/posters/images/full/"+str(title_2)+"-poster1.jpg&h=333&w=225" user_agent = 'mozilla/5.0 (windows nt 6.1; win64; x64)' headers = {'user-agent': user_agent} req = urllib.request.request(url_image, none, headers) print(url_image) #image, h = urllib.request.urlretrieve(url_image) urllib.request.urlopen(req) response: the_page = response.read() #print (the_page) open('poster.jpg', 'wb') f: f.write(the_page) traceback (most recent call last): file "c:\users\luke\desktop\scraper\imager finder.py", line 97, in urllib.request.urlopen(req) response: file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 162, in urlopen return opener.open(url, data, timeout) file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 465, in open response = self._open(req, data) file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 483, in _open '_open', req) file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 443, in _call_chain result = func(*args) file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 1268, in http_open return self.do_open(http.client.httpconnection, req) file "c:\users\luke\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 1243, in do_open r = h.getresponse() file "c:\users\luke\appdata\local\programs\python\python35-32\lib\http\client.py", line 1174, in getresponse response.begin() file "c:\users\luke\appdata\local\programs\python\python35-32\lib\http\client.py", line 282, in begin version, status, reason = self._read_status() file "c:\users\luke\appdata\local\programs\python\python35-32\lib\http\client.py", line 264, in _read_status raise badstatusline(line) http.client.badstatusline:
my advice use urlib2. in addition, i've written nice function (i think) allow gzip encoding (reduce bandwidth) if server supports it. use downloading social media files, should work anything.
i try debug code, since it's snippet (and error messages formatted badly), it's hard know error occurring (it's not line 97 in code snippet).
this isn't short be, it's clear , reusable. python 2.7, looks you're using 3 - in case google other questions address how use urllib2 in python 3.
import urllib2 import gzip stringio import stringio def download(url): """ download , return file specified in url; attempt use gzip encoding if possible. """ request = urllib2.request(url) request.add_header('accept-encoding', 'gzip') try: response = urllib2.urlopen(request) except exception, e: raise ioerror("%s(%s) %s" % (_errors[1], url, e)) payload = response.read() if response.info().get('content-encoding') == 'gzip': buf = stringio(payload) f = gzip.gzipfile(fileobj=buf) payload = f.read() return payload def save_media(filename, media): file_handle = open(filename, "wb") file_handle.write(media) file_handle.close() title_2 = "10-cloverfield-lane" media = download("http://www.joblo.com/timthumb.php?src=/posters/images/full/{}-poster1.jpg&h=333&w=225".format(title_2)) save_media("poster.jpg", media) 
Comments
Post a Comment