python - urllib2.HTTPError: HTTP Error 404: Not Found for valid url -



python - urllib2.HTTPError: HTTP Error 404: Not Found for valid url -

i'm using python opengraph library parse website's opengraph tags https://github.com/erikriver/opengraph.

import opengraph url = 'http://www.foxnews.com/world/2014/10/20/uk-gun-owners-now-subject-to-warrantless-home-searches/' og = opengraph.opengraph(url=url) print og.to_json()

when run script next error

traceback (most recent phone call last): file "test.py", line 16, in <module> raw = urllib2.urlopen(url) file "/usr/lib/python2.7/urllib2.py", line 127, in urlopen homecoming _opener.open(url, data, timeout) file "/usr/lib/python2.7/urllib2.py", line 410, in open response = meth(req, response) file "/usr/lib/python2.7/urllib2.py", line 523, in http_response 'http', request, response, code, msg, hdrs) file "/usr/lib/python2.7/urllib2.py", line 448, in error homecoming self._call_chain(*args) file "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) file "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default raise httperror(req.get_full_url(), code, msg, hdrs, fp) urllib2.httperror: http error 404: not found

urllib2 used under hood grab html before parsed https://github.com/erikriver/opengraph/blob/master/opengraph/opengraph.py#l50-l52

why receiving 404 error? can access url browser , retrieve open graph tags url using php library https://github.com/scottmac/opengraph.

the python library able retrieve open graph tags other urls url seems anomaly.

updated:

you 404 response because request hasn't passed user-agent. installed opengraph on virtualenv test it, works after adding missing user-agent in header:

url = 'http://www.foxnews.com/world/2014/10/20/uk-gun-owners-now-subject-to-warrantless-home-searches/' req = opengraph.opengraph.urllib2.request(url, headers={ 'user-agent': 'mozilla/5.0' }) og = opengraph.opengraph() og.parser(opengraph.opengraph.urllib2.urlopen(req).read()) og.to_json() '{"site_name": "fox news", "description": "registered gun owners in united kingdom of britain and northern republic of ireland subject unannounced visits homes under new guidance allows police force inspect firearms storage without warrant.", "title": "uk gun owners subject warrantless home searches", "url": "http://www.foxnews.com/world/2014/10/20/uk-gun-owners-now-subject-to-warrantless-home-searches/", "image": "http://global.fncstatic.com/static/v/all/img/fn_128x128.png", "scrape": false, "_url": null, "type": "article"}'

python python-2.7 facebook-opengraph urllib2 opengraph

Comments

Post a Comment

Popular posts from this blog

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* -

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* - i've inherited doc publishing process takes docbook , builds html , pdf output, using apache fop both. project had been started, never completed. lots of tweaking, i've able of doc sets build (20+), except handful, , fail severe code 1 of these 2 conditions: "fo:flow" missing kid elements. required content model: marker* "fo:block" not valid kid of "fo:root" the xml output comes out of docbook , docbook-fop checks out well-formed. the fop error in 1 book, "endecagloss", example, points end of string: <fo:flow flow-name="xsl-region-body" start-indent="4pc" end-indent="0pt"/> here’s log output in case: [java] severe: javax.xml.transform.transformerexception: file:/scratch/publishing/hudson/jobs/build-main-endeca-documentation/workspace/serverdoc/w...
Read more

mediawiki - How do I insert tables inside infoboxes on Wikia pages? -

mediawiki - How do I insert tables inside infoboxes on Wikia pages? - i want insert tables in infoboxes. example, have proteinbox i've developed using infobox template (http://health-and-medicine.wikia.com/wiki/template:proteinbox), , fasta field big fit comfortably in proteinbox. hence collapsible table within box. this tried: {{infobox | row 1 = {| class = "mw-collapsible mw-collapsed wikitable" |- | <!-- table content --> |} }} your problem is, putting nested table within template. pipe characters ( | ) in table syntax collides pipe usage in templates. the commonly used hack around create template called template:! , containing pipe character, , utilize when need set tables, parser functions or other stuff using pipe characters, within templates. table (with every | replaced {{!}} ): {{{!}} {{!}}- {{!}} a1 {{!}} b1 {{!}}- {{!}} a2 {{!}} b2 {{!}}} ...the equivalent of {| |- | a1 | b1 |- | a2 | b2 |} furthermore, hav...
Read more

Local Service User Logged into Windows -

Local Service User Logged into Windows - i created windows service tracks app server user connected through load balancer. it retrieves user logged windows on machine, cross references active directory (ad) business relationship homecoming first name, lastly name , e-mail (for notification through website created project) there 1 scheme on network users remote system, vista system, reason returning values "local service" logged machine, it's weird , security guys impossible "local service" business relationship logged windows. here how user logged windows managementscope ms = new managementscope("\\\\.\\root\\cimv2"); objectquery query = new objectquery("select * win32_computersystem"); managementobjectsearcher searcher = new managementobjectsearcher(ms, query); foreach (managementobject mo in searcher.get()) { _username = mo["username"].tostring(); } // remove domain part username string[] usernameparts = _...
Read more