Skip to content
This repository has been archived by the owner on Dec 8, 2017. It is now read-only.

dodsbir scraper is breaking #167

Open
arowla opened this issue Jan 2, 2015 · 1 comment
Open

dodsbir scraper is breaking #167

arowla opened this issue Jan 2, 2015 · 1 comment
Labels

Comments

@arowla
Copy link
Contributor

arowla commented Jan 2, 2015

It gets a long way through the scrape and then fails like so:

Starting dodsibr-nightly Fri Jan  2 11:18:15 EST 2015
FBOPEN_URI = http://localhost:9200
FBOPEN_INDEX = fbopen
Scraping topics JSON...
Traceback (most recent call last):
  File "scrape.py", line 7, in <module>
    s.get_all_topics() #takes one second per topic (roughly 90 seconds total)
  File "/Users/alisonrowland/.virtualenvs/fbopen/lib/python3.4/site-packages/dodsbir/scrape.py", line 130, in get_all_topics
    topic = self.get_topic(key)
  File "/Users/alisonrowland/.virtualenvs/fbopen/lib/python3.4/site-packages/dodsbir/scrape.py", line 114, in get_topic
    topic = self.html_to_topic(resp.text, topic_id)
  File "/Users/alisonrowland/.virtualenvs/fbopen/lib/python3.4/site-packages/dodsbir/scrape.py", line 78, in html_to_topic
    topic.acquisition_program = rows[0].findAll('td')[1].contents[0].string
IndexError: list index out of range
@arowla arowla added the bug label Jan 2, 2015
@DavidEBest
Copy link

This is corrected in PR 18F/dodsbir-scrape#9.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants