Skip to content

Commit

Permalink
add jbovlaste xml export download script
Browse files Browse the repository at this point in the history
The readme is also amended to use this script rather than wget, since
the introduction of the captcha on jbovlaste has broken the old way.

Closes #4
  • Loading branch information
tsani committed Nov 2, 2015
1 parent 92cdec5 commit f1d45d4
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 2 deletions.
28 changes: 28 additions & 0 deletions download.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/usr/bin/env python

import requests

CREDENTIALS = dict(
username='YOUR JBOVLASTE USERNAME',
password='YOUR JBOVLASTE PASSWORD',
)

# jbovlaste definitions language to download
LANG = 'en'

if __name__ == '__main__':
s = requests.Session()
r = s.post(
'http://jbovlaste.lojban.org/login.html',
data=CREDENTIALS,
)
r.raise_for_status()

r = s.get(
'http://jbovlaste.lojban.org/export/xml-export.html?lang=' + LANG,
)

r.raise_for_status()

with open('vlasisku/data/jbovlaste.xml', 'wb') as f:
f.write(r.text.encode('utf-8'))
6 changes: 4 additions & 2 deletions readme.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,17 @@ Recommended for developers:
Installation for developers: |
git clone https://github.com/lojban/vlasisku
cd vlasisku
wget 'http://jbovlaste.lojban.org/export/xml-export.html?lang=en' \
-O vlasisku/data/jbovlaste.xml
# Isolated Python environment
mkvirtualenv --no-site-packages vlasisku
# Installs to that environment
pip install -r requirements.txt
# Modify download.py to use your jbovlaste credentials, then use it
# to download a jbovlaste XML dump. (It takes a while to run.)
./download.py
# This takes some 20 seconds the first time
# and must be done whenever the jbovlaste export is changed
./manage.py runserver
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ simplejson
pystemmer
Twisted
jellyfish
requests

0 comments on commit f1d45d4

Please sign in to comment.