First complete version

.gitignore

@@ -0,0 +1,6 @@
+*.pyc
+bggcli.egg-info
+build
+dist
+.idea/
+.coverage

.travis.yml

@@ -0,0 +1 @@
+language: pythonpython:  - "2.7"install:  - pip install python-coveralls  - python setup.py installscript:  - coverage run --source bggcli -m py.test -safter_success:  - coverallsaddons:  sauce_connect:    username: "bggcli"    access_key:      secure: "JxDE3vOYP5GIELg60rZSp0p+zkxolTSF4xXwqi/zZuz5a5lXHVLKwuWU+Q7Sj/jZYx0t9/Ggmxa3lnJwKqp4BrYES6jRt9gUfTDtQJlSkHe9Z6YqSaFq7UYHT0/7rBX3LYaip3y1gHp6oiQu/gq3fp1o0MK766rWnXi7xaQYsL56HereMcf8CRRH0iI0PIgLV6OZU6+a6YGCaagmLIfCVIP7TEFKP+hJ8I1BRtFaMYKFdNJVigrK/MQCfWM4E2UzvPW5qtJWlcRe90xresFaduLcI8B6P6fuKDcnrXp++yQJb0YisBu3KS1KCYRMKjqc/q+B3bVa0uENRP0umuoo1XzhQABRkMCZVg4ccgU82lF82/dTUMRMOHddP6N90ZMugtVYnmnURcAA4jzM0MeeZVh4mUVr9oshjB8couGZ8IzmIzAYkbtoBpfMzwT5+bToqJkA625SErKPv+vRqdCaJGlXKlxCWjB+pOsYV4ol1p+GwtvaPXcIe4Rk0YtAOyNbvXNXbom6X5ihDJr6LLnA8cJqZ7RZv+A2m1um3K1sdrQ6SNP+C1Hf5p9jcwIrvuSp9mDmJYZGNlfErRwWsMcL1lS1vXwId7CX0qT+iTmc9GVdUvMLHfJoZxA2bZ3DRv8PBSxEF+BHcCCYtmen4qsUgnA7e/XjrDjJcq803Fd6tjM="

README.rst

@@ -0,0 +1,136 @@
+=====================================================
+bggcli - Command Line Interface for BoardGameGeek.com
+=====================================================
+
+.. image:: https://travis-ci.org/syllant/bggcli.svg?branch=master
+    :target: https://travis-ci.org/syllant/bggcli
+
+
+.. image:: https://coveralls.io/repos/syllant/bggcli/badge.svg?branch=master
+  :target: https://coveralls.io/r/syllant/bggcli?branch=master
+
+Introduction
+============
+
+
+``bggcli`` is a Command Line Interface providing automation for tedious tasks on
+`BoardGameGeek <http://www .boardgamegeek.com>`__ (aka BGG). It relies on the Web UI and not on the
+`official API <https://www.boardgamegeek.com/wiki/page/BGG_XML_API2>`__ which doesn't offer all features available.
+
+Only 3 operations are implemented at this time:
+
+* bulk import/update for your game collection from a CSV file
+* bulk delete from a CSV file
+* bulk export as a CSV file, WITH version information (game's version is missing in the default export)
+
+Warning:
+
+   * Use it at your own risks, you may damage your game collection by doing mistakes! An export is strongly recommended
+     before any operation.
+   * This tool is not supported by BoardGameGeek, this is an independent development
+   * Be respectful regarding BGG web site: this kind of automated tools can impact performance when used
+     "aggressively" (plenty of requests per second). Provided features are intended to be used for
+     one-shot needs. Also they rely on a real web browser, and should conform with their
+     `Terms of Services <https://www.boardgamegeek.com/terms>`__
+
+
+Installation
+============
+
+::
+
+    pip install bggcli
+
+Usage
+=====
+You'll need **Firefox** to be installed; Firefox will be automatically controlled by ``bggcli`` to perform operations
+(through Selenium library).
+
+Type ``bggcli`` to get the full help about available commands.
+
+Here is an example to export a collection from an account *account1* and import it to another account *account2*:::
+
+    $ bggcli -l account1 -p mypassword1 collection-export mycollection.csv
+    $ bggcli -l account2 -p mypassword collection-import mycollection.csv
+
+Update a collection
+-------------------
+Here are some use cases this operation could be used for:
+
+* Create a new account on BGG and transfer your collection: export the collection from the old account first, then use **bggcli** to import it
+* Make a bulk update for all or some of your games: export the collection from your account first, modify details in the CSV file (using a text editor, OpenOffice, MS Excel, or whatever) and use ``bggcli`` to import the file
+
+Export can be done with this tool or manually when you view your collection online.
+
+Example:::
+
+    $ bggcli -l mylogin -p mypassword collection-import mycollection.csv
+
+Notes:
+
+   * Column names are those exported by BGG. Any column not recognized will just be ignored
+   * When a game already exists in your collection, game is just updated with CSV values
+   * Update is only done for fields defined in the CSV file. Other fields already filled online are not impacted. E.g. you could only update one field for all your game
+   * Games are identified by their internal ID, named ``objectid`` in CSV file (name used by BGG)
+
+
+Remove games from a collection
+------------------------------
+Goal is to remove from your collection all games identified in the CSV file you will provide as input.
+
+Example:::
+
+    $ bggcli -l mylogin -p mypassword collection-delete mycollection.csv
+
+Notes:
+
+  * Only the ``objectid`` column will be used for this operation: this is the internal ID managed by BGG. All other
+    columns will just be ignored.
+
+Export a collection
+-------------------
+Will create a CSV file with all your games, as you will do with the UI.
+
+Example:::
+
+    $ bggcli -l mylogin -p mypassword collection-export mycollection.csv
+
+Notes:
+
+  * Only the ``objectid`` column will be used for this operation: this is the internal ID managed by BGG. All other
+    columns will just be ignored.
+
+
+Limitations
+===========
+
+* Only *Firefox* is supported. This tools relies on Selenium to control browser, and only Firefox is supported
+  natively by Selenium (i.e. without additional requirements). Support of additional browsers could be introduced,
+  but I'm not sure it's worth it.
+* Performance: Selenium+Firefox association is not the fastest way to automate operations, but it's
+  probably the best regarding stability (no Javascript emulation, Firefox does the work) and simplicity (no need to
+  install anything else), which is the most important in the current context.
+* Some fields related to the game's version are not exported by BGG: the ``barcode`` and the `language``. Note
+  although this is only for custom version you define yourself, which should be rare
+
+
+Ideas for future versions
+=========================
+
+Here are some ideas of additional tasks that could be implemented:
+
+* Generic import for collections, based on game names and not on the BGG internal identifier. A confirmation would be required for each ambiguous name to choose among matching games
+* Update/Delete for plays
+* Update/Delete for forum subscriptions
+
+Links
+=====
+
+* *BoardGameGeek*: http://www.boardgamegeek.com
+* *Officiel XML API 2*: https://www.boardgamegeek.com/wiki/page/BGG_XML_API2
+* *boardgamegeek - A Python API for boardgamegeek.com*: https://github.com/lcosmin/boardgamegeek
+
+Final note
+==========
+
+Does it really deserve such a development? Probably not, but my second goal was to discover the Python ecosystem!

bggcli/__init__.py

@@ -0,0 +1,20 @@
+import os
+
+if os.environ.get('CI') == 'true':
+    # Issues with Sauce Labs and HTTPS
+    BGG_BASE_URL = "http://www.boardgamegeek.com"
+else:
+    BGG_BASE_URL = "http://www.boardgamegeek.com"
+
+UI_ERROR_MSG = "Unexpected error while controlling the UI!\nEither the web pages have " \
+               "changed and bggcli must be updated, or the site is down for " \
+               "maintenance."
+
+BGG_SUPPORTED_FIELDS = ['objectname', 'objectid', 'rating', 'own',
+                        'fortrade', 'want', 'wanttobuy', 'wanttoplay', 'prevowned',
+                        'preordered', 'wishlist', 'wishlistpriority', 'wishlistcomment',
+                        'comment', 'conditiontext', 'haspartslist', 'wantpartslist',
+                        'publisherid', 'imageid', 'year', 'language', 'other', 'pricepaid',
+                        'pp_currency', 'currvalue', 'cv_currency', 'acquisitiondate',
+                        'acquiredfrom', 'quantity', 'privatecomment',
+                        '_versionid']

bggcli/commands/__init__.py

@@ -0,0 +1,12 @@
+import os
+
+from bggcli.util.logger import Logger
+
+
+def check_file(args):
+    file_path = args['<file>']
+
+    if os.path.isfile(file_path):
+        return file_path
+
+    Logger.error("File does not exist: %s" % file_path, sysexit=True)

bggcli/commands/collection_delete.py

@@ -0,0 +1,63 @@
+"""
+Delete games in your collection from a CSV file. BE CAREFUL, this action is irreversible!
+
+Usage: bggcli [-v] -l <login> -p <password>
+              [-c <name>=<value>]...
+              collection-delete [--force] <file>
+
+Options:
+    -v                              Activate verbose logging
+    -l, --login <login>             Your login on BGG
+    -p, --password <password>       Your password on BGG
+    --force                         Force deletion, without any confirmation
+    -c <name=value>                 To specify advanced options, see below
+
+Advanced options:
+    browser-keep=<true|false>       If you want to keep your web browser opened at the end of the
+                                    operation
+    browser-profile-dir=<dir>       Path or your browser profile if you want to use an existing
+
+Arguments:
+    <file> The CSV file with games to import
+"""
+import sys
+
+from bggcli.commands import check_file
+from bggcli.util.csvreader import CsvReader
+from bggcli.ui.gamepage import GamePage
+from bggcli.ui.loginpage import LoginPage
+from bggcli.util.logger import Logger
+from bggcli.util.webdriver import WebDriver
+
+
+def execute(args, options):
+    login = args['--login']
+
+    file_path = check_file(args)
+
+    csv_reader = CsvReader(file_path)
+    csv_reader.open()
+
+    game_count = csv_reader.rowCount
+
+    if not args['--force']:
+        sys.stdout.write(
+            "You are about to delete %s games in you collection (%s), "
+            "please enter the number of games displayed here to confirm you want to continue: "
+            % (game_count, login))
+
+        if raw_input() != game_count.__str__():
+            Logger.error('Operation canceled, number does not match (should be %s).' % game_count,
+                         sysexit=True)
+            return
+
+    Logger.info("Deleting games for '%s' account..." % login)
+
+    with WebDriver('collection-delete', args, options) as web_driver:
+        if not LoginPage(web_driver.driver).authenticate(login, args['--password']):
+            sys.exit(1)
+
+        Logger.info("Deleting %s games..." % game_count)
+        game_page = GamePage(web_driver.driver)
+        csv_reader.iterate(lambda row: game_page.delete(row))
+        Logger.info("Deletion has finished.")

bggcli/commands/collection_export.py

@@ -0,0 +1,138 @@
+"""
+Export a game collection as a CSV file.
+
+Usage: bggcli [-v] -l <login> -p <password>
+              [-c <name>=<value>]...
+              collection-export <file>
+
+Options:
+    -v                              Activate verbose logging
+    -l, --login <login>             Your login on BGG
+    -p, --password <password>       Your password on BGG
+    -c <name=value>                 To specify advanced options, see below
+
+Advanced options:
+    save-xml-file=<true|false>      To store the exported raw XML file in addition (will be
+                                    save aside the CSV file, with '.xml' extension
+    browser-keep=<true|false>       If you want to keep your web browser opened at the end of the
+                                    operation
+    browser-profile-dir=<dir>       Path or your browser profile if you want to use an existing
+
+Arguments:
+    <file> The CSV file to generate
+"""
+import csv
+import urllib2
+import time
+import sys
+import xml.etree.ElementTree as ET
+
+from bggcli import BGG_BASE_URL, BGG_SUPPORTED_FIELDS
+from bggcli.ui.loginpage import LoginPage
+from bggcli.util.logger import Logger
+from bggcli.util.webdriver import WebDriver
+from bggcli.util.xmltocsv import XmlToCsv
+
+BGG_SESSION_COOKIE_NAME = 'SessionID'
+EXPORT_QUERY_INTERVAL = 5
+ERROR_FILE_PATH = 'error.txt'
+
+
+def execute(args, options):
+    login = args['--login']
+    dest_path = args['<file>']
+
+    Logger.info("Exporting collection for '%s' account..." % login)
+
+    # 1. Authentication
+    with WebDriver('collection-export', args, options) as web_driver:
+        if not LoginPage(web_driver.driver).authenticate(login, args['--password']):
+            sys.exit(1)
+        auth_cookie = web_driver.driver.get_cookie(BGG_SESSION_COOKIE_NAME)['value']
+
+    # 2. Export
+    # Easier to rely on a client HTTP call rather than Selenium to download a file
+    # Just need to pass the session cookie to get the full export with private information
+
+    # Use XML2 API, see https://www.boardgamegeek.com/wiki/page/BGG_XML_API2#Collection
+    # Default CSV export doesn't provide version info!
+    url = '%s/xmlapi2/collection?username=%s&version=1&showprivate=1&stats=1' \
+          % (BGG_BASE_URL, login)
+    req = urllib2.Request(url, None, {'Cookie': '%s=%s' % (BGG_SESSION_COOKIE_NAME, auth_cookie)})
+
+    # Get a BadStatusLine error most of times without this delay!
+    # Related to Selenium, but in some conditions that I have not identified
+    time.sleep(8)
+    try:
+        Logger.info('Launching export...')
+        response = default_export(req)
+    except Exception as e:
+        Logger.error('Error while fetching export file!', e, sysexit=True)
+        return
+
+    # 3. Store XML file if requested
+    xml_file = options.get('save-xml-file')
+    if xml_file == 'true':
+        xml_file_path = write_xml_file(response, dest_path)
+        Logger.info("XML file save as %s" % xml_file_path)
+        source = open(xml_file_path, 'rU')
+    else:
+        source = response
+
+    # 4. Write CSV file
+    try:
+        write_csv(source, dest_path)
+    except Exception as e:
+        Logger.error('Error while writing export file in file system!', e, sysexit=True)
+        return
+    finally:
+        source.close()
+
+    # End
+    Logger.info("Collection has been exported as %s" % dest_path)
+
+
+def default_export(req):
+    response = urllib2.urlopen(req)
+
+    if response.code == 202:
+        Logger.info('Export is queued, will retry in %ss' % EXPORT_QUERY_INTERVAL)
+        time.sleep(EXPORT_QUERY_INTERVAL)
+        return default_export(req)
+
+    if response.code == 200:
+        return response
+
+    # Write response in a text file otherwise
+    try:
+        with open(ERROR_FILE_PATH, "wb") as error_file:
+            error_file.write(response.read())
+        Logger.error("Unexpected response, content has been written in %s" % ERROR_FILE_PATH)
+    except Exception as e:
+        raise Exception('Unexpected HTTP response for export request, and cannot write '
+                        'response content in %s: %s' % (ERROR_FILE_PATH, e))
+    raise Exception('Unexpected HTTP response for export request, response content written in '
+                    '%s' % ERROR_FILE_PATH)
+
+
+def write_xml_file(response, csv_dest_path):
+    dest_path = '.'.join(csv_dest_path.split('.')[:-1]) + '.xml'
+    with open(dest_path, "wb") as dest_file:
+        dest_file.write(response.read())
+
+    return dest_path
+
+
+def write_csv(source, dest_path):
+    with open(dest_path, "wb") as dest_file:
+        csv_writer = csv.DictWriter(dest_file, fieldnames=BGG_SUPPORTED_FIELDS,
+                                    quoting=csv.QUOTE_ALL)
+        # csv_writer.writeheader() use quotes
+        dest_file.write('%s\n' % ','.join(BGG_SUPPORTED_FIELDS))
+
+        for event, elem in ET.iterparse(source, events=['end']):
+            if event == 'end':
+                if elem.tag == 'item' and elem.attrib.get('subtype') == 'boardgame':
+                    row = XmlToCsv.convert_item(elem)
+                    csv_writer.writerow(row)
+