Home

This is a work in progress, I’m documenting my installation process as I go.

Installing on Ubuntu

Install PostgreSQL with Tsearch2

sudo apt-get install postgresql postgresql-client postgresql-contrib libpq-dev

Create database

If you just try to do
createdb opencongress_dev

you’ll get something like this:
FATAL: ident authentication failed

This is explained and solved here: http://www.depesz.com/index.php/2007/10/04/ident/
Long story short: edit /etc/postgresql/8.3/main/pg_hba.conf and change your local lines to have trust instead of ident sameuser as the authentication method. Read the article for security implications.

After that you need to restart postgres:
sudo /etc/init.d/postgresql-8.3 restart

By default, postgres comes only with the admin user, called postgres, so we’ll use it from here on. To create the database:
createdb -U postgres opencongress_dev

Now add the tsearch2 stuff. The file that’s in the opencongress repo (db/tsearch2.sql) blows up with all sorts of errors for me. This is what worked:

psql -d opencongress_dev -U postgres < /usr/share/postgresql/8.3/contrib/tsearch2.sql

Repeat the same steps to create the opencongress_test database.

Install necessary gems:
sudo gem install ferret bluecloth simple-rss postgres-pr

Set up your database.yml file to use adapter: postgresql and host: /var/run/postgresql.

The environment config tries to read config/api_keys.yml which isn’t present by default. Just copy the dummy over from config/api_keys.example.yml for now. For migrating that’s all you need. You can fill out the api keys later.

Before you run your migrations, you’ll also need to set up Solr, the search server. Thankfully, it already comes bundled with the acts_as_solr plugin, which is in our vendor/plugins directory.

Create the logs and tmp directories:

mkdir vendor/plugins/acts_as_solr/solr/logs
mkdir vendor/plugins/acts_as_solr/solr/tmp

And fire it up:

rake solr:start

It should now be running on port 8982.

And then:
rake db:migrate

Ok, not so fast:

    ==  PolymorphicPageviews: migrating ===========================================
    -- add_index(:page_views, [:viewable_id, :viewable_type, :created_at])         
    rake aborted!                                                                  
    An error has occurred, this and all later migrations canceled:                 

    RuntimeError: ERROR     C42703  Mcolumn "viewable_id" does not exist    Findexcmds.c    L779    RComputeIndexAttrs: CREATE  INDEX "index_page_views_on_viewable_id_and_viewable_type_and_created_at" ON "page_views" ("viewable_id", "viewable_type", "created_at")

Open db/migrate/2009010145551_polymorphic_pageviews.rb, uncomment all the code in self.up and remove the line add_index :page_views, :ip_address (this index was already created at migration #063). (or just use this patch: http://github.com/obvio171/opencongress/commit/8305fabb1e4160c0df7ea94b0bf6f1679794f6f9.diff )

You’ll also need RedCloth installed for the EditableBlogroll migration.

sudo gem install RedCloth

And now, what we’ve all been waiting for:

rake db:migrate

That one should be a winner! All migrations run! w00t!

Now let’s import some data.

Importing data

(From the README)

Next, you will have to get the data from govtrack and fill in your database with the parsed govtrack data. Right now, you do have to use the /data/… absolute path (relative to the root of the filesystem). You’ll probably need sudo for this.

  mkdir -p /data/govtrack/111
  rsync -az govtrack.us::govtrackdata/us/111/bills /data/govtrack/111
  rsync -az govtrack.us::govtrackdata/us/111/repstats /data/govtrack/111
  rsync -az govtrack.us::govtrackdata/us/111/bills.index.xml /data/govtrack/111
  rsync -az govtrack.us::govtrackdata/us/111/committeeschedule.xml /data/govtrack/111
  rsync -az govtrack.us::govtrackdata/us/111/rolls /data/govtrack/111

Now, you will have to actually parse the data. You’ll need the htree library for that:

cd /tmp
cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/ruby co htree
cd htree
sudo ruby install.rb

Bizarre, yes. Anyone wanna package that as a gem?

You also need to fix the data paths in the data import scripts. This patch will take care of that for you:
http://github.com/obvio171/opencongress/commit/de85309520f9663a88022a5712062838edd5650b

So now go back to your application root directory and run:
Note: this will take a REALLY long time. Go get some coffee.

  ruby bin/govtrack_parse_people.rb

The following doesn’t work, because govtrack_parse_bills.rb expects a different format from the one in the file it tries to read (bills.index.xml). It seems to expect the format of a bill file (like this one). The other parsers also break.

  ruby bin/govtrack_parse_bills.rb 
  ruby bin/govtrack_parse_committees.rb 
  ruby bin/govtrack_parse_committee_schedules.rb 
  ruby parse_individual_bills.rb