couchdb-blazegraph-sync

Keep a CouchDB database and a BlazeGraph namespace in sync.

⚠️ Do not use for loading a full database, as that would be way less performant than just sending a dump to BlazeGraph

Summary

• Install
• Add config
• Set last seq
• Start
• Add to systemd
• License

Install

git clone https://github.com/maxlath/couchdb-blazegraph-sync
cd couchdb-blazegraph-sync
npm install --production

Add config

cp ./configs/example.js ./configs/my_db.js

Then customize ./configs/my_db to your needs

Set last seq

The sync will restart from the last known seq, which is persisted in a file per database: ./data/my_db.last_seq

This tool isn't designed to load a full database, rather to keep up with the changes, so first make sure you to import a dump directly

# Use your serializer to generate a dump of your CouchDB database, this is out of the scope of this tool
curl http://my-blazegraph:8080/bigdata/namespace/kb/dataloader -H 'Content-Type: application/x-turtle' -d@./my_db.ttl
# Set the current last seq
curl "http://username:password@localhost:5984/my_db" | jq '.update_seq' > ./data/my_db.last_seq
# Or set an arbitrary point to start from
printf '1079700' > ./data/my_db.last_seq

Start

npm start

Add to systemd

# From the project root, assumes that you need sudo rights
npm run add-to-systemd
# Start the process
sudo systemctl start couchdb-blazegraph-sync
# Start following the logs
journalctl -fan 100 -u couchdb-blazegraph-sync

License

AGPL-3.0