Web service to sync server's repo with github's repo

[jar398]

Inspired by the Unix 'sync' system call, I think it would be nice if there were a method that atomically
(a) flushed out object (study, collection, etc.) state to server's disk
(b) did a 'git pull' (or fetch+merge) to bring changes made on github back to server's repos + mirrors
The return value would tell you whether there were merge conflicts.

This can be unprivileged - it is safe to do and fairly low cost.

Not sure what to do if there are merge conflicts. Then someone who can log in to the server needs to do a synchronous shutdown (or lock acquire), manually fix the merge, and restart (or unlock).

Part of this discussion on gitter

[mtholder]

There really isn't any state to be stored to disk (that I am aware of). We'd just need to recreate the in memory indexes after the pull.
This should not be too hard to implement. But given that redeploying should be a reasonable workaround for this behavior (and we rarely need it), this does not seem like a high priority to me.

[jar398]

No, not high priority. Thanks for confirming that a deploy does the trick. Does an apache restart (less drastic and less risky) also do the trick? I think the technical issue is what happens if we're in the middle of a transaction - we might have a situation where a commit succeeds but the method call doesn't return. Git takes care of integrity for us, but the client might be confused. But this is only a remote possibility at present.

[mtholder]

I don't see how an apache restart would do it. The git_refresh part of the full deploy seems like it would be crucial.

There is locking of the phylesystem-1 repo during write operations. So I don't think we'd interrupt a transaction.The call could come between the lag between committing to the working repo and the push from the mirror to GH. Presumably this web service would be written to lock and attempt a fast-forward merge only, so that if the master on GH was not up-to-date the call would fail. That would probably mean that the lagging push from the mirror would also fail (because presumably this web service would be called when someone had pushed new content to the master branch of GitHub via some tool other than the API). It seems like we'd need a human to sort out such cases, and the phylesystem-api would be blocked from sharing data to GitHub until that happened. A side effect of that would be the OTI index getting stale.