expose the Twisted-testing internals of treq.testing for testing things that are concurrent with HTTP requests

[glyph]

Twisted is great because you can do lots of stuff concurrently. MemoryReactor is great because you can test the completion of concurrent operations in memory. StubTreq is great because it lets you do this in a super high-level idiomatic way in tests.

However, these do not always play nice together. Specifically, if I have an HTTP request that kicks off some other network I/O on the back end, which might have timeouts, and I want to wrap it all together nicely in the same MemoryReactorClock:

• I can't pass one in; it's not an argument
• I can't get one out; it's not a public attribute (in fact it's like, hyper private; stub_treq.get.__wrapped__.__self__._agent._memoryReactor is how you apparently have to scrape it out of the innards)

Let's expose this in a nice way so that we have a supported & documented public way to test interactions with other subsystems within Twisted. If a web request updates a conch terminal I want to be able to test it.

[twm]

By all means, let's do this.

For testing which requires control of the reactor I have been using HTTPClient directly, passing it a RequestTraversalAgent directly. It's fortunate that so many of the internals of treq.testing are public in this way. The one caveat is that treq.testing._SynchronousProducer isn't, so you must either:

• Reimplement _SynchronousProducer; or
• Advance the clock enough for the cooperator inside FileBodyProducer to send any request body, then flush().

When doing this sort of no-I/O testing the cooperator API seems to come up a lot. It seems to creep into places where it is difficult to override, like FileBodyProducer or twisted.internet._producer_helpers._PullToPush. Maybe it should be attached to the reactor directly, instead of coupled through the global module namespace?

---

TBH I am not sure why StubTreq is a separate API, vs. documenting use of HTTPClient for testing. I suppose StubTreq provides a few extra members — text(), json(), et. al., but those are also methods on an augmented response. It feels like we have a lot of duplicated API surface.

It is also really weird to me that the treq module functions and particularly the HTTPClient functions accept an agent parameter. I feel that HTTPClient should encapsulate an agent (and implicitly the reactor the agent is associated with), rather than being a thing which... wraps up an agent you give it, except when you don't give it one it uses some default?

---

See also #226.