-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fail-over to a second server if the first is unreachable #51
Comments
I think this is a great idea. In the case of a primary going to Seq and a secondary going to a Seq forwarder, it may be useful to define the benefit. Opposing view may say that, “I have to setup the Seq forwarder on the local host either way, why try to bypass it?”
Disclaimer: I am not currently using Seq in any deployed scenarios, so may not be the best one to take an opinion from. My experience is with prior experimentation with POC’s.
* I believe a configurable retry count would be useful, but not required.
* I think a primary and secondary is best for keeping configuration simple.
* Can’t comment.
* I think having a property indicating that fail-over has occurred would be valuable. I would add the property wherever it’s cheapest to implement, whether client or forwarder I do not know. I seem to recall lower levels of abstraction possibly being more costly to manipulate the event, but my recollection could be hazy.
|
Thanks for the thoughts Ben! For completeness we should probably also consider that adding a generic Log.Logger = new LoggerConfiguration()
.WriteTo.Seq(...,
failoverTo: w => w.Seq(..., ))
.CreateLogger(); The API is obviously just one possible sketch. Batching makes this pretty tricky, but not impossible. Adding @serilog/reviewers-core in case this sounds worthwhile to explore further. (It would be nice to start extracting more of the common concerns from the Seq/Splunk/Elasticsearch sinks, which all tend to evolve these capabilities in parallel....) |
Just some more thoughts ... Maybe a more "general" way would be to wrap it all , something like this : Log.Logger = new LoggerConfiguration()
.WriteTo.Try(w => w.Seq(primary, ), failoverOptions)
.ThenTry(w => w.Seq(secondary, ), failoverOptions)
.FallbackTo(w => w.File(secondary, ))
.CreateLogger(); The idea being that the And then there's the question of |
Would be nice if this can be generalized to (also) support durable mode. So to support sinks that now have this as part of their code base, but actually have similar code (and issues). Log.Logger = new LoggerConfiguration()
.WriteTo.Durable(w => w.Seq(primary, ), durableOptions)
.ThenTry(w => w.Seq(secondary, ), failoverOptions)
.FallbackTo(w => w.File(secondary, ))
.CreateLogger(); The durable option wraps the actual sink and, based on the options, preserves the logevents and tries to push them to the inner sink. The sink, however, needs to indicate that it could not deliver the items. Maybe using a specific exception, although that might be a bit heavy, or by inheriting a special BufferedPeriodicSink? Any thoughts? The implementation in the ES sink is tricky. Duplication of code, different ways of sending the data etc. Can fix it in the sink itself, but a more common pattern might be nicer? |
@nblumhardt also in reference to this issue (serilog/serilog#668) and what I m trying to fix in the durable sink for the ES sink. Although the functionality is great, the implementation differs from the real source. A more generic solution would be nice. Curious about your thoughts. |
Another might be introducing |
Interesting idea. As long as the Durable functionality is capable of temporary buffering LogEvents and writing to/reading from file, they can be passed to the next Emit function. That is also one of the issues of the ES buffer; the input for ES is JSON as that is how it is serialized in the buffer when written to disk, but it misses the context to build the right index name and reuse the real sink logic as it does not pass in the LogEvents again. By providing a durable wrapper you can potentially make all sinks durable without code duplication. |
Any movement on this? Just curious? |
Hi @kevingates!
There's no official A PoC that demonstrated the API and mechanics (either as a gist or example nupkg) would help zoom in on any potential issues, illustrate the API possibilities, and might be enough for us to open up some minimal extension points to make this all possible (either via Serilog.Sinks.PeriodicBatching, or with some core APIs). HTH! |
Hi all; we're working on supporting these kinds of scenarios server-side using clustering, and on the client if redundancy is needed, two separate servers can be targeted. Closing this as stale, but thanks for all the input! 👋 |
If the attempt to send events to the primary Seq server fails, the sink could be configured to try a secondary Seq server, or more conveniently, a Seq Forwarder that will persist the events and transmit them back to the primary server once it's available.
For example:
Some questions:
The text was updated successfully, but these errors were encountered: