Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve integration documentation #57

Open
adenhertog opened this issue Mar 2, 2020 · 5 comments
Open

Improve integration documentation #57

adenhertog opened this issue Mar 2, 2020 · 5 comments
Labels
enhancement New feature or request

Comments

@adenhertog
Copy link
Contributor

#56 indicates that the documentation is lacking around how to integrate all the components into a solution, especially around how to create a message-driven set of distributed services.

NServiceBus has a set of examples/tutorials that demonstrate some messaging core concepts and how to implement them with that library, which could be a useful reference to base this documentation off.

@adenhertog adenhertog added the enhancement New feature or request label Mar 2, 2020
@Zehelein
Copy link
Contributor

Zehelein commented Mar 3, 2020

Sorry to add this here: is there some forum/discord/... to ask questions? Like for RabbitMQ transport, I see that a dead-letter queue is created and associated. But I could not figure out what would trigger a message to be moved here like some maximum number of retries or anything.

For a test I adjusted the rabbitmq-transport.ts with the following code to send a message to the dead letter queue after the first retry which would do that:

  async returnMessage(
    message: TransportMessage<RabbitMqMessage>,
  ): Promise<void> {
    if (message.raw.fields.redelivered) {
      // Message retries exhausted, send to DLQ
      this.logger.info(
        'Message retry limit exceeded, sending to dead letter queue',
        { rawMessage: message.raw },
      );
      this.channel.nack(message.raw, undefined, false);
    } else {
      this.logger.debug('Returning message', { rawMessage: message.raw });
      this.channel.nack(message.raw);
    }
  }

But are there any other configurations on the RabbitMQ setup level maybe to configure something?

@adenhertog
Copy link
Contributor Author

Hey @Zehelein, I tried this today and confirmed there is no DLQ routing using RabbitMQ. I've just added this in now and have released it in @node-ts/bus-rabbitmq:0.5.0 so if you upgrade your local package you should see this failing dead letters after the default 10 attempts.

The number of attempts is configurable as per RabbitMqTransportConfiguration.maxRetries (https://node-ts.github.io/bus/packages/bus-rabbitmq/#configuration-options), and I've set up a quick demo of it working at node-ts/bus-starter#18 which is based of your other example.

Let me know how it goes, and thanks for letting me know!

@Zehelein
Copy link
Contributor

Zehelein commented Mar 3, 2020

Hi, thanks for looking so fast into it and providing a fix! But I am not sure - I fear this fix is not working. I also checked this message.raw.fields.deliveryTag but when I debugged/logged this tag then it would continue to increment. Meaning if I started with one message it was "1", and then the next failed had "2". It was not counting per message but per channel according to https://www.rabbitmq.com/confirms.html

This article seems to provide a few solutions - the ones with a bit of a delay in between would likely be nicer for transient errors.

@adenhertog
Copy link
Contributor Author

I think you're right, that delivery tag is what you describe. I had more of a read into how rabbit/amqp deals with retries and they all seem to defer this to the client (ie: they don't track retries themselves). There has been ongoing work in rabbit to support this, but there's only been a small amount of progress in the last couple of years and looks like it's currently just a feature for quorum queues.

Adding this to rabbitmq-transport.ts means that a header should be incremented ("x-redelivery-count") on each receive, but this isn't possible without ack the original message and publish a new message with the amendments. It feels quite hacky, and sends the message to the back of the queue (this can be good or bad depending on the use case).

I'm working on this implementation on https://github.com/node-ts/bus/compare/rabbitmq-message-receives but this seems to break the DLQ routing.

@Zehelein
Copy link
Contributor

Zehelein commented Mar 4, 2020

Sorry I totally missed to provide the link to the article I was referring to:
https://engineering.nanit.com/rabbitmq-retries-the-full-story-ca4cc6c5b493

Maybe option 2 or 3 would be doable? Option 4 would depend on a plugin - not sure if that would be available in every setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants