What is the preferred way for bulk sending a lot of emails through Sendgrid API?

[ericpopivker]

I am currently using the following approach:

var tasks = new List<Task>();
for (int i=0; i <100; i++)
{
    var task = SendEmailUsingSendgridApiAsync(message);
    tasks.Add(task);
}

Task.WaitAll(tasks.ToArrayList());

This approach takes 4s for 100 emails vs 20s if I just send emails one by one. So about 5X faster.
(Single email takes 200ms.)

Is there a recommended approach to send 1000s of emails quickly using SendGrid .NET API?
For 100K emails, even using async approach it still will take about an hour.

Also are there plans to just add REST endpoint on Sendgrid API that would allow sending batches of emails (for ex 100 at a time) in a single request. This way you guys can handle it on Sendgrid side.

[Jericho]

@ericpopivker I have two suggestions for you:

First of all, you can batch up to 1,000 recipients in a single API call which should be much more efficient than sending emails one by one. Check the "Limitations" section in this document. Keep in mind that you can personalize the content for each recipient, if desired. The SendGrid C# client has the CreateMultipleEmailToMultipleRecipients method to achieve this and the StrongGrid client has a similar method called SendAsync which gives you even more personalization flexibility (disclaimer: I'm the author of the StrongGrid library).

But I'm afraid that batching recipients will not be good enough when you need to send 100k emails. My recommendation would be to utilize SendGrid's marketing campaigns feature.

A typical flow would be:

• Upload your 100k contacts to SendGrid. You can also include custom fields with each contact which can be used for personalizing the content (simple example would be: Dear %first_name%).
• you can either add the desired contacts to a list or you can define a query (called "segment") to match the desired contacts. For example: "all contacts where country=Canada".
• you create the campaign by specifying your HTML and text content, what IP pool you want to use (assuming you purchased IP addresses from SendGrid), what lists(s) and/or segment(s) you want to use, etc.
• finally, you either send the campaign immediately or schedule it to be sent at a later date/time. The API call for sending/scheduling the campaign completes immediately but behind the scene SendGrid will take care of sending your 100k emails which may take a while.

If you send a recurring email (say a monthly newsletter for example), there will be no need to re-upload your contacts. You can simply reuse the list and/or segment whenever you create subsequent campaigns. Also, one very helpful feature is that SendGrid will help you manage unsubscribes meaning that a given recipient will not receive your newsletter in the future (even if they are on your list/segment) if they previously clicked on the "Unsubscribe" link that you add at the bottom of your campaign.

StrongGrid makes it particularly easy to accomplish all these tasks:

• Importing contacts
• Creating a list
• Creating a segment
• Creating a campaign
• Scheduling your campaign
• etc.

One final thought: if you are going to be sending such a large number of emails, especially if you are sending regularly (daily, weekly, monthly), it's a good idea to invest in your own IP addresses (you purchase them from SendGrid) instead of relying on the pool of shared addresses. This will allow you to build your own reputation as an email marketer, avoid your reputation being influenced by what other senders in SendGrid's pool are doing and, ultimately, reduce the likelihood of your emails ending up in your recipients "junk" folder. Please note that I said "reduce the likelihood", as there is no way to absolutely guarantee "junk-free" delivery. If you decide to invest in your own IP addresses, you will want to gradually increase the number of emails you send on your own IPs as opposed to sending all 100k emails at once on the new IPs you purchased because large spikes in volume on new IPs hurt your reputation and therefore increase the likelihood of your emails being flagged as spam. Fortunately, the StrongGrid library has a convenient "Warmup Engine" to help you with that (see the 'Warmup Engine' section in the readme for more details).

Let me know if this helps.

[thinkingserious]

Wow, thanks for the wonderful explanation @Jericho!

@ericpopivker,

I think @Jericho's advice is solid. Please let us know if you need additional help.

[thinkingserious]

We should add an edited version of Jericho's comment (perhaps linking to this issue) to the USE_CASES.md section.

[jer-tx]

As a follow up to the original question, what if we need to sent to multiple thousands of people a data-driven email? Like a usage summary? We send weekly summary emails detailing what the user did for the past week (we're a fitness platform) so this includes very user-specific data like workout count, steps count, calories etc. Is there a way to do this with sendgrid to several thousand or more at once?

I'm not sure a marketing campaign works here given its data-driven nature, unless theres a way to store custom data like that where a campaign email template can reference.

For example, In ExactTarget you have "data extensions" to save custom data about a user, which can be updated at any time in json via an api. Then, when we want to send that weekly summary, we kick off a "triggered send" per user (thousands), and that reads from the data extension all of that custom data and sends the email.

Vaguely looking at the documentation I think sendgrid can do this on a per-user basis but, how would one do this for 100k users?

[thinkingserious]

Hello @jer-tx,

I suggest you reach out to our support team for the latest best practices.

With Best Regards,

Elmer

[sophiakaile]

thanks for this API it is really helpful for me to bulk email software

[startechsoftwares]

I have a similar issue, i want to shoot out approx 1k mails at one time, however the drawback with the current sendgrid's v3 api is we cannot set the Reply-To emails address for individual emails we send (through personlization). Hence i wanted to checkout the SMTP Api, but my problem is smtp takes a huge time while sending emails. I am currently sending mails one-by-one in a for loop. (even tried Parallel ForEach) however for each mail it need to create an smtp object, can i send emails using one smtp connection. Could you please show me some demo code in c# to send bulk mails using SMTP Api, i can't find the same in Sendgrid documentations.

[childish-sambino]

@startechsoftwares Recommend reaching out to our support team for the latest best practices.

[BrianJRobinson]

Are there any further updates around sending bulk emails. My use case is I need to send a transactional email where each email has different custom arguments and each email has a PDF attachment that contains that specific users details of their account. As far as I can tell, when sending bulk, the attachments can't be different. I tried reaching out to the support team, and after long delays between emails getting bounced around (2 months worth) I have been sent here. Does anyone have a code snippet that shows how this can be done, or is it simply just not possible?

[JamalSBS]

But I'm afraid that batching recipients will not be good enough when you need to send 100k emails. My recommendation

So what happens when you try to send 100,000?

99,999 works, but 100,000 won't?

You'd think they'd put this in the documentation on this page: https://docs.sendgrid.com/api-reference/mail-send/limitations

Also, what is "the preferred way to TEST bulk email sending a lot of emails through Sendgrid API"? Specifically the CreateMultipleEmailsToMultipleRecipients method.

[ahmadakra]

We spent 2 months developing the email engine in our business software. Its applications include generating and dispatching large numbers of payslips and invoices in email attachments each month.
That feature was going to rely on SendGrid, for bulk sending of emails using personalizations to batch many emails in one API request... or so we thought.
Only at the end near the deadline did we discover a tiny little detail, said personalizations do not support attachments! It was an oversight not to verify this from the start but it didn't occur to us in a million years that an email service provider with the size and popularity of SendGrid would not have a way for doing this. Imagine wanting to send 200,000 invoices to that many customers at the end of the month and having to do it in a loop of 1 API request per invoice.

[BrianJRobinson]

We spent 2 months developing the email engine in our business software. Its applications include generating and dispatching large numbers of payslips and invoices in email attachments each month. That feature was going to rely on SendGrid, for bulk sending of emails using personalizations to batch many emails in one API request... or so we thought. Only at the end near the deadline did we discover a tiny little detail, said personalizations do not support attachments! It was an oversight not to verify this from the start but it didn't occur to us in a million years that an email service provider with the size and popularity of SendGrid would not have a way for doing this. Imagine wanting to send 200,000 invoices to that many customers at the end of the month and having to do it in a loop of 1 API request per invoice.

You can get a limited amount of success by batching it into groups and sending each email on a thread(Task). I had to try a few different iterations of how many per batch as I found I needed to do a small wait in between each batch otherwise the network throughput would struggle.

[ahmadakra]

We spent 2 months developing the email engine in our business software. Its applications include generating and dispatching large numbers of payslips and invoices in email attachments each month. That feature was going to rely on SendGrid, for bulk sending of emails using personalizations to batch many emails in one API request... or so we thought. Only at the end near the deadline did we discover a tiny little detail, said personalizations do not support attachments! It was an oversight not to verify this from the start but it didn't occur to us in a million years that an email service provider with the size and popularity of SendGrid would not have a way for doing this. Imagine wanting to send 200,000 invoices to that many customers at the end of the month and having to do it in a loop of 1 API request per invoice.

You can get a limited amount of success by batching it into groups and sending each email on a thread(Task). I had to try a few different iterations of how many per batch as I found I needed to do a small wait in between each batch otherwise the network throughput would struggle.

Thank you. This is indeed the only way we could think of to save our face. Now we're batching all emails into chunks of 200 each and then for each chunk we're doing a Task.WhenAll to dispatch the 200 emails in parallel, it will take significantly longer to dispatch all the chunks, and it's easy to overwhelm the network or run into SendGrid API request limits, but we have no other option right now