The MCN Member Email Incident

Developing a website is always an adventure, especially when building improvements onto a live site like mcn.edu. This blog post is to share some of the behind the scenes of an incident that occurred this morning, including why it happened because we want to be transparent about all of the development activities around the website improvements. Hopefully it provides some useful information beyond an explanation. First, we want to update you and reassure you that everything is back on track.

Before getting into the details, to our members: 

We are very sorry for the trouble this email incident may have caused. The process that triggered these emails was stopped, but unfortunately at the speed of processing, it still let through far too many emails. For this, our most sincere apologies.

For the tech curious, some more detail on why this happened, because we believe in transparency. A big part of website functionality is search, we want to be sure you can find your posts, replies and everything else on mcn.edu—but especially to be able to effectively search your past posts. The SIG message boards were indexed in a server job to ensure findability of information, this effectively updated them in the database which triggered thousands of emails. 

Our dev team worked as quickly as possible to stop the flow of email, but unfortunately with how fast things move, a TON got through before we got it turned off. Hopefully not too many of our friends in IT can empathize with the panic we all woke up to this morning, nobody likes to wake up to this many emails!

Next, some of the improvements in the works that will NOT create an email avalanche:

  • New membership subscription options (launched!)
  • Mentorship application inside the portal (launched!)
  • Continued improvements to Search
  • Notifications for replies not just initial posts to keep conversations alive
  • Weekly digest of SIG notifications
  • SIG Resources pages (for chair-curated selections of content for your groups)
  • Resources repository (including past conferences)
  • Community partners directory

Again, we are very sorry for flooding your inboxes, but rest assured that it is in the service of larger improvements of the membership portal and it will not happen again. 

Thank you for your patience and continued support.

Eric Longo, Executive Director