It's Not You, It's Us

I woke up this morning to an avalanche of emails from people assuring me that they have been creating great content and engaging with their network and that there was no way their Klout scores should have dropped so sharply. Something was definitely wrong and it quickly became clear that all of our scores had fallen with most people landing at 1.

Every day between 4-5am PST we load updated scores for over 75 million social media profiles. Due to the growing dataset size, scoring took longer than expected and our pipeline started to load today’s scores prematurely. Since that data was in an indeterminate state for the last step of our pipeline, what generally would cause a red flag, instead, defaulted everyone to a score of 1. By 6:30am PST the team was working on the issue and by 8:00am PST scores were reloaded and returned to normal.

While our API and site have averaged more than 98% uptime for this year, this issue caught us off-guard and we apologize for any inconvenience this may have caused our users or API partners. Today we will be re-instrumenting our data load process to make sure we are immediately notified and that the system rolls back any drastic score changes. We will also be formalizing our communication flow to make sure that messages immediately are posted on the site and that our API partners are notified of these issues.

We apologize for any inconvenience caused by our scoring issue this morning and appreciate your patience as we grow.

Advertisements


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s