Red Bull Can You Make It? 2020

Social Scoring Model

Last Updated: 2020-02-06

Overview

Below you'll find examples and a description of the scoring model we've prepared.

The three types of content as we'll classify them are:

1. IMAGE POST
   • Typically square
   • Available metrics via IG API: reach, impressions, engagement, saved
2. VIDEO POST
   • Typically square
   • Available metrics via IG API: reach, impressions, engagement, saved, video_views
3. STORY
   • 9:16 ratio - can be in video or image format
   • Available metrics via IG API: reach, impressions, replies, exits, taps_forward, taps_back

For a basic definition of each metric listed above, here's a good reference.

Key Principle:
Score content based on it's measured "quality", not necessarily reach or impressions.

For this scoring model, keep these notes in mind:

• Challenge: Quality content should be scored highest.
• As this is a complex scoring model, we don't ever intend for these formulas to be exposed to teams. Instead, when communicating to teams, explain that we use a formula for identifying quality content but can't give them specifics that would compromise the integrity of scoring by allowing them to focus on and exploit the metrics that we rely on most.
• Again, as this a fairly complex scoring model, there is a great deal of nuance and thought put into the measurements. If you see scenarios where you believe not enough/too many points are awarded, ensure you visit the formula and do a gut-check by remembering we want to focus on the best, most engaging content - not necessarily just content that was seen by many people. (Often this isn't the same - users with high follower counts can have low quality content seen by many.)
• Instead of focusing on raw quanity of metrics (ie- engagement), look at these metrics in relation to how much reach or how many impressions, saves, etc. the content had. This will give us a more accurate way to identify quality content when comparing users with 10 followers or 1000 followers.
• Users with low followers may have very high content rating percentages, but that doesn't mean their content is good. (A post with a reach of 3 people that has 3 "saves" still might not be good.) We give extra boosts when content performs at a high percentage but ALSO is seen by many users.
• Keep in mind, the highest content points (up to 100) will rarely be achieved in this "complex" model - if ever. This is intentional and we expect most user content to fall in single digit points (~0-9) or in many cases mid double digits (~20-60).
• We cap scores at a maximum of 100 points. We don't want any single piece of fantasic content "breaking" the system and carrying a team to victory all by itself. So if you see a scenario where content with 500 impressions is doing as well as content with 100000 impressions, you're likely seeing the "cap" effect.
• Remember as the example data below is viewed, this model focuses on "rating" content by how much positive response, repeat views, saves, etc. it receives. Not necessarily the maximum number of any single metric.
• The formula for this approach is much more complex, but we believe this is the best way to attempt to identify the best content based on the metrics we have available to us.
• Lasty, there are MANY scenarios in the formula where we cap certain metrics to ensure we aren't giving too much value to any particular metric. So you may see that increasing one metric appears to not have any affect on scoring with a similar set of metrics. Again, this is intention to prevent any "snowball" effect.
• There are hundreds of millions of scenarios to simulate and we've run through them, but it's too much data to list completely in a table here. If you'd like a deeper dump of test data, let us know and we can export it for you.
• When in doubt, reference the formula below to see how the score is calculated.

const MAX_POINTS_PER_CONTENT = 100;
/*
Determine a multiplier or "weight" (used later) to apply based on reach 
(highly dependent on followers)
This is important because a user with 100% enagement but only 1 follower
should not get similar content rating as a user with 50% engagement and
10000 followers.
However, to avoid skewing too heavily in favor or those who have high
follower counts, we "dampen" this multiplier (divide by 500) but ensure
a minimum of "1" here.
*/
$reachMultiplier = max(0.001, min(1.0, (min(500, $metrics['reach']) / 500.0)));
/*
Determine a similar multiplier or "weight" for impressions (this will be
used later for giving SMALL bonuses to content with large impressions)
*/
$impressionsMultiplier = 1.0 + max(0.0, min(0.5, (min(10000.0, $metrics['impressions']) / 10000.0) * 0.5));
/*
Determine the percentage of impressions relative to how many unique user
views, capped at 500%, minimum 0% (higher percentage indicates users
keep coming back to content)
*/
$percentImpressions = max(0.0, min(5.0, floatval($metrics['impressions']) / max(1,$metrics['reach'])));
/*
Determine the percentage of engagement (likes, comments, etc.)
relative to how many unique user views, capped at 300%, minimum 0%
*/
$percentEnagement = max(0.0, min(3.0, floatval($metrics['engagement'] / max(1,$metrics['reach']))));
/*
Determine the percentage of unique user saves relative to how many
unique user views, capped at 100%, minimum 0%
*/
$percentSaved = max(0.0, min(1.0, floatval($metrics['saved'] / max(1,$metrics['reach']))));
/*
Apply our weighted and dampened "reach multiplier" for impression %
- normalizing our 500% maximum to max of 100%
*/
$reachWeightedImpressions = ($percentImpressions / 5.0) * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for engagement %
- normalizing our 300% maximum to max of 100%
*/
$reachWeightedEnagement = ($percentEnagement / 3.0) * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for unique user saves %
*/
$reachWeightedSaved = $percentSaved * $reachMultiplier;
/*
Impressions account for 10% of this content's score (this content is
seen multiple times)
*/
$scoredImpressions = $reachWeightedImpressions * 0.10;
/*
Enagement account for 50% of this content's score (likes+comments
are easy to do but still should be factored somewhat)
*/
$scoredEnagement = $reachWeightedEnagement * 0.50;
/*
Saves account for 45% of this content's score (if someone saved
this content-they REALLY liked it)
*/
$scoredSaved = $reachWeightedSaved * 0.45;
/*
NOTE - At this point, total % of all metrics can be higher than 1.0 (100%)
- this is intentional
*/
/*
Set score by combining separate scoring factors and ensure score
never tops 1.0
*/
$score = min(1.0, ($scoredImpressions + $scoredEnagement + $scoredSaved) * $impressionsMultiplier);
/*
Scale content score relative to maximum value
*/
$points = (int)max(0, ceil(self::MAX_POINTS_PER_CONTENT * $score));
/*
if impressions or reach are 0, points are 0
*/
if($metrics['reach'] <= 0 || $metrics['impressions'] <= 0){
	$points = 0;
}					

const MAX_POINTS_PER_CONTENT = 100;
/*
Determine a multiplier or "weight" (used later) to apply based on reach 
(highly dependent on followers)
This is important because a user with 100% enagement but only 1 follower
should not get similar content rating as a user with 50% engagement and
10000 followers.
However, to avoid skewing too heavily in favor or those who have high
follower counts, we "dampen" this multiplier (divide by 500) but ensure
a minimum of "1" here.
*/
$reachMultiplier = max(0.001, min(1.0, (min(500, $metrics['reach']) / 500.0)));
/*
Determine a similar multiplier or "weight" for impressions (this will be
used later for giving SMALL bonuses to content with large impressions)
*/
$impressionsMultiplier = 1.0 + max(0.0, min(0.5, (min(10000.0, $metrics['impressions']) / 10000.0) * 0.5));
/*
Determine the percentage of impressions relative to how many unique
user views, capped at 500%, minimum 0% (higher percentage indicates users keep coming back to content)
*/
$percentImpressions = max(0.0, min(5.0, floatval($metrics['impressions']) / max(1,$metrics['reach'])));
/*
Determine the percentage of engagement (likes, comments, etc.) relative
to how many unique user views, capped at 300%, minimum 0%
*/
$percentEnagement = max(0.0, min(3.0, floatval($metrics['engagement'] / max(1,$metrics['reach']))));
/*
Determine the percentage of unique user saves relative to how many
unique user views, capped at 100%, minimum 0%
*/
$percentSaved = max(0.0, min(1.0, floatval($metrics['saved'] / max(1,$metrics['reach']))));
/*
Determine the percentage of video views relative to how many unique
user views, capped at 5,000%, minimum 0%
*/
$percentVideoViews = max(0.0, min(50.0, floatval($metrics['video_views'] / max(1,$metrics['reach']))));
/*
Apply our weighted and dampened "reach multiplier" for impression %
- normalizing our 500% maximum to max of 100%
*/
$reachWeightedImpressions = ($percentImpressions / 5.0) * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for engagement %
- normalizing our 300% maximum to max of 100%
*/
$reachWeightedEnagement = ($percentEnagement / 3.0) * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for unique user
saves %
*/
$reachWeightedSaved = $percentSaved * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for video views %
- normalizing our 500% maximum to max of 100%
*/
$reachWeightedVideoViews = $percentVideoViews * $reachMultiplier;
/*
Impressions account for 25% of this content's score (this content is
seen multiple times by same user)
*/
$scoredImpressions = $reachWeightedImpressions * 0.25;
/*
Enagement account for 50% of this content's score (likes+comments are
easy to do but still should be factored somewhat)
*/
$scoredEnagement = $reachWeightedEnagement * 0.50;
/*
Saves account for 45% of this content's score (if someone saved this
content-they REALLY liked it)
*/
$scoredSaved = $reachWeightedSaved * 0.45;
/*
Video Views account for 10% of this content's score (videos watched
multiple times by same user)
*/
$scoredVideoViews = $reachWeightedVideoViews * 0.1;
/*
NOTE - At this point, total % of all metrics can be higher than 1.0 (100%)
- this is intentional
*/
/*
Set score by combining separate scoring factors and ensure score never tops 1.0
*/
$score = min(1.0, ($scoredImpressions + $scoredEnagement + $scoredSaved + $scoredVideoViews) * $impressionsMultiplier);
/*
Scale content score relative to maximum value
*/
$points = (int)max(0, ceil(self::MAX_POINTS_PER_CONTENT * $score));
/*
if impressions or reach are 0, points are 0
*/
if($metrics['reach'] <= 0 || $metrics['impressions'] <= 0){
	$points = 0;
}					

const MAX_POINTS_PER_CONTENT = 100;
/*
Determine a multiplier or "weight" (used later) to apply based on reach 
(highly dependent on followers)
This is important because a user with 100% enagement but only 1 follower
should not get similar content rating as a user with 50% engagement and
10000 followers.
However, to avoid skewing too heavily in favor or those who have high
follower counts, we "dampen" this multiplier (divide by 500) but ensure
a minimum of "1" here.
*/
$reachMultiplier = max(0.001, min(1.0, (min(500, $metrics['reach']) / 500.0)));
/*
Determine a similar multiplier or "weight" for impressions (this will be
used later for giving small-to-medium bonuses to content with large impressions)
*/
$impressionsMultiplier = 1.0 + max(0.0, min(0.5, (min(10000.0, $metrics['impressions']) / 10000.0) * 0.5));
/*
Determine the percentage of impressions relative to how many unique user
views, capped at 500%, minimum 0% (higher percentage indicates users
keep coming back to content)
*/
$percentImpressions = max(0.0, min(5.0, floatval($metrics['impressions'] / max(1,$metrics['reach']))));
/*
Determine how many times users tapped forward (possible "skip") relative
to how many saw the content
Next, remove 50% to "dampen" this effect, as its normal for good content
to have some taps_forward
Then subtract from 1.0 (100%) to give content that was skipped less a
higher rating
*/
$percentSkips = 1.0 - min(1.0, max(0.0, floatval($metrics['taps_forward'] / max(1, $metrics['impressions'])) - 0.5));
/*
Determine how many times users exited relative to how many saw the content
Next, remove 50% to "dampen" this effect, as its normal for good content to
have some exits
Then subtract from 1.0 (100%) to give content that was exited less a
higher rating
*/
$percentExits = 1.0 - min(1.0, max(0.0, floatval($metrics['exits'] / max(1, $metrics['impressions'])) - 0.5));
/*
Determine how many replies were sent relative to how many unique user
views, capped at 100%, minimum 0%
Also, as replies would typically have a fairly low % relative to
unique user views, we want to "scale" this percentage so that 1-20% = 1-100%
*/
$percentReplies = max(0.0, min(1.0, floatval($metrics['replies'] / max(1,$metrics['reach'])) / 0.20));
/*
Apply our weighted and dampened "reach multiplier" for impression %
- normalizing our 500% maximum to max of 100%
*/
$reachWeightedImpressions = ($percentImpressions / 5.0) * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for tap forward/skip
percentage
*/
$reachWeightedSkips = $percentSkips * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for exit percentage
*/
$reachWeightedExits = $percentExits * $reachMultiplier;
/*
Apply our weighted and dampened "reach multiplier" for replies %
*/
$reachWeightedReplies = $percentReplies * $reachMultiplier;
/*
Impressions account for 20% of this content's score (this content is
seen multiple times by same user)
*/
$scoredImpressions = $reachWeightedImpressions * 0.20;
/*
Low Skip % accounts for 15% of this content's score
*/
$scoredSkips = $reachWeightedSkips * 0.15;
/*
Low Exit % account for 25% of this content's score
*/
$scoredExits = $reachWeightedExits * 0.25;
/*
Reply % accounts for 45% of this content's score
*/
$scoredReplies = $reachWeightedReplies * 0.45;
/*
NOTE - At this point, total % of all metrics can be higher than 1.0 (100%)
- this is intentional
*/
/*
Set score by combining separate scoring factors and ensure score never
tops 1.0
*/
$score = min(1.0, ($scoredImpressions + $scoredSkips + $scoredExits + $scoredReplies) * $impressionsMultiplier);
/*
Scale content score relative to maximum value
*/
$points = (int)max(0, ceil(self::MAX_POINTS_PER_CONTENT * $score));
/*
if impressions or reach are 0, points are 0
*/
if($metrics['reach'] <= 0 || $metrics['impressions'] <= 0){
	$points = 0;
}