Every movement begins with a moment.

Establishing Validity in Social Media Measurement

Posted 10 December 2013 12:00 AM by Becca Anderson

The big marketing question these days centers on understanding the value of social media, specifically in terms of ROI. Articles abound discussing formulas and algorithms to calculate this valuable number. However, the output of any algorithm is only as good as the input, i.e., “garbage in, garbage out,” as the saying goes. The best way to prevent this pitfall is to be as accurate as possible in measuring social media key performance indicators (KPIs). 

There are two things to consider when increasing measurement accuracy: validity and reliability. Validity is the extent to which you are actually measuring what you set out to measure. Reliability is the extent to which you are measuring what you set out to measure consistently over time. Priority is placed on validity, as reliability assumes that the measurement is valid. So this blog post will focus on establishing measurement validity. Let’s walk through how to determine whether or not measurement is valid. 

To understand validity, let’s think through an example. Remember analog televisions (you know, the ones with the rabbit ears)? In order to view a TV show, the placement of the rabbit ears had to be just right. Otherwise, you could end up with two shows playing at once, but neither playing clearly, or you could end up with complete fuzziness. The same is true with social media measurement. If you’re not precise in your measurement, you could be seeing the results of both awareness and advocacy goals, for example, without being able to parse the output completely to one goal or another. To ensure your measurement plan’s accuracy, you want to create a validity portfolio that includes the following: face validity, content validity, convergent validity and divergent validity. 

Face validity and content validity are the first barriers to be considered. In some cases, ensuring your measure is face- and content-valid can be good enough, depending on the purpose, scope and cost of the measure. 

 - Face Validity – This is the use of the inner ocular test. Look at your measurement plan and have others look at your measurement plan to determine if it seems reasonable to everyone that you are measuring what you should be measuring. 

 - Content Validity – This is a test to determine that the measure covers a representative sample of all possible behaviors. 

Convergent and divergent validity are deeper barriers to be considered and are related to triangulation of a measure. Triangulation is the idea that multiple measures of the same phenomena be employed in order to best interpret the data that is collected. For instance, using behavioral data in combination with self-reported data will allow the analyst to understand the overall phenomenon better than either set of data will on its own. 

 - Convergent Validity – This is a test to determine the extent to which the data collected from the newly created measure shows a positive correlation with other measures that are reported to measure similar behaviors. For example, if you have self-reported data from a given audience and have created a measure to collect behavioral data from the same audience, the behavioral data should be positively correlated with the self-reported data. 

 - Divergent Validity – This is a test to determine the extent to which the data collected from the newly created measure shows a negative correlation with other measures that are reported to measure differing behaviors. For example, if you have self-reported data from a given audience and have created a measure to collect behavioral data from a different audience, the behavioral data should be negatively correlated with the self-reported data. 

The stronger the validity portfolio, the more accurate the measure is. To create the most accurate measure of social media data, you must complete as many of these validity tests as time and cost allow. The next step is reliability, which will be a topic for a later discussion. 

  Becca is a Digital Analyst at Moxie.





#crisis #socialmediacrisis #socialmedia #digitalmarketing #socialmarketing #strategy #influencers #communication #pr #media #ideation #creation #content #planning #culture Snapchat Snap Inc. advertising marketing social media community management brand millennials authenticity Spectacles Wearables customer customer service Paid Media Podcasts Content Sponsored Ads Content Marketing #content #marketing #media #analytics #digitalmarketing #ideation #creation #production DecisionIQ strategy big data personalization Humanizeit Facebook Facebook Live Video Instagram Stories ftc guidelines social influencer social influencers social media influencers social media broadcasting live video Processing prototype prototyping user experience ux technology programming coding java ideas design visual arts mix reality mixed realities data tutorial sketch Twitter Carousel Ads Ad Exchange Programmatic FBX Disney virtual reality projection mapping RFID Tumblr Voice Tone Axure wireframing UX software collaboration Measuring Usability UX Best Practices Designing User Journeys Stop counting clicks Designing for completion Designing for satisfaction Dieter Ram and User Experience Less is more Less but better Influencer Marketing learnability user interface user-created content gaming maker editor Super Mario Maker Nintendo digital media chat bots messaging mobile applications automation F8 Mark Zuckerberg Business F8 Conference Email office agency communication Slack Facebook at Work meetings productivity Wearable tech Healthcare Experience Patient Experience Service Design Strategy Creative Moxie VR Six Flags Six Flags Over Georgia theme parks Samsung Samsung Gear Samsung Galaxy VR headset Oculus Rift Playstation VR roller coasters Dare Devil Drive Websites Web dev development digital reactions emotion analytics dislike social metrics clicks Brands Consumers CPG Retail Consumer Marketing Consumer Insights Super Bowl Commercials Football Cultural Phenomenon Game adtech martec auto-responders automated marketing Experience Design Interaction Design Ecosystem iOS Healthkit Ad blocking Forbes Condé Nast GQ interactive media ad-light hashtags hashflags audience visual emojis campaign management 1-to-1 marketing omnichannel CES CES 2016 Oculus Sony PlayStation uSens HTC special mapping hand tracking games PetBot WonderWoof BowTie Pets OTT digital distribution Netflix Reed Hastings on-demand #yearahead #2016 #newyear #2016predictions #digitalstrategy #modernism user-generated content UGC Consumer Connection Integrated Marketing Manufacturers Accessibility usability engagement rate disability awareness creative process creativity fail fast hiring ideas imperfectionist perfectionist personnel progress test and learn fast casual restaurants social media campaigns QSR social strategies in-store promotions in-store social media promotions successful social media campaigns Logos Typography Branding #CPG #Retail #ConsumerInsights #CRM #ConsumerDirect #Seamless #OmniChannel #MoxieMakers #frontierism #intelligence #architecture insight moments Unit3C FutureX lab innovation open house Q&A UnitFilms 48in48 General Assembly Non-Profit Ponce City Market Branded design technical design style guide wireframes brand guidelines Consumer Products Interactive Marketing GIF social campaign modern marketing Cannes Lions new media pop culture meme Web Accessibility QA media media buying mobile apps targeting wires Advertisers Apple Store EasyPay self-checkout app mobile payment Sporting Events Sitecore CMS delivery Content Marketing Strategy Brand Content Marketing Content Strategy Brand Content Strategy Social Content Strategy Cannes humanity what3words life saving dot Agency Partnership Agency Relationship Competitive Review Millennial Marketers email marketing omni-channel digital marketing esp email service provider process WCAG 2016 Planning Connected Omni AR Commerce Social Innovation Create Content Strategy OMMA MediaPost Internet Week TV & Video Media Planners Creative Conundrum real-time scalability Customization smartphone loyalty advocacy live streaming content democracy Open Source Node.js Cheerio Handlebars Hyperquext MongoDB Atlassian Stash PhantomJS Shopping On-demand economy Sharing economy Everything on demand Instant delivery iPhone App Amazon Now Build Automation DevOps Native Advertising Machina Forrester SXSW Machine Learning Data Streams Web search results Data Platforms ROI Instagram Purchase Behavior Like Content Consumer Channel Role and Purpose ExactTarget Salesforce Cloud Responsys Oracle Silverpop IBM Agency Culture Innovation and Technology Client Relationships Platform Partnerships Ever-evolving Digital Age geo-fencing tablet Android second-screen Austin FOMO Meerkat scaleable vector graphics Flash animation interactivity code browsers uber-savvy consumers integrated digital-connectedness Valentine's Über Bloom that Open Table Waze marketing research google music art sports illustration sculpture food green entertainment movies photography real time marketing cheat sheet smart homes vine print 3d printing street art painting fashion influencer movie billboard outdoor mobile wallet toys film halloween history space films cars anniversary


Error: All fields are required.