Do Metrics Matter

How do we use metrics, and how do they matter

Louis: nuance: some marketing/corporate person wants you to measure your success

prove that you are performing, show that your work is successful

"give me a break"

value, who's an influencer, ...

Rachel: community site "" (now data driven), without content except for "download"

Metrics to help make decisions, to make communities better, stronger

Example Metrix (Web-Analytics):

source of traffic
time on site
keyword times
paths through the site
popular content

Commits ("what people do")

how much is a commit "worth" ? "how much work is happening?"
file touches, lines, commits , code authorship

Metrics only useful in context. You have to explain them.

"Provide as many you can and explain"

Obamba Reelection campaign as a big example of use of metrics / big data

Have to be careful and accureate in terms of the questions you're putting out and how you represent them.

Explains reasons behind numbers, and reasons, causes for changes.

Ross: "Metrics are another sense that you have".

Jen: "Use the metrics to tell a story"

Awareness, inbound marketing, Marketing goals, increase engagement, specific goals for people on the community

Telling stories about the success / effects of those goals. Metrics should map to the goals.

etsy talk: every last element on their website is A/B tested, continuous integration system with feature flags

Zohar: community health dashboard - tracks everything imaginable, don't care about any of the metrics

only uses the metrics when you need them

have a solid strategy that you understand, have the metrics back the strategy

John: Metrics tell you where there is a fire, when they change dramatically you're able to see the high level overview

Jason (opensource.com): Daily reports, Weekly (shared), Monthly: new followers, comments on blogs, downloads, ...

analysing impact/success of talks: pageviews, comments, video traffic

See example weekly report below - report

Jen: "use listia?" to track detailed video

Jason: metrics decide feature decisions, make pivots, run experiments, constantly evolve / prototype

Louis summary:

Metrics are important
Narratives are important
Act on the metrics (immediate basis, respond)

Jason: set goals for new things, if goals are not met scrap it

Kyle: measure how successful actions are, measure response, usefulness of technical content, tweet,publish times, gauge what community said, also less concrete things (touchy, feely, style of writing

Bryan: Metrics are only good for snapshots in time

Zohar: ask the community, use PRs, ask "why", "what is important for you"

Ward: Metrics for everything, "you can use metrics for everything that you want to be supported", constantly challenge everyone others metrics, metrics for decisions - purchase new hardward

Metrics for community "Qlikview dashboard", every transaction, create competition between business units

Louis: two models

shapre community according to strategy (manipulate)
metrics provide insights, cater to community
monitor community's health, heal it

Luke: Targets for metrics, but longterm, but too late after a time, Pivot vs. hitting a target

"what are the right metrics"

how to I work with it, what do I measure, which measure to choose, what do they mean, do people understand the metric, why do choose it

how well anaylse the metrics-data

"it is your job to give your boss the right metrics"

Ross: a metric that everyone wants and no one can get, e.g. in open-source: usage, in the business world it is "licenses"

Metrics backfire, if used to prove the wrong thing

Bryan: SMART, specific metrics, you can track but you cannot really measure

Jason: Leading indiciator (target), track (lagging indicator), cannot directly influence lagging indicator, the measures are helping to understand the impact and make decisions

Metrics that you can actually influence.

Zohar: you need to know what you want to a achieve

Ryan: things that you cannot measure ....

Metrics are not the right way to influence,

qualitative metrics

Louis:

metrics: downloads vs. usage
people ignore explanations, they just take the numbers

Asheesh Ubuntu: track first contributions, assign mentor on that

regular meetings, you want to take control of this discussion, you don't want a marketing person take control of it

Tools? Template narratives:

Google Analytics, Conversions, Google Dashboard, Kissmetrics
Chart.io
R, Gnuplot
Ducksboard
Spreadsheet
Sproutsocial
Moz

Metrics help you to see if something that you worked on actually worked.

Example metrics report

Open source downloads, Summer reading, and open source project management are just a few topics that were popular last week.

And now, your weekly stats...Here is your weekly snapshot from opensource.com. Check out some of the metrics that our team tracks. And thank you for sharing your favorite content on Hackernews, Reddit, Twitter, identi.ca, Google+, Facebook, Slashdot and many other places!

== From 7/6 - 7/12 ==
Page Views: 34,131 (previous week 37,891)
New registered users: 24 (previous week 27)
Comments: 34 (previous week 21)

Twitter: 12,440 (+110)
Facebook: 8,321 (+48)
Google +1's: 951 (+13)