Introduction to Mobile Analytics & Reporting

I recently spoke as a panelist at the MG West conference on mobile game analytics and after the panel was very surprised by how few companies and people in the industry seemed to know about analytics overall as well as how to properly execute an effective analytics program.

In this post I’d like to answer the first two of the following key analytics questions:

  1. How should you think about analytics at a high level?
  2. What should you be measuring?
  3. What metrics do you need for a successful game?

In a future post, I’ll also address the third question from the above.

1. How should you think about analytics at a high level?

First of all, you should take the approach of trying to think about analytics from a Role and Application perspective. In other words, who is looking at the data (Role) and what are you using the analytics for (Application).

Let’s talk about some of the key applications for a moment first:

  1. Audit: How well is the game doing and how should we think about the game?
  2. User Acquisition (UA): What are the target metrics we’re shooting for and how and how much should we spend to acquire users?
  3. Optimization: How do I uncover insights about how to make the game better and how do we do that?
  4. Anti-Hacking: How can I detect cheaters or hackers in the game?

In a typical mobile game company setting this may break down as follows (simplified view below):

Role Based Reporting Views

In an ideal scenario every Application/Role would have a separate dashboard view that helps focus the user on exactly what they need to know about the game being analyzed.

2. What should you be measuring?

Let’s now dive into a bit more detail about the typical metrics you should look for in each application area:

Audit:

  1. User:
    1. DAU (Daily Active Users): How many users used the app on a particular day
    2. DAU/MAU: DAU / Monthly Active Users; to get a sense for marketing and user loss
  2. Monetization
    1. ARPDAU (Average Revenue Per Daily Active User): Daily Gross Revenue / DAU
    2. ARPPU (Average Revenue Per Paying User)
    3. LTV or some shorter term LTV proxy e.g., 30 Day Cohort ARPU
    4. First Day Conversion
  3. Retention:
    1. D1/7/30: Retention on Day 1, Day 7, and Day 30. This is computed by calculating the actual % of users that logged into the app on the specified day from day 0
    2. % Loss: % of users that are considered inactive after not logging in after a specified # of days e.g., 7 days
  4. Engagement:
    1. Sessions per Day: # of sessions / DAU
    2. Session length: Average amount of time spent by each user in a session
      1. Session length by level
  5. Stability:
    1. Crashes per day:
      1. Server: Number of server or server process crashes per day
      2. Client: Number of client crashes per day
    2. Crashes per 1K (or other # depending on your traffic) DAU
    3. Top 3 crash types: You should try to characterize crashes and isolate where crashes are occurring.
      1. Client: It’s typically more difficult to characterize a client crash but at the least you should send up a client dump to help developers try to figure out what happened.
      2. Server: Server crashes are relatively easy to isolate and you should be able to tell even by the line number where problems occurred
  6. Performance:
    1. Load time for X by device type: One of the biggest sources of user loss is wait times for loading. After playing your game you should try to identify a few potential problem spots and opportunistically track load/wait times for those areas.
      1. Device type: By characterizing load/wait times by device you can see if there are measurable differences in KPIs based on different wait times you will see on the various device types

UA:

From a UA perspective, we like to measure source effectiveness and do source/user attribution. Basically how effective is a $ I spend to acquire users? What is it getting me?

  1. Marketing:
    1. CPI: Cost per install
    2. Effective CPI: CPI including organics, social, etc. that may reduce the overall cost for a user install
    3. LTV (or LTV proxy)
  2. Attribution
    1. UDID/IDFA lists: You should always check to see if you actually got the users that your vendor claimed to have sent you

Is LTV – Effective CPI > 0?

3 Side Points:

  1. For the majority of the top grossing games on iOS App Store the answer to the above should be: NO. Unfortunately, we still have a broken App ecosystem from Apple… but I digress
  2. Many companies totally screw up LTV calculation and make forward looking projections where they project an overly optimistic LTV, overspend on UA, and in the end get screwed (*cough* *cough* GREE *cough* *cough*)
  3. CPI in itself is not the full story as cheap CPI may come with poor traffic. Do not make the assumption that your overall LTV should be applied to any new source of cheap CPI traffic (*cough* *cough* Tapjoy bots *cough* *cough*)

3. Optimization:

This is the most custom part of building analytics for your game and will require a deep understanding of your game to determine what specific metrics are necessary. The overall objective is to tune/balanc:

  1. Game Economy
  2. User Progression e.g., how fast are users moving through the game in PVE, PVP, any other loops
  3. User Activity e.g., what users are actually doing in your game… how much, how long?
  4. Game Content/Balance
  5. Monetization
  6. Retention
  7. Social/Viral

Here’s a typical list of stuff I came up with for some of my games:

Game Economy:

  1. Sinks and sources of soft currency by level
    • Stack bar graph of each source
  2. Item by popularity (last 3 days, overall)
    • Soft Currency
    • Hard Currency

User Progression:

  1. By Level:
    1. % of players
    2. % of plays
    3. Average amount of money
      1. Highest amount of money
    4. Average net worth (including item value)
      1. Highest net worth
    5. Average # of days
    6. Average # of sessions
    7. Average # of battles
  2. By Loop (e.g., PVE)
    1. Show average progression by Level
      1. E.g., Level vs. Battle # (PVE progression) as a % of users and as a total user count
    2. Show highest progression by Level
    3. Average mastery level by enemy/map

User Activity:

  1. Sessions:
    1. Average sessions per day
    2. Highest sessions per day
    3. Average time per session
  2. Sessions by Level:
    1. Average sessions per day by level
    2. Average time per session by level
  3. Activity Type
    1. % of activity type e.g., PVP vs. PVE vs. Other Loop

Game Content/Balance:

  1. % character type owned
  2. % win by character type
  3. Show top 10 squad types (e.g., {archer, mage, knight}, etc.) by win %
  4. Show top 5 character types by win %
  5. Show bottom 5 character types by loss %

Monetization:

  1. ARPDAU
  2. ARPPDAU (Average revenue per paying DAU)
  3. % of users who purchase
    1. % of users who purchase by level
    2. % of users who purchase by source
  4. ARPU
    1. 1 day, 7 day, 30 day
    2. LTV
  5. Average $ amount spent
    1. $ spent by level
  6. First day buyer conversion %
  7. Store
    1. % first item purchased
    2. % spend by category
    3. Top 10 items purchased
    4. Bottom 5 items purchased
    5. Store heat map by clicks (where do users click the most on the store)

Retention:

  1. Tutorial completion
  2. 1 day, 7 day, 14 day, 30 day
  3. Funnel by tutorial and level

Social/Viral:

  1. % FB connect
  2. Average # of FB friends installed
    1. by FB Connect users
  3. Average # of FB posts (per user, per active user)
    1. by FB Connect users
  4. % of users who have FB liked
    1. by FB Connect users

4. Anti-Hacking

All games can be hacked. Especially mobile games with primarily client-based architectures on Android that will be especially prone to hacks and cheats.

Here is a simple list of things you should track:

  1. Revenue velocity change
    1. If revenue by day decreases or increases by > X% (set X = 20%)
  2. If user net worth > 10% in 1 day after level 5 alert and log
  3. If # of sessions > 10 per day alert and log
  4. If user increases > 1 level per day after level 10 then alert and log

That’s it for now… I’ll also answer the 3rd key question around typical heuristics to indicate a successful game in a future post.

UPDATED: Crash/performance analytics added to this post on 6/11/’13 based on feedback from Lukasz Twardowski. Also Lukasz makes a very relevant point that the term “analytics” is often used in our industry (and me too) where “reporting” is actually the more appropriate term. Thanks Lukasz!

13 Comments

  • Anibal Garcia

    Posted June 17, 2013 11:25 pm

    On the brick&mortar side of gaming trying to learn about igaming and be prepared.

    Nice job.

    Thanks

  • Rami

    Posted June 21, 2013 6:38 pm

    awesome post ! Nice break down. The thought process does actually help a lot with how to view different parts. Cheers!

  • Tim

    Posted July 9, 2013 8:41 am

    Hey Joseph, thanks for this really brilliant article (I am not exagerating).
    I’ve attended many industry events in recent months around the globe and the truth is that many (even very big) players in the market are not close to where they could or should be in terms data driven decision making. I think its just such a still jung, nacsent industry that its hard for everyone to keep up with the tremendous pace of change…think things need some time to evolve. cheers, tim

  • Joseph Kim

    Posted July 9, 2013 9:09 am

    Thanks for the compliments! Definitely appreciate that people find value in my posts…

  • Francis K.

    Posted July 15, 2013 9:01 pm

    Joseph, I’ve been working on social mobile games for several years, attended dozens of workshops from Casual Connect to GDC 20XX, but your site has been the most informative, easy-to-understand breakdown of how social games tick and what to do to maximize returns. Thanks again and keep up the great work!

  • Joseph Kim

    Posted July 16, 2013 5:31 am

    Thx Francis. I’ll keep it up. I’ve got about 20 posts I’d like to write right now so hopefully I’ll get some time to get to them soon…

  • villekautto

    Posted August 17, 2013 9:57 am

    Nice one, goes along the lines of the Lean Analytics book, but you were able to list the actual metrics and how to measure them in a slightly clearer manner, thanks! It seems everybody is talking about this but very few are actually doing anything about it. 🙂

    • Joseph Kim

      Posted October 19, 2013 6:36 pm

      I’ve never heard of the Lean Analytics book, but I’ll try and check it out. Thanks for the note!

  • Abinash Tripathy

    Posted October 19, 2013 4:24 pm

    It would be great if you can list the tools available for all the analytics/retention metrics you are tracking. Is there one tool or are you using a combination of these?

    • Joseph Kim

      Posted October 19, 2013 6:35 pm

      I’ve used a combination of tools depending on the game and company (typically there may be existing relationships or special deals in place).

      For cost conscious guys who are more technical and know what they are doing, I recommend a combination of Leanplum + custom analytics basically filling in where you don’t have third party support. Have also used Kontagent and like what I’ve seen from Swrv. There are advantages/disadvantages for each platform.

      I’ve also looked at Playnomics (which is slightly different) and like what they can do from a theoretical perspective but haven’t actually used it yet. Also, heard a lot of good things about Staq (acquired by Playhaven) but haven’t used it myself yet.

  • π (@KhaTarNak_KhauF)

    Posted September 22, 2014 9:08 am

    I have been looking for what game data matrices to track for my game and am elated to find this resource. Would help me and my team to plan and measure the game’s performance henceforth.

  • Joseph Kim

    Posted September 23, 2014 2:17 am

    Glad I could be of help. Let me know if you have any questions!

  • Chad Kihm

    Posted May 31, 2016 5:37 pm

    I can’t believe I paid $800 to go to GDC, when I’ve learned 10x as much about mobile gaming development here. Thanks Joseph.

Leave a Reply

%d bloggers like this:
Read more:
2016 Mobile Gaming Trends Report: Key Take-aways

SEGA Networks (mobile operating division of SEGA) partnered with Sensor Tower this year to present the 2016 Mobile Gaming Trends...

Close