Introduction to Mobile Analytics & Reporting

I recently spoke as a panelist at the MG West conference on mobile game analytics and after the panel was very surprised by how few companies and people in the industry seemed to know about analytics overall as well as how to properly execute an effective analytics program.

In this post I’d like to answer the first two of the following key analytics questions:

1. How should you think about analytics at a high level?
2. What should you be measuring?
3. What metrics do you need for a successful game?

In a future post, I’ll also address the third question from the above.

1. How should you think about analytics at a high level?

First of all, you should take the approach of trying to think about analytics from a Role and Application perspective. In other words, who is looking at the data (Role) and what are you using the analytics for (Application).

Let’s talk about some of the key applications for a moment first:

1. Audit: How well is the game doing and how should we think about the game?
2. User Acquisition (UA): What are the target metrics we’re shooting for and how and how much should we spend to acquire users?
3. Optimization: How do I uncover insights about how to make the game better and how do we do that?
4. Anti-Hacking: How can I detect cheaters or hackers in the game?

In a typical mobile game company setting this may break down as follows (simplified view below):

In an ideal scenario every Application/Role would have a separate dashboard view that helps focus the user on exactly what they need to know about the game being analyzed.

2. What should you be measuring?

Let’s now dive into a bit more detail about the typical metrics you should look for in each application area:

Audit:

1. User:
   1. DAU (Daily Active Users): How many users used the app on a particular day
   2. DAU/MAU: DAU / Monthly Active Users; to get a sense for marketing and user loss
2. Monetization
   1. ARPDAU (Average Revenue Per Daily Active User): Daily Gross Revenue / DAU
   2. ARPPU (Average Revenue Per Paying User)
   3. LTV or some shorter term LTV proxy e.g., 30 Day Cohort ARPU
   4. First Day Conversion
3. Retention:
   1. D1/7/30: Retention on Day 1, Day 7, and Day 30. This is computed by calculating the actual % of users that logged into the app on the specified day from day 0
   2. % Loss: % of users that are considered inactive after not logging in after a specified # of days e.g., 7 days
4. Engagement:
   1. Sessions per Day: # of sessions / DAU
   2. Session length: Average amount of time spent by each user in a session
      1. Session length by level
5. Stability:
   1. Crashes per day:
      1. Server: Number of server or server process crashes per day
      2. Client: Number of client crashes per day
   2. Crashes per 1K (or other # depending on your traffic) DAU
   3. Top 3 crash types: You should try to characterize crashes and isolate where crashes are occurring.
      1. Client: It’s typically more difficult to characterize a client crash but at the least you should send up a client dump to help developers try to figure out what happened.
      2. Server: Server crashes are relatively easy to isolate and you should be able to tell even by the line number where problems occurred
6. Performance:
   1. Load time for X by device type: One of the biggest sources of user loss is wait times for loading. After playing your game you should try to identify a few potential problem spots and opportunistically track load/wait times for those areas.
      1. Device type: By characterizing load/wait times by device you can see if there are measurable differences in KPIs based on different wait times you will see on the various device types

UA:

From a UA perspective, we like to measure source effectiveness and do source/user attribution. Basically how effective is a $ I spend to acquire users? What is it getting me?

1. Marketing:
   1. CPI: Cost per install
   2. Effective CPI: CPI including organics, social, etc. that may reduce the overall cost for a user install
   3. LTV (or LTV proxy)
2. Attribution
   1. UDID/IDFA lists: You should always check to see if you actually got the users that your vendor claimed to have sent you

Is LTV – Effective CPI > 0?

3 Side Points:

1. For the majority of the top grossing games on iOS App Store the answer to the above should be: NO. Unfortunately, we still have a broken App ecosystem from Apple… but I digress
2. Many companies totally screw up LTV calculation and make forward looking projections where they project an overly optimistic LTV, overspend on UA, and in the end get screwed (*cough* *cough* GREE *cough* *cough*)
3. CPI in itself is not the full story as cheap CPI may come with poor traffic. Do not make the assumption that your overall LTV should be applied to any new source of cheap CPI traffic (*cough* *cough* Tapjoy bots *cough* *cough*)

3. Optimization:

This is the most custom part of building analytics for your game and will require a deep understanding of your game to determine what specific metrics are necessary. The overall objective is to tune/balanc:

1. Game Economy
2. User Progression e.g., how fast are users moving through the game in PVE, PVP, any other loops
3. User Activity e.g., what users are actually doing in your game… how much, how long?
4. Game Content/Balance
5. Monetization
6. Retention
7. Social/Viral

Here’s a typical list of stuff I came up with for some of my games:

Game Economy:

1. Sinks and sources of soft currency by level
   • Stack bar graph of each source
2. Item by popularity (last 3 days, overall)
   • Soft Currency
   • Hard Currency

User Progression:

1. By Level:
   1. % of players
   2. % of plays
   3. Average amount of money
      1. Highest amount of money
   4. Average net worth (including item value)
      1. Highest net worth
   5. Average # of days
   6. Average # of sessions
   7. Average # of battles
2. By Loop (e.g., PVE)
   1. Show average progression by Level
      1. E.g., Level vs. Battle # (PVE progression) as a % of users and as a total user count
   2. Show highest progression by Level
   3. Average mastery level by enemy/map

User Activity:

1. Sessions:
   1. Average sessions per day
   2. Highest sessions per day
   3. Average time per session
2. Sessions by Level:
   1. Average sessions per day by level
   2. Average time per session by level
3. Activity Type
   1. % of activity type e.g., PVP vs. PVE vs. Other Loop

Game Content/Balance:

1. % character type owned
2. % win by character type
3. Show top 10 squad types (e.g., {archer, mage, knight}, etc.) by win %
4. Show top 5 character types by win %
5. Show bottom 5 character types by loss %

Monetization:

1. ARPDAU
2. ARPPDAU (Average revenue per paying DAU)
3. % of users who purchase
   1. % of users who purchase by level
   2. % of users who purchase by source
4. ARPU
   1. 1 day, 7 day, 30 day
   2. LTV
5. Average $ amount spent
   1. $ spent by level
6. First day buyer conversion %
7. Store
   1. % first item purchased
   2. % spend by category
   3. Top 10 items purchased
   4. Bottom 5 items purchased
   5. Store heat map by clicks (where do users click the most on the store)

Retention:

1. Tutorial completion
2. 1 day, 7 day, 14 day, 30 day
3. Funnel by tutorial and level

Social/Viral:

1. % FB connect
2. Average # of FB friends installed
   1. by FB Connect users
3. Average # of FB posts (per user, per active user)
   1. by FB Connect users
4. % of users who have FB liked
   1. by FB Connect users

4. Anti-Hacking

All games can be hacked. Especially mobile games with primarily client-based architectures on Android that will be especially prone to hacks and cheats.

Here is a simple list of things you should track:

1. Revenue velocity change
   1. If revenue by day decreases or increases by > X% (set X = 20%)
2. If user net worth > 10% in 1 day after level 5 alert and log
3. If # of sessions > 10 per day alert and log
4. If user increases > 1 level per day after level 10 then alert and log

That’s it for now… I’ll also answer the 3rd key question around typical heuristics to indicate a successful game in a future post.

UPDATED: Crash/performance analytics added to this post on 6/11/’13 based on feedback from Lukasz Twardowski. Also Lukasz makes a very relevant point that the term “analytics” is often used in our industry (and me too) where “reporting” is actually the more appropriate term. Thanks Lukasz!