Advanced Minecraft Marketing Analytics

I’ve mentioned a few times the power of my analytics system in the Spigot community and recently in AdminCraft on Reddit. Most advertising campaigns simply track impressions and clicks. But to a server owner, we need more data than that…. We need to know who actually joined the server and continues to play!

I have designed such a system, where I can track clicks, how many joined the game server, then out of those, who stay to the day 5, 15, 30, 60 and 90 markers! I can see when campaigns generate a lot of clicks, yet low players, or worse: players who don’t actually stay.

This article will give a rough idea on how I did it incase someone is curious to implement it on their own.

I want to be straight that the implementation isn’t perfect or the most optimized, but hey it gives you data where you normally would not have data, so that’s a clear winner!

Requirements / Disclaimer

This system only works for advertising campaigns that involve website clicks. Campaigns that serve purely as a banner and a server connect address can not be realistically tracked.

My implementation all revolves around Google Analytics, but the concept can work for any form of marketing system you want to adapt it to.

You should be using analytics.js (I do not think this works with ga.js, the old Analytics system) on every page of your site.

You will also need to be an experienced developer or have one on hand.

I WILL NOT HELP ANYONE IMPLEMENT THIS. I do not do contract work / consulting services, but feel free to ask questions on Spigot IRC.

Tracking that click

First off, my system requires that all inbound campaigns need to land the user on the website. If someone see’s your server connect address and connects directly, then you can’t track that – but hey! That’s usually a FREE conversion , so don’t complain :)

Here is an example database table that I use:

CREATE TABLE `conversions` (
 `clientid` varchar(255) CHARACTER SET latin1 NOT NULL,
 `convert_id` varchar(255) CHARACTER SET latin1 NOT NULL,
 `ip` int(10) unsigned NOT NULL,
 `ua` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
 `date` int(11) unsigned NOT NULL,
 `lp` text CHARACTER SET latin1,
 `converted` tinyint(4) unsigned NOT NULL DEFAULT '0'
 PRIMARY KEY (`clientid`),
 KEY `ip` (`ip`),
 KEY `converted` (`converted`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

our goal is to assign a Google Analytics Client Id to every user. Additionally, I assign a unique ID to every user separate to the client ID, as some users may have addons that constantly change their Google Analytics Client ID, so this prevents inserting an entry into DB every time their client id changes.

When a user comes in and has a Google Client ID set, assign them a convert ID. then insert an entry into the database.

Explanation of fields:

  • clientid: Google Analytics Client ID
  • convert_id: Field you generate to associate to a session
  • ip: users IP address – to be passed later to Google Analytics for conversion and match to a newly joined player
  • ua: Users User Agent – to be passed later to Google Analytics for conversion
  • date: current time when seen
  • lp: URL the player first landed on – so you can know what web page they visited.
  • converted: has this record triggered a conversion not.

So, once you have their IP and GA client ID tagged into the database off that first page load… You got them :) If they join, you will know what campaign they came from.

Monitoring Conversions

Now that you have them in the database, you need to have a monitoring system that checks your game server. This is the trickiest part of pulling this off, as how you do this will really vary based on your servers data infrastructure.

We have a users database table that we can easily query on and find who has played in the past few hours and has not triggered conversion events. So we run a query, get the users IP address, and then look for matching conversion table entries on the same IP address that is also not marked as converted.

In the event 2 people joined on same IP from 2 diff devices, you would have 2 entries in the conversions table, and then each would match up to each other, still resulting in 2 conversions. If 2 accounts join but only 1 device hit the website, then only 1 conversion should be marked initially, but they are likely to then trigger a 2nd conversion event once they hit the website with a new device (that doesn’t match the convert_id you generated).

Now… the actual marking of a conversion when this happens!

Setting up your conversion event

Now you need to configure Google Analytics to have a goal to count the conversions.

Go to Admin > Your Property > Your Primary View > Goals

Add a new Goal using an Event like so:

This step doesn’t have to be done first, but it does need to be done to view it and there’s no benefit to do it after rather than before.

Google Analytics Measurement Protocol

The way the Google Analytics tracking system works is extensively documented, and even encouraged for you to do your own backend server side reporting of the event on behalf of the user!

Read all the documentation: Google Analytics Measurement Protocol.

Now we’ve got a list of rows from the conversion table of who joined the game server, with their GA Client Id IP and URL.

Now your backend monitoring system needs to call out to Google Analytics in the exact same fashion the client browsers do, but I’ll save you the trouble, here is the code I use to do this:

 

I do even further events like Day 5/15/30/60/90 tracking, so I pass in other events for those. To support that type of data, you will have to analyze your data and figure out the best way to compute who should be converted for those cases.

Tagging Every Inbound Link

Now you have the infrastructure set up, you need to ensure every direct campaign you control is tagged! This involves setting utm_* parameters on your inbound links.

Learn more about UTM Parameters

I personally use our emc.gs url shortner, then create links that redirect to pages with the utm parameters, so that I can do “reddit.emc.gs” and it will tag that link with the Reddit tracking parameters. If you want to set up a url shortener like we have, check out YourLS.

I have modified it for our needs though, making it so &gac = &utm_campaign, &gas = &utm_source and &gam  = &utm_medium for quick changes of properties like reddit.emc.gs/?gam=ad-4

I suggest setting up a redirect system on your main website domain and not an alternate domain like we do, because many advertising systems dislike domain mismatches, and we had to set up empireminecraft.com/go/foo to redirect to emc.gs/foo which then redirects back to the target of emc.gs/foo just to trick some of those platforms.

Enjoy having data about your campaigns and who sends you the best traffic :)

Bash Colors for Minecraft Shell Scripts

If you ever wanted to write scripts in Node.JS (or any JavaScript runtime) for Minecraft, one may be interested in the ability to convert the Minecraft Color Codes into their Bash Color Code versions.

I have wrote a few methods (color map credit to mcrcon) that will help with outputting colored text using Minecraft color codes.

Usage is like

console.log(mccolor(c("3", "red Text) + c("a", "green text")));

Async Development – Task Chain – Java Control Flow for Bukkit

Reposting because Google has bugged out and ended up dropping this from Google :/ So making it look new to get it back on Google.

So as any Bukkit developer knows, the API is not thread safe! And to make matters worse, there is no concrete Java Control Flow API in Bukkit.

However at Empire Minecraft our server very heavily depends on our MySQL Database to provide features. So running database queries on the main thread is common but undesired, and Java control flow is needed.

Running queries async creates complicated java control flow issues, need to run this query… now need to access the bukkit api, so return to Sync processing, oh wait, now I need to act again with another database query!

Easy to avoid all that java control flow trouble by running everything sync – but then performance can be hurt.

Node.JS / JavaScript has this problem in great detail, and turns code into call back hell, so there are plenty of Flow Control libraries out there like Q, Async, Chainsaw and more.

Therefor, to avoid Java running into this same callback mess, I wrote an elegant Java Flow Control system on top of the Bukkit Scheduler. Continue reading

Productivity – A key skill in development

Been a while since I posted, but I’ve recently got a few things down lately I felt the need to share.

A major asset to being a developer is the ability to be productive. Remember, time is money. One thing that always irks me when working with junior developers or designers is them doing actions that I felt could be done much quicker with a proper IDE or practice. So much time is wasted doing repetitive things every single day.

Ever watched someone navigate a folder hierarchy to get to a file they want to open? A good 30 seconds or more can be spent there, especially if that file is on a network mount!

With a proper IDE with good navigation support, navigating to a desired file can be drastically reduced. I personally own a license to IntelliJ IDEA 14 Professional, and use it for my Java projects and until recently Web Projects. I value efficiency so much, that I now have even purchased PhpStorm 8, even though IDEA supported PHP. Namely for things like Debugging, but thats another topic.

PhpStorm tailors the experience to web development more, but some of the key notes are still in IDEA.

Things such as opening a file, if you work in an extensive codebase thats been developed over many years, can likely be pretty deep and take a while. But with IDEA and PhpStorm (And likely other IDE’s too, I know VIM does similar), a Key Combo away gives you a nice search box that you can type in a class name or file name and find the exact file you want.

If that file is on a network mount and you have a varying file structure that you cant remember exactly where that file resides, how many minutes per ACTION are you wasting opening up that file?

Then inspections… The ability to see errors in your code before you even run it. I’m not talking about just syntax errors here. Typo in a variable name? That’s not a syntax error, but a good IDE will show you that variable is unused, and another variable hasn’t been defined yet, giving you a sign you have a problem before you even leave your editor.

This is just the tip of the iceberg. IDE’s are built to save you time, yet so many people feel they are hard core because they built a website in Notepad++. You are not hard core, you are wasting time. Use the tools that have been built to make you more productive, and strive to increase your productivity.

I personally recommend Jetbrains products as they have a STRONG focus on productivity (Help -> Productivity Guide) and get more small productivity features all combined into 1 product that many other smaller editors have. You have so many goodies to play with to make you more productive. But the key point, use an IDE, be productive, save you / your company time, and stop trying to show off by claiming you did it all in Notepad. The only people your impressing is juniors who don’t know any better.

If you have a good developer skillset, don’t waste your precious time on inefficiencies in workflow.

Apache Macros – Simplify your config

Using Apache Macros

Many people host small time hobby websites or even websites for family members, friends and clients on a single server. This will lead to quite a lot of repetition for the same apache site definitions over and over again. Thankfully Apache Macros mod will solve many of these issues. This mod will let you create config templates, that can then be re-used over multiple sections of code, allowing you to pass in variables to fill in on use.

Lets get it installed!

sudo apt-get install libapache2-mod-macro
sudo a2enmod macro

Now, you can start using Macros in your site definitions, to replace common configurations.

» Official Documentation for Apache Macros

Examples of Apache Macros

In this you can see a pretty simple Domain macro. This will set the ServerName, and sets a www. alias. Now we look at the Site macro. In this example you will see it calling other macros for Log, GrantAccess and ForceDomain.

To use this, one could simply add inside of the <VirtualHost> this line:

Use Site mysite.com

And then accessing mysite.com would redirect to www.mysite.com, and log to /var/log/apache2/sites/mysite.com_access.log. Needless to say that likely cuts out 99% of the configuration you’re doing for a simple wordpress site you host for a relative.

And since the Apache Macros are parsed at config load, there’s no impact to your servers performance for using it!

For enterprise grade setups, you’re likely already using Puppet to get the same benefits and only running 1 product per server any-ways, but for those of us kicking it hobby level, Apache macros helps quits a bit! Enjoy :)