Why Is My Direct Traffic Increasing? Dark Search, Probably

If you look at your Google Analytics metrics as often as I do, you have probably noticed a steady increase in your direct traffic over the past year or so. And if you’re like me, it drives you crazy. Why the increase you ask? Well, this is because when traffic is reported as direct, we can’t see how it’s being acquired. Part of my job as an SEO Manager is to drive traffic to a client’s website and tell them how its being done. We SEOs rely on these metrics to make informed decisions on our campaign strategy, and direct traffic is the equivalent to “not provided” keywords: we can’t do anything about it. Or we couldn’t, until now.

Direct traffic, why!?!?!?

Marshall Simmonds Of Define Media Group had a great presentation at MozCon 2015, one of his main topics being this idea of “dark traffic.” Dark traffic is actually comprised of three points: Dark Search, Dark Mobile, and Dark Social, but this post is specifically about Dark Search, and what it means for us SEOs.

Let’s first define what dark traffic is: it’s a URL that cannot be tracked because it doesn’t pass along a referrer string. When this happens, Google Analytics doesn’t know what to do with the visit and dumps it into Direct Traffic. When does this happen? More often than you think. Referral strings are stripped when users go from a secure site to non-secure, links are clicked in apps, under specific browser conditions, and more.

You might be thinking, “ok this is interesting, but why should I care?” Well, the traffic that’s “dark” and being reporting as direct could actually be organic or the result of a successful social campaign. As SEOs our jobs depend on giving our clients results, and dark search is skimming off the top of our hard work. We need to know how to talk to our clients and bosses about these surges in direct traffic, and how they could actually be a result of our campaigns.

Through a segmentation process, Simmonds was able to find that out of 215 million page views about 18% of direct traffic was pointed to a deep link that users wouldn’t actually reach directly.

That’s 18% of direct traffic that could be organic, social referrals, or even incorrectly tagged paid search. 18% of traffic that was “Dark Traffic.” How was Simmonds able to find this out? Let’s dig into his process.

1. Pull Your Direct Traffic

Remember, dark traffic is reported as direct. You won’t find it anywhere else.

2. Remove Your Homepage & Section Fronts

Your homepage probably gets legitimate direct traffic from people typing in your URL or bookmarking your page. The same goes for section fronts, or pages within your navigation menu. These are (hopefully) some of your more popular pages, and should be excluded based on the likelihood of being bookmarked or having an easy-to-type url. Use advanced filters in Google Analytics to accomplish this.

Filters are your friends!

Remember to exclude, not include, and use “Exactly Matching” to remove data from only that page. You don’t want to accidentally exclude an entire directory.

3. What’s Left Is “Dark Social”

Dark Social is one of the three major buckets of Dark Traffic. This traffic is “direct traffic” reaching deep links that may not be commonly bookmarked or reached by an exact match URL. Dark Social includes Dark Search traffic in it, which is why it has to be mentioned, but again this is focusing on Dark Search. To segment out the social and get search, we need to take this a few steps further.

4. Compare Filtered Links With Social Campaign Metrics

The reason why this bucket is Dark Social is because some social sites use HTTPS, rather than HTTP. We have to cross reference our social campaigns against the URLs found after our filters are implemented. Most major social media platforms have some type of referrer tag implemented, but you always want to double check.

5. Filter For New Users

Filtering for new users will remove users that may have bookmarked your site and returned recently, or memorized a URL and re-typed it in. After this last filter is done, what’s left is dark search.

6. I Have My Dark Search, Now What?

Analyze all of your leftover long tail URLs. There’s a good chance that these URLs were not typed in directly, and were not tied to a social campaign, so how else could a user have reached it? There are quite a few possibilities, but the point is you ruled it out being direct or social traffic, which leaves us with Organic or some unknown referral source (we are assuming your Paid Search manager is properly tagging their campaigns). Let your client know about this section of traffic, especially if you have seen major increases in direct traffic as of late or during a period that correlates with a specific marketing campaign.

I ran some of our client’s data through this process, and found that 11% of 33 million page views were dark traffic pointing to a deep link.

Traffic pulled from actual accounts.

I unfortunately couldn’t cross reference every account with each social campaign, but the data is still pretty significant. This 11% is traffic that our social media managers or SEO managers can speak to as being “dark” and not actually direct.

Dark Traffic can’t be prevented (yet) but there are some steps we can take as online marketers to cut this number down.

1. Tag All Of Your Marketing Campaigns Correctly

Whatever your preferred tagging method is, make sure any URLs you’ve built that are pointing to your site are tracked.

2. Look For Correlations In Your Marketing Campaigns & Direct Traffic

You can’t go back and tag old traffic, but you can analyze spikes in direct, and look for possible causes of dark traffic.

3. Suggest Switching To HTTPS

Although this is a lot of work and should be thoroughly vetted before implementing, referral strings are passed from one HTTPS site to another.

There you have it! This is what’s (probably) going on with your direct traffic. Always remember to double check your metrics and look for possible bot traffic as well. Hopefully you can now speak to your clients, boss, or in-law about Dark Traffic in confidence.