How to Build a Linking Solution for Every Device

This is the second in a two-part series about the future of links. Part one recapped the history of links on the web and how things became so broken on mobile. In this post, we will cover the solution, and what we have learned at Branch as we built it.

---

We need to take a fresh look at how we handle linking on the internet. A new approach to linking infrastructure is needed, adopting the best of what has worked in the past, but designed from the start to accommodate both websites and mobile apps, and flexible enough to adapt as future technologies come on the scene. “Deep linking” will be part of it, but what we need now goes far beyond that.

Requirements for a New Linking Infrastructure

1. Here are the eight critical aspects that this system must fulfill:
2. A single, backwards-compatible address (URL) for each link that is valid everywhere links can be used.
3. Delivers users to the requested destination (the basic definition of a ‘link’).
4. Capable of handling both public destinations that should be discoverable, and private destinations that require authentication or authorization.
5. Able to detect the device and environment details of the user, and intelligently route to the best-available experience (desktop web, mobile web, native app). This should be possible even if the preferred option is the native app, and it needs to be installed first.
6. Centralized implementation of technical details (e.g., handling failure edge cases), preserving ease of creation and use for individual links.
7. Updatable on the fly (including retroactively for existing links) to incorporate emerging technologies and new edge case failures as they are discovered.
8. Analytics reporting (such as the HTTP referer header and UTM parameters) for the entire traffic pipeline.
9. Able to accept additional, contextual link data (often implemented on the web as URL query parameters).

In other words, a remote link service. An API for link routing. A dedicated link redirection server in the cloud. Hosted deep links. Whatever name you prefer, the reason for it is the same: as the mobile ecosystem becomes more fragmented, link routing is becoming too complex to handle locally on the user’s device. We are going to need something more powerful.

Working on Branch for the last two years, we have realized that a system like this actually does not need to replace existing link standards. Instead, it can mesh parts from all of them together, filling in the holes and adding glue between the pieces to complete the user experience. There are two basic components necessary to make this happen:

1. A remote, hosted service. This determines which experience to use for each visitor.
2. Local, client modules. These allow each app or website to receive instructions from the remote service about what content to display.

Part 1: Remote Service

On the web, we never needed anything like this. Every web browser could handle its own routing based only on the data contained in the link URL, and there was no risk that it might be unable to load the experience. But that happens all the time on mobile — if the app isn’t installed, for example — and the result is frequently an error message. Not ideal. The primary purpose of this remote service is to take everything we know about incoming visitors (device type, app installation status, data from link opened, etc.), use that to determine the best possible user experience (website, App/Play Store, directly to app), and then send them there.

Aside from the simplified routing process, there are several other advantages to implementing this routing logic remotely instead of on each device:

1. It’s possible to follow users in places where they would usually disappear from view (e.g., when passing through the App or Play Stores). The most common example of this is “deferred deep linking,” or the ability to link directly to a piece of content inside a native app that isn’t yet installed.
2. Links can be shorter, because the link address is a unique key for a set of data parameters stored on the remote service. In other words, you no longer need to cram all of the link data and behavior control information into the link itself. This makes it incredibly simple to do “contextual deep linking,” or the ability track things like where the link was clicked, who originally shared the link, and an almost unlimited amount of other custom data and tracking parameters in addition to the link destination.

The remote service is also responsible for creating links. This involves accepting a set of data keys — the content locations on each platform, behavior control parameters, tracking and display configuration, plus other custom data — and returning a link address to reference them. That link is then valid for use anywhere, and if you pass up a system-wide unique ID for each piece of content, it’s even possible to deduplicate for data integrity when multiple links all point to the same thing.

Part 2:  Local Client Modules

Once the remote service has determined which experience is best for a visitor, we need to communicate what should happen next. The app or website needs link data to know what content the visitor requested, and there are tracking and analytics housekeeping tasks to be done. All of this is handled by the local modules, which are packaged as platform-specific SDKs.

Each SDK is optimized to best fit its programing language and environment, and is broadly responsible for the following tasks:

1. Every single time the app or website is loaded/opened, find out if the remote routing service was involved (check if a link was clicked).
2. Retrieve the data from that link and pass it over to the app or website for use.
3. Track that all this happened, and report back to the remote service for analytics purposes.

Links on the web have conditioned us to expect an immediate response, so our new links should be no different: all of this must take place instantly, which sounds easy, but can sometimes require careful choreography so as to remain invisible to the user. And of course the SDKs also facilitate easy link creation by wrapping convenient programming interfaces around calls to the remote service.

A Snapshot of the System

This is the platform began building at Branch in 2014. What is the result, and what have we learned so far?

We’re far past “deep linking” now. This is something much bigger.

SDKs for app linking make perfect sense. There has never been a broadly-accepted, universal approach to app deep linking, so this is brand new territory and Branch now has an SDK for every major platform and development toolkit. On the other hand, web links already “just work,” so while the Branch web SDK technically can be used as a standalone method for routing into websites, it’s almost always better to use a hybrid approach together with standard redirect rules. Any data returned from the remote link service is then used for added value rather than primary navigation.

As for the remote link service, Branch does use a remote backend for much of the heavy lifting, but in practice, we often have to lean on local web browsers for “last mile” functionality. Even if support for the remote service is eventually built directly into device operating systems, routing via web browser will always be necessary for full backwards compatibility.

The end result is a linking infrastructure that does indeed fulfill all eight of the requirements above, and works better than anything else available, in almost every situation. The Branch platform has now been widely adopted by over 11,000 active apps, including most of the top names in the industry.

But the most surprising thing we have learned is just how many ways things can go wrong. Apple and Google both try to pretend that mobile deep linking is a simple thing. A check-the-box, “nice to have” feature that can be implemented by following a few guides buried deep in the developer documentation for each platform. Unfortunately, this couldn’t possibly be further from the truth, and more importantly, it misses the true scale of the problem: if links are to be used everywhere, they need to work everywhere. The true complexity of how difficult this is to accomplish only becomes clear with much painful experimentation.

First of all, there are numerous competing mobile deep link standards, all with different functionality and requirements for implementation:

Note: iOS and Android together control 99.3% of the mobile market. Because of this, Branch has very limited support for other platforms, and they are omitted here for simplicity.

Of course, none of these standards is supported on every platform or OS version: 

Facebook deserves special recognition for inventing an open-source deep linking standard that actually showed promise, and then completely abandoning it:

 

Even within each platform, every web browser does things differently. This matters, because the majority of links clicks still happen in browsers, and every link routing system relies on browsers for at least some core functionality:

  

Things get even messier outside the web browser category. So many other apps are capable of opening links that attempting to generate a comprehensive list would be an impossible task, but here are a few notable highlights:

The Difficulty of Building a Linking In-House

First of all, building a linking system that can accommodate all these edge cases is a lot of effort. At Branch, our team spends every waking moment exclusively working on this, and we have learned that the devil is in the details.

For example, what if you add app deep linking to all your emails, only to realize your unsubscribe link needs to go to the web on every platform? Now your emails are broken even worse than they were before. And even once the infrastructure is built, ecosystem changes happen so frequently that nothing can be relied upon to “just work” in the future. Keeping up to date with ever-changing standards is a full time job you don’t want, because you already have more exciting things to do.

But more importantly, there is a critical final piece we haven’t discussed yet: search. App content doesn’t show up in web searches, and the best workaround Google has developed involves (yet again) the hack of tying apps and websites more closely together. As the app ecosystem continues to develop, this cannot be our best answer to the problem of content discoverability. Just like the vast majority of us use Google for web searches without a second thought (how many other search engines have become their own verb?), you will want your app’s links to be part of the largest mobile content index in the world. This index is Branch, and it is growing every day.

For better or worse, the age of the native mobile app is here. But no advancement comes without costs, and the move to apps means that links — a core concept of the internet — have taken a big step backward. It’s up to us as developers to find a solution that brings back the openness of the web, otherwise apps will be doomed to live inside walled gardens forever.

Appendix: What Could Go Wrong?

This is a pretty fundamental adjustment to the way we’ve grown up handling links on the web, so of course there are some valid objections to address.

What if the service goes down? I won’t know where to send my users!

This is probably the biggest risk of using a centralized system for link routing. But in reality, we already rely on remote services like DNS and CDNs for much of the modern internet. The benefits of these systems outweigh the risks, and link success rates over 99.9% (meaning less than one failure per 1000 attempts) are well within reach.

What about generating links without an active network connection?

When this happens, whether due to network congestion or simply because an extra network call is unwanted, all the link data can be encoded into a long-form URL and then passed up to the remote service the first time that link is loaded. The result is a much longer link address, but the same behavior.

Shortened link URLs are harder to read.

One of the great things about web links is that the addresses seen by users often follow a path/to/content structure. In fact, this is so common that it is taken as a best practice. Structuring things this way was originally reflective of the underlying filesystem of the server where the site was hosted, but has already started to break down with the rise of URL shortener services like Bit.ly and Goo.gl. The Branch platform (along with knockoff attempts like Firebase Dynamic Links) have used a similar short URL approach, because there is no guarantee that content inside a mobile app will be structured the same way as the corresponding website. This frees links from being tied to the technical implementation of any single platform, but nothing prevents building a structure into the addresses of links, and the Branch system already allows customization of these addresses if desired.

How does this fit with Universal Links and App Links?

Even though Apple and Google intended these standards to more closely tie apps and websites — the opposite goal from a comprehensive linking infrastructure focused on flexibility — it’s easy to incorporate them into the system. For our purposes, they function as pre-authorized shortcuts to speed up the user experience.