Remote Configuration for Mobile Apps

Imagine your software is running on millions of powerful computers. You want to change the behavior of your software by making a configuration tweak. Mobile app developers- congratulations, you are already living the dream. You run a large distributed apps today. These computers are mobile phones, and I want to discuss configuration changes for mobile apps here.

We can easily change the code and redeploy backend services in case of issues. That's not true for mobile clients. A new version of a mobile app with the hotfix can't be rolled to all the customers immediately. This makes configurations (also called remote configurations) especially important.

Most teams with large-scale mobile applications already use some form of remote configuration, feature flags, percentage rollouts, and A/B testing. If you run a small or medium-size app, you should seriously consider remote configuration for your app today.

A fictional app

Let's take a fictional example of an app that allows customers to add products to the wishlist right from the product listing page.

A fictional bug

Wishlisting seems like a relatively easy feature, but it's easy to make mistakes in asynchronous flows. Let's assume there is a race condition in our wishlist feature, and when customers wishlist a product and start scrolling immediately, they see a crash.

Hotfix or mitigation

We now have a nasty bug; our app's crash rates have spiked because of it. While we must hotfix the issue, we must find a way to stop the app from crashing for our existing customers.

If we were running a system where API responses controlled the presence of a wishlist, then it would be relatively easy for us to stop sending that flag from the backend. But in most systems, it's considered a waste of bandwidth to send the exact value of a property (wishlist enabled) for every product. It's a bad design when APIs send 'same' redundant information for all customers.

If we designed this well, we should be able to turn off the wishlist configuration remotely, and our app's UI should stop displaying the wishlist button, hence mitigating the crashes.

Remote configuration - meet A/B.

When I discuss this type of remote configuration, people often wonder why this feature would even be behind remote settings. Many teams that run small apps indeed don't worry about this, but apps with 10s of millions (or more) daily active users would keep this type of feature behind a remote configuration feature flag.

These popular apps would also A/B test wishlist feature. Remote configuration and A/B testing products come together and join at the hip. Rolling out new features with A/B or incremental %age rollout is an excellent product practice, and it becomes a surprisingly effective defense against bugs in-app code.

Remote config, A/B, etc., are different features, but I talk about these concepts in the same breath because the products implementing these features are often a single product. Let me define these features here:

Remote config

The ability to change the configuration values for all customers (perhaps based on app version, country, etc.) is how we look at the remote config.

A/B testing

We are doing A/B testing when we want some of our customers to see one behavior while another set of customers sees another behavior based on different configurations- we are doing A/B testing.

We do this to test product and business hypotheses.

Percentage rollout or feature control

When we roll out a new feature, we should control how many customers see the change using percentage rollout even if we decide not to A/B test the product. IMHO, this is a great defensive strategy and has prevented issues in new features from escalating multiple times for me.

Build or buy?

Isn't this always the question? I almost always buy before I build this type of feature. My teams have used Firebase, LaunchDarkly, and our in-house implementations. I often feel that Firebase and Supabase are heavily underutilized, and we can fully develop many small apps on just these systems.

Most of these systems will allow lazy initialization if app start times are concerned. You can also configure %age rollouts, A/B tests, default values. If you change one value in your configuration with 50 other values, many mature remote configuration systems will only deliver the differences to clients.

Firebase quick start

If you are part of a small engineering team, I recommend starting with Firebase; it's effortless to integrate. There are six steps as described by their remote config implementation document. Their quickstart repositories are pretty helpful too.

1. Add Firebase and the Remote Config SDK to your app
2. Get the Remote Config singleton object
3. Set in-app default parameter values (more about this in next section)
4. Get parameter values to use in your app
5. Set parameter values in the Remote Config backend
6. Fetch and activate values

Local configuration

When remote configuration systems face issues, having default / local configuration in your code saves the day. It also makes sense for you to cache old configuration values and use them in case of failures.

Mobile networks are notoriously unreliable, and if our app relies on remote configuration without having a local default configuration, we may be doing injustice to our customers. It would be best to ship a default value for the configurable properties in your code. Choose your default / local value based on your A/B hypothesis so that you can remove conditions in remote configuration easily.

Small teams can start by keeping default configurations in code, but most mature engineering teams use continuous integration (CI) to update the defaults before every release. The default value of error strings and other localized text are examples of using CI to update default values in builds.

Effective remote config needs segmentation.

It's powerful to change the configuration for all customers. Still, it's even more potent if we can make the changes based on app versions (maybe your app was crashing for a specific version only). Here are some common properties you can use for config control:

1. Device type
2. Mobile make
3. OS version
4. Mobile app version
5. Country

Country-specific remote configs can be a lifesaver when launching a new country. You can define which countries need a GDPR flow, which countries have a different logo/brand, or whether your app is fully functional or in preview mode.

Many teams use remote config to control forced upgrades. App version-based configs shine when it comes to soft or forced upgrades.

The usual best practice of having different configs for dev, QA, staging, and production environments is applicable for mobile app remote config as it's applicable for backend applications.

Asynchronously loading configuration

Loading remote configuration asynchronously is a good practice because it optimizes app start time. However, if you change any visible elements as soon as configuration changes, the customers might experience a janky UI that seems to be adjusting after it's already loaded.

It's better to wait for the configuration to be loaded or use the default UI for the given session when significant UI layout changes happen due to delayed configuration loading.

Analytics and testing

A/B tests and percentage rollouts rely on analytics to be effective. Unfortunately, I have been in the middle of hotfixes where a percentage rollout became troublesome only after going to 50 or 100% customers because we didn't do the smaller rollout properly and failed to verify rollout against analytics.

Having different possible configurations also means your automation has to test some common combinations of parameter values.

What should be in the configuration?

In one of the apps I managed, we had more than 300 remote configuration parameters in another app, and we touched nearly 1000. There is a time to retire some A/B conditions and conditional configurations; I start almost all new changes behind feature flags. Keep the configuration size under control.

If your configuration is too big, it may affect your app's performance. Regularly remove branched code used for A/B and consider deleting code and remote config that's stable and is now considered default behavior.

What's your mobile app config stack? Have you used remote config to turn on / off features to mitigate issues?