It’s neither possible nor desirable for changelogs to be bland lists of features
Aug 29, 2015
Changelogs can be straightforward for simple apps that have no server-side component, but for those where most features are turned on/off or tweaked heavily server-side, publishing a client app’s changelog will be either incorrect, or useless, or both.
TheNextWeb argues that changelogs for each client update should be “useful”, but that’s sometimes neither possible, nor desirable.
There are several reasons for this.
The release train model
Many large app publishers follow a release train model. They don’t wait to push app updates until a new feature is ready. They’ll push every two weeks (as Facebook say in their release notes), or once a month, or at some other predetermined interval.
This ensures that important bug fixes unrelated to upcoming features are not held up for indeterminate amounts of time. It makes sure that delays in one feature do not impact other unrelated features. If a feature is ready for a particular release, it gets enabled in the shipping release; if not, it gets cut from that release and can be enabled in the next one.
In such a release model, it’s hard to write accurate release notes and keep them accurate throughout the release testing phase. It risks leaking confidential features before they’re ready. It risks raising user expectations and then shattering them if a feature does not make the cut.
Server-side feature rollout
Some features in large apps are controlled server-side. Client apps are published according to each platform’s review delay timelines. iOS apps need to leave time for a review by Apple. Android apps need time for review by Google. Coordinating all of these to launch a feature at a specific time is tricky.
New features are often rolled out in stages. When feature rollout is controlled server-side, it can be enabled for 1% of users first, then 10% and so on, up to 100%, across Android, iOS, and Web. Managing this rollout entirely server-side is much easier than trying to do it client-side, e.g. through individual app store staged rollouts.
For the 90% of users who have not yet seen a feature enabled when the rollout is at 10%, the changelog will be flat out incorrect. It’s best to leave it out and announce the feature in-app when it’s enabled for that specific user.
A/B experiments
Even after a feature has been rolled out widely, there will be tweaks made between individual users. If you start listing all the combinations of A/B experiments going on at any given time, the combinatorial explosion will make the changelog impossible to read or parse.
Some A/B experiments must be kept secret from the users; if you reveal too much about what is being tested, you tend to skew the results of the experiment. Such experiments need to be kept away from media attention as well, lest a subset of potential users in affected buckets go on to read about them. There is nothing nefarious in this; just proper experiment design that requires that participants not be aware of what is being evaluated.
Upgrading across multiple versions
What should a changelog say when you go from version 1.0 to version 6.0 in one fell swoop? That sometimes happens, if app update checking mechanisms have been misconfigured, or an app requires manual approval for updates.
In such cases, what should a changelog meaningfully say? Should it aggregate all changelogs between 1.0 and 6.0? Should it show only the delta between 5.9 and 6.0? It’s just not feasible or practical to list exactly the set of things that are new in the most recent update because you may never know the baseline from which the user is upgrading.
Humor is part of branding
Changelogs that make me chuckle are a nice touch on otherwise bland days. Some brands explicitly espouse humor as part of their branding. Including playful funny text (well, funny to some, perhaps) is their way of strengthening their brand, of grabbing the attention of new users, and subtle advertising. What’s the harm in that?
Instead, prefer in-app promos or mentions
Not all users will read changelogs, so even if they contain useful information, it will likely be skipped. If you want to convey to your users when a new feature is ready for them, or want to highlight how to use it, prefer to use in-app mentions or promos. A quick text field with a line or two of text, or a bubble with a link, can be far more informative than putting that same information in a changelog.
Of course, make sure you don’t interrupt the user flow or annoy them with these promos.
