In recognizing that our pre-deployment testing had raised unexpected alarms with some customers and our own Support staff, I decided to straighten out the mess by creating a better feature migration path.
We were winding down the days to our next major product release and I thought the planning had been going well. There are always a number of moving parts in a big deployment and it takes no small bit of coordination across teams to pull off a trouble-free rollout.
One of the big announcements this time around was how we had overhauled one of our more prominent features to make it easier to use. And while we had usage data that showed it wasn't exactly mission critical for our existing customers, we still wanted to ensure a smooth migration.
What drove this decision
The upgrade approach we had planned was simply swapping in the new feature in the place of the old one. Any account that was actively using the old feature would continue to work for the 30 days following the release but after that time, all accounts would be have been automatically switched over.
This seemed straightforward enough and indeed, our internal testing and QA efforts proved that this was a clean transition. Where we went wrong however, was not effectively communicating the plan, especially to our internal stakeholders.
The most obvious blunder was in the Release Notes (I wrote) that we circulate internally, weeks before the go-live date. I had made a copy/paste error using some outdated text and reported the exact opposite intent around this migration! As you might imagine, the Customer Success team freaked out when the old feature "suddenly just disappeared".
All fingers were rightfully pointing back to me and my Product Team. What I thought had been weeks of good planning and internal discussion were negated by what was now a small crisis. I needed a good plan to set things straight.
The decision: Instead of forcing a more immediate upgrade on our users, I would stretch out the feature deprecation period and give customers more time and more autonomy to make the transition.
It is worth clarifying that nothing was indeed broken in any customer account and nothing would break as a result of pushing the new feature in the upcoming release. Actually, I could have tried to smooth over the current unrest and push forward as we had originally planned. After all, no actual customer had experienced any problems in their accounts.
But in this case I chose a more cautious path. Even though it would be more work for our tech teams - actually, it would be re-work since we had already removed the old feature and now had to quickly retest the reverted code - I knew the extra effort to create an extended upgrade period would cost us a lot less than having to soothe any confused or disgruntled customers.
Plan of attack
Patience is a virtue - especially in times of crisis. When the voices are beginning to elevate, when email threads are growing exponentially in a matter of hours, and when emergency meetings are being scheduled first thing in the morning, it helps to be able to keep a calm head.
I would have to calibrate our response based on the actual size of the problem.
One of the things I wanted to make sure was that, in attempting to address the problem, we didn't make things any worse than they were already perceived to be. The key to that, in my opinion, was to properly scope the effort. In short, I would need to be able to calibrate our response based on the actual size of the problem.
Understand the full impact of the change
When we did assemble the interested parties in a meeting room and reviewed the situation, one of the first questions I asked was whether we knew the number of customers/accounts that would be affected by the deprecated feature. My own Product team had gathered some numbers weeks earlier but I wanted to know if we had been off in our estimates.
Because there wasn't enough certainty, I had the Engineers run a new set of exhaustive queries to get our hands around the total affected population. Early totals were supplemented by more thorough search results but all reports showed that the impact was far less significant than the recent hype would have suggested.
Call attention to the deprecated feature
With the Engineering team's help, I committed to reinstating the deprecated feature into the product to help affected customers with their inevitable migration. But in doing so, I insisted that we incorporate some blatant visual cues for the end user so they understood that some action was required.
With minimal work, we were able to introduce a glaring design element intended to draw attention to the deprecated feature. This new call to action would be reinforced with other messaging inside the product and externally as well.
Broadcast (better) to internal and external users
The steps required for a customer to move to the new, improved feature were straightforward and would likely only take a few minutes to complete. However, I have learned the hard way about underestimating users and so, to err on the side of caution, I committed to providing multiple migration artifacts.
I had the Product team scramble to develop new collateral including a 1-page cheat sheet and a short instructional video to help guide customers on the migration process. In both, we explained how the deprecated feature would be disappearing in the subsequent release to be replaced by the improved option available now and we included a recommended course of action to avoid any problems in their account.
The impact
In the post-mortem discussion, all the teams agreed that the new feature is a much better option for our end users - we had just stumbled a bit in our plan to roll it out to our customers. It was a good lesson for me and one that I'm happy to have tackled before the official release where it could really have escalated.
Look for more reports from theProductPath around product releases, product culture, and PM credibility here on PM Decisions.
To address one of the most critical problem areas for customers using our software platform, I decided to roll up my sleeves and conduct the fieldwork myself.
The Product Decision: Enlist the help of our User Research expert to coordinate formal interviews with a suitable group of administrators and end users from our active customer base.