Tell me about a challenging product or platform

Context

The company was using a SaaS billing provider that was not moving fast enough to provide capabilities and features. The integrations were also encoding lots of hidden business logic that made it difficult to understanding what was occuring. The company was, however, processing over $40M a year at that point through this self-service billing infrastructure.

Problem

There were several problems. One of the immediate ones was that the current provider did not support PayPal. As the company grew internationally, the ability to take foreign credit cards was limited on the current platform, so the near-term goal was supporting PayPal.

However, there were underlying "challenges" that emerged after doing interviews across the spectrum with sales, partnerships, product, finance, the front end team, and customer support.

This was a complicated problem based on "futures" or pain points that weren't being addressed.

This resulted in documenting existing flows across the different business, and capturing what they desired, and then categorizing them as must do or nice to have.

Opening this can of worms revealed a lot of issues, such as:

Part of the background research was talking to other companies which worked with either SaaS companies or tried to build their own.

The research resuted in very few successes. In fact, most companies had a two-year deployment cycle (way long that one was appropraite for ours), and still ended up in failure.

I could not find a single, successful deployment after searching for other billing product managers.

Even companies that used well-known tools like Zuora often, after the implementation, would talk about how difficult it was to maintain and upgrade. I dove deep into Zuora as an example -- their implementation and even talked with their team -- about Zuora and saw lots of challenges.

This led me down to look at open source solutions and, the reality was, all of them were bad and poorly supported.

The constraints around the biling platform could not be SaaS was a difficult constraint.

At the same time, parts of engineering felt that they should build it themselves and that needed to be quashed because it seemed daunting.

Actions

Given the background, the most important was to keep the project moving from a requirements perspective. The first was getting clear and detailed on the requirements document. I would review each item, categorize them, and then needed to go back to the stakeholders for every requirement and ask them if, as specified, it would meet their needs.

This involved lots of back and forth because I wanted the specifications to be precise, and many of the requirements had conditions. For example, If X happens, the billing needed to do Y; but if X2 happened, teh biling engine needed to do Z.

The second area that was hard was getting clear on the existing internal logic. This not only meant what the logic was, but proving it by identifying where in the code base this existed.

This was difficult because the logic was spread not only in the front end, back end, but also database triggers and the SaaS provider itself.

I worked through to ensure that all of these components were documented so that if we needed to pull things out, I would be able to do so.

As the requirements became clearer, and while still doing due diligence on finding an on-premise provider that could meet our needs, I needed to address the user flow.

The user flow broke down into categories of type of users, type of purchases, and the hardest of all -- edge cases.

We had different plans, and we were also rolling out ways to purchase "add-ons" at the same time, which broke the typical subscription plan model.

A core part of the business was migrating from free, to pro, to business plans, and this was not always a seamless approach to do implementing business logic.

Also, part of the plans at the times were linked tightly to the configuration and deployment of the actual service, meaning, customers payments only went through when part of the set-up was properly set up in the backend to fully provision the service. A failure on the back end would resut in a failure in payment, and so we needed to discuss from a business perspective what happens in those circumstances.

To do this, I then create a list of human readable user flows, including failures and attestations to ensure it was successful.

We then found a provider by sheer luck and lots of searching and networking to a company which was a SaaS but started to find that their on-premise deployment mattered. This was a godsend in many ways because they were flexible in their software.

So this involved a huge effort to work with the finance and special projects team as well as the vendor on constructing the right business terms that made sense for both parties, including risk assessment.

Once we reached agreement, I was then able to write a simple test harness in YAML that allowed me to convert the human readable test cases into a ton of machine-readable but also human-friendly test cases.

This, needlessly to say, became a huge document.

I also began to provide regularly summaries in a blog format and shipped that to different stakeholders. Here I probably could have done a better job socializing more of what I did in terms of full on presentations to the entire company, but I tried to do them based on milestones. Instead, I would have calendared them and just worked towards milestones on each period.

Once the documents were in place, I then teamed with the enginerering team of two people to build the integrations.

Here I translated the overall user flow into a diagram so we could determine how edge cases were addressed from a systems perspective.

The engineers then developed a plan in which they described the integrations, using an abstraction layer, and we progressed along the integration.

The first major user story was where someone wanted to use PayPal. I prioritzed that since it was a revenue generator and net new. This would not impact existing users or those who did not want to use PayPal.

Then we introduced a migration path for net new users. This allows us to continue running in parallel and test the new user flows.

We then had to deal with more complex situations with our existing users. What happens when they cancelled, downgraded, and how would we migrate them for this smaller existing use case. This helped us to address the logic for migration and slow wean some customers from the existing service. This was a trickier decision because, after looking at the data of who migrated and upgraded, it was smaller segment of the population.

So we sequenced those who went down to free first, and then up the stack from there.

The final migration was what I called the "big bang" migration.

For this I rallied a team of people across the functions to sit in a war room to plan and be part of the migration live. This included support, finance, billing team, and front end teams.

The migration was a 24 hour stoppage of any billing activity across all customers. We turned on the "switch" which migrated a frozen point in time of all customers in the old system and put them and their subscription plans into the new system. Then turned on the new system.

We then put in monitors to watch for billing failures, loss of revenue, and whether there were straggles in the old system versus the new system.

Results

This in the end being successful. Once the lift was done, I was put onto another product and a dedicated billing PM was brought in to augment the UIs.

We saw an increase in revenue from PayPal, but it wasn't super substantial, but incremental nevertheless (I don't remember the value).

We had a clean migration to a new team both PM and engineering.

The feature set let us roll out shortly the usage based and add-on pricing models quickly, which contributed to net new revenue (I was later the beneficiary of this).

Things I would have done differently

I think the communication cadence is something I would have done with more regularity and clarity.

This was difficult to do at the time because I felt overwhelmed, but I think now it would be better to "chunk" up each phase at a given point and then socialize it broadly. This would have benefitted me by being clear on what we going on and forcing a summarization exercise.

It also would have given me more confidence that I had the support of the broader company around the progress.

Even though we hit our projected deadlines, without more frequent updates and celebrations, it felt under valued.

I would have tried to pull out of being the project manager as much.

I was concerned about whether we could hit the milestones, but instead I would have laid out milestones and explicitly given the engineering team responsibility for the execution schedule and just reported their delivery.

Instead, I would have stayed focused on communicating and clarifying what the edge cases and requirements were and done so more proactively. This may have seemed repetitive at the time, but I think that would have ensured everyone knew what we were doing and its potential impact.

Although this ended up being done proactively by engineering, I would have added a spec for the dashboards of the health of the billing system from a business perspective.

Last, I would have set some kind of metrics (if possible) that I could use for future projects. This was one of the most strategic initiatives of the company that had been kicked down the path for two years prior to my joining because it was so ugly.

I think I would have tried to highlight it as an example for other projects and scope. This was super difficult, touched so many stakeholders, and impacted the company strategically but I didn't really get the broad kudos for it.