Talljoe

About two-and-a-half years ago we started work on a brand new payment-processing service. We knew we were going to need to store some data and based on our roadmap we knew that schema was going to have to be flexible. Several days of discussion followed regarding what technology we use, which provider, schema, etc. We weren’t getting anywhere and then we had a crazy idea.

“Let’s create an implementation of the Transaction Store implemented in memory.” It was kind of absurd but it was also exactly the right thing to do. It took almost no time to build and test and we didn’t have to worry about migrations, ORMs, etc. We were a long way from production and all we needed was a place to store transactions so that we could work on payment processing.

By deferring the decision about how to store the data we were able to focus on the data itself and how that data was used. We were in a much better position to answer all of the questions once we had a better picture of the shape and usage of the data. This also allowed us to discuss the merits of each solution in the context of the actual problem instead of everyone focusing on their favorite data-storage technology.

Something similar happened several years ago on the Microsoft Store team. The product needed an engine to calculate discounts on the site. A vendor had quoted us a couple of months and they weren’t willing to commit to a date that we wanted (plus the design was bonkers). With one month before ship and a blank slate I sat down to tackle the problem myself.

The first task was asking the business team to give me the top five discount types they wanted to handle. Dollar off, percent off, linked discount (i.e. discount on B with purchase of A), promo code, etc. I sat down and started knocking them off the list one by one. I didn’t pre-plan the schema, just relied on good tests and simple migration strategies.

Evolution of the schema:

• Dollar off: [ SKU | Discount ]
• Percent off: [ SKU | Discount | Fixed/Percent? ]
• Linked discount: [ SKU | Discount | F/P? | Required SKU ]
• Promo code: [ SKU | Discount | F/P? | Required SKU | Promo Code ]

By the end of the month I had implemented the top 4 or 5 discount types they wanted giving the business team several months of runway. If I had tried to design a schema to handle all of the discounting types the business team could imagine before writing code it’s unlikely I would have had a plan in the first month.

This wasn’t without its problems. Due to lack of time and resources the “user interface” for the business team was a spreadsheet that they uploaded to the website. And in later iterations the design suffered from combinatorial explosion: when a discount was “buy a copy of Windows and get 10% off a mouse” you had to deal with multiple SKUs for Windows (in the “Required SKU” column) for each mouse SKU. But it worked and every discounting type that was asked for could be handled with the existing schema (promo code for a linked discount? we can do that) or a simple change to the schema.

Side note: solving the combinatorial-explosion problem turned out to be pretty simple. We were building a new UI to enter discounts (to replace the spreadsheet method). The business team wanted to be able to select product families instead of per SKU. Rather than rip apart the data store we built a UI data store with enhanced information and then projected to the existing data store. This meant we didn’t have to change any of the code in the discounting engine and we were able to decouple changes in the UI from changes in the engine and vice versa.

Many people are nervous about delaying decisions until the last possible moment. There is comfort in an up-front design. In my experience, however, by delaying decisions until they absolutely have to be made you have the most information available to make the decision, which can only be a good thing. So many of my “last-moment” designs turned out much more elegant than I could have come up with at the start.