Bazaarly - A Thought Exercise - Universe A
AI can speed up coding, but architecture and organisational design determine whether that speed translates into actual delivery throughput.
I have been working as a Staff+ Engineer for a while. Even though I recently transitioned into an engineering manager role, I am still hands-on, writing code and leading architectural initiatives as part of my day-to-day work. As a senior contributor, I get to work on architectural initiatives that impact multiple teams and sometimes broader parts of the organisation. Architecture is all about trade-offs, and when evaluating architectural options, I often have to consider different approaches based on current and future organizational needs.
A few days ago, while taking a long walk over the weekend, I found myself thinking about these trade-offs. I was trying to figure out how to improve an organisation’s productivity with the help of AI tools - would that simply require introducing AI tools directly, or would it also require architectural changes? If further changes are needed, what would they be?
At some point, I had the idea for this thought exercise, which I have now turned into a blog post. I would encourage you to answer the questions I have asked in this post and in the follow-up post. Once you do, please share your answers with me at contact at my website’s domain. If you are based in Berlin, I would also love to meet for a coffee and discuss these ideas further. I am always open to meeting like-minded people and talking about tech and architecture.
Finally, the views and opinions expressed here are my own only and in no way represent the views, positions or opinions – expressed or implied – of my employer (present and past). Bazaarly, Best Bazaar, and all companies, teams, people, and scenarios described in this post are entirely fictional. Any resemblance to real companies, organisations, products, or events is purely coincidental.
Find the second post here.
Bazaarly - New Marketplace Business in Town - Universe A
Let’s imagine a marketplace business - Bazaarly - a new startup has just launched. Their goal is to connect sellers with buyers, both private and commercial, and create value by establishing a sustainable circular economy marketplace.
Initially, the company was small, with only a few teams, and the architecture of the entire system looked like this -
The Bazaarly founding team consisted of highly skilled engineers who were also pragmatic - they knew that the zero-interest-rate period was over. So, starting with an architecture consisting of 50+ microservices was going to delay their profitability goal. Instead, they focused on aligning their initial architecture with their organisational structure. They had three teams - User, Ad / Listing, and Search - and so they ended up with three services. This way, they were able to fully focus on making their product profitable first, ensuring their business survives without having to rely on funding or expensive loans.
Since there were only three teams, meetings did not usually take long. It was easy to reach a consensus on design and architecture by simply talking to one or two engineers from another team. Feature prioritisation was also quick - product managers and engineers literally worked side-by-side to get things done. The user team were taking care of all user profile-related things (i.e., account creation, profile management), the ad/listing team was focused on sellers and making the listing experience as smooth as possible, and the search team was making sure buyers could find those listings as efficiently as possible. Things were moving fast - they were shipping business values within days, sometimes even within hours!
Bazaarly Turns Profitable!
After two years of relentless execution, Bazaarly had become successful! Every seller and buyer was talking about them. They were also able to monetise their marketplace successfully via advertisements and paid listings, and have just turned profitable! They were also gaining market shares rapidly in many listing categories. In order to scale the business even further, Bazaarly’s executive team had now decided to verticalize the business by creating specialised category teams -
A new Real Estate Team were to be created to cater to real estate agents’ unique needs
A new Motors Team were to be created to cater to car dealers’ unique needs
A new Fashion Team focused on gaining market share in circular fashion categories
Each of these teams had its own business units to do market research, to figure out what exactly was needed to gain market shares in those categories. They were working with the existing product manager to build small features here and there. Things were looking exciting.
The initial verticalization of the categories did not have much impact on the architecture. The Listing Team, wanting to keep things simple, added category-specific functionalities by looking up the specific categories in the code. The same is for the Search Team. Since the requested features were not that extensive, they were still able to manage the lookups. However, engineers in both of these teams started to feel a bit “uneasy” - what if these categories also become successful and they start getting requests for features that would require extensive work? Would the “if (it_is_real_estate_category) then enable_this_great_real_estate_feature” logic still be enough to support these needs?
Vertical Teams Getting Staffed
After another six months of execution, the future started looking bright in all the verticalized categories. Both sellers and buyers were liking the features, and so the business decided to increase the pace of work in each of the specialised categories. The business had now decided to create fully-staffed specialised category teams with their own engineers and product managers, managing their own category-specific roadmaps.
First Major Feature for Real Estate Listings: Virtual Tours
During these past six month’s product market research, the Real Estate (RE for short) Team discovered that in order to become the market leader in this segment, Bazaarly’s RE listings must have virtual tour capabilities - a feature where a buyer interested in renting/purchasing a property could view it online by browsing a 3D rendered view of the rooms, kitchens etc. This means that the Listing model now has to support additional properties/attributes needed for the virtual tours. The devs from the RE Team then had an alignment meeting with the devs from the Listing Team and requested that they support the initiative by extending the Listing model to add these new fields. The Listing Team, however, already had a planned roadmap for the full year. In addition, they were also getting requests from the other verticalized category teams. In order to simplify things, they agreed to use Innersourcing, where the Real Estate Team implements the initial changes, which will then be maintained by the Listing Team.
The changes, even though they sounded simple, took a while. This is because the Listing Team was also getting big - they had backend engineers who were working on the server-side/backend, frontend engineers who were working on the Listing frontends, and native engineers working on the Android/iOS applications. Aligning all these people on API contracts, figuring out semantics, etc., took time. In addition, the teams were also geographically distributed across different countries, where they had different working hours, holidays, etc., adding to the communication overhead. All in all, it took roughly one month to come up with the execution plan. The Listing Team started feeling a bit uneasy - they felt communication and alignment on a plan did not take this long before, when there were only three teams in the company.
Once the alignment was done, the actual implementation did not take long. Everything was done in two weeks’ time. Both teams felt really good about delivering a major feature supporting one of the most important categories for the business.
Monetising Virtual Tours
One year had passed. The Real Estate Team was gaining market share rapidly. Everyone was talking about how great the virtual tour feature was - you could already check the house you want to buy online, and only go check a limited number of listings personally once you have created a shortlist. Both the seller’s and buyer’s time was being saved.
The RE business team, seeing the popularity of the feature, decided that Bazaarly would no longer offer this feature for free. If a real estate seller wants to add this feature to their listing, they would have to pay for a monthly subscription product.
In order to make this happen, an API integration between the Listing Team and the RE team needed to take place. The monthly subscription product offering was being developed by the RE devs. They were managing the subscripion lifecycles and wanted the Listing Team to call one of their APIs to check if a seller would be eligible for virtual tours. If the API said “Yes”, then their listings would display the feature, they would be able to add/edit virtual tour details, etc. If the API said “No”, then the virtual tours would be disabled. In addition, the RE Team also wanted to retain the virtual tour attributes in the database even if a seller’s subscription was cancelled, because they could renew their subscription any time and would be immediatley able to use the feature.
A new round of discussions and planning started between the Listing Team and the RE Team. The Listing Team was now getting more worried, because they felt like they were getting involved with a feature which better aligned with the responsibility of another team. Plus, their plates were already full - they had a full product roadmap for the entire year. In addition, the Motors Team and the Fashion Team were also coming up with their own requests, which needed similar integrations.
Next, there were technical challenges as well. Adding one API call when rendering the “View Listing” page would slow down the page load time. These pages are latency-sensitive: every 100ms latency increase would reduce the company’s revenue because users lost patience and, as a result, would close the page before it even rendered, losing advertisement revenue. The advertisements displayed on the View Listing page, until now, had been the primary source of revenue for the company. The RE team pushed back against this concern, saying that adding just one more API call wouldn’t hurt the latency much. After all, the RE API war pretty fast, it responded with a p99 of 20 milliseconds, so how bad would that be? The Listing Team, however, did not agree. They were pretty familiar with the Fallacies of Distributed Systems. In addition, Motos and Fashion Teams were also pushing for API integration, which increased the chance of Tail Latency Amplifications, resulting in slower rendering of the View Listing Page.
The negotiations around these concerns dragged on for weeks. At one point, the Listing Team gave in to the demands because the business wanted to remove its dependencies on advertisement revenues by creating specialised categories, and the RE Team played a big role in that. However, the entire planning and negotiation this time took two months.
Similar negotiations also took place between the Search Team and the RE Team. Even though they were run in parallel to the Listing Team’s discussion, it was Summer, and a lot of people from the Search Team were on holiday, so it took a total of three months to come to an agreement with this team. In the end, this is what the final architecture looked like:
When it came to the actual implementation, it did not take more than two weeks to get it done. Bazaarly teams were well-versed in modern software engineering practices. They were really into the DevOps Culture (You Build It You Run It), had fully automated test suites which, when passed, provided the teams with enough confidence that they could deploy the changes to production immediately, had monitoring tools in place to monitor and alert for known-unknowns, and also had tooling to debug unknown-unknowns in production. The actual implementation, hence, was a piece of cake. And indeed, once the feature had gone out, everyone celebrated by ordering lots and lots of delicious cakes!
Next Request: Extend Listing Expiry Date for Real Estate Listings
By default, all listings created in Bazaarly had an expiry date of 30 days. If a seller had to keep the listing active beyond that, they would need to pay money to be allowed to extend the lifecycle. The RE business team wanted to treat their RE sellers more favourably - instead of paying for $20 per listing lifecycle extension, they came up with an idea of a new monthly subscription product, which, when purchased, would allow a RE agent’s listing to have an expiry date of two years, up to a maximum of 20 listings per month. The subscription would cost $250 per month, thus saving the agents $150 per month in extension fees.
The RE business team were sure this would help them to both prevent the RE agent churns they were seeing recently because of intense competition, and at the same time monetise the RE vertical better. As readers can tell, this would require another round of negotiations and planning with the Listing Team.
This time, the negotiations were more intense. The Listing Team was pushing back hard - their logic was that incorporating the RE-specific logic like this in the Listing lifecycle was blurring the domain boundaries between their team’s responsibilities and the RE Team’s responsibilities. The RE Team, on the other hand, argued that keeping all the lifecycle logic in one place and under a single team’s control kept things simpler. They also said that they would be more than happy to expose an endpoint that the Listing Team can call to check if a listing should get an extended lifecycle, and if yes, set the end date to two years in the future. The Listing Team pushed back further - another API call would really worsen the user experience of the View Listing page. The RE Team countered - what if they merge this with the virtual tour API endpoint, and expose an aggregated endpoint for all RE-specific features? The Listing Team pushed back - they mentioned that creating a coarse-grained interface goes against the Interface Segregation Principle, which says that refactoring this type of interface in the future would be a nightmare. And so the discussion continued, and continued, and continued. At one point, the CTO of Bazaarly got involved, and the teams decided to go with one coarse-grained RE API.
Next Request: Monetising Listing Expiry Via Better Subscription Differentiation
A few months had gone by. RE agents had stopped churning for a while, but now a new trend seemed to have started. Smaller RE agents had now started churning, and the business team started researching the cause. After some research, they realised that not too many agents were willing to pay $$$ for a two-year extended lifecycle. Bigger real estate companies loved it as they have the money and a large inventory of properties they would like to advertise. The smaller agents did not find the offering a good value for their money. However, if Bazaarly offered a different product with a lower cost, which allowed a three-month listing lifecycle, they would happily buy it. The RE business team now would like to have it. Which meant - another round of negotiations!
(I left it as an exercise to the reader to figure out what happened next).
Challenging Time Ahead: A New Competitor Comes to Town!
Best Bazaar, a new VC-funded startup, had started taking the market by storm. They were flush with VC money and were spending aggressively to gain market share. Bazaarly felt threatened by Best Bazaar’s growth trajectory - they were disrupting the market left and right. Bazaarly CEO thus decided to initiate a “Code Red - All Hands on Deck!” - all teams must now start delivering features 3x faster than before, otherwise it will be game over for the company.
Questions to Readers
Here are some questions for the patient readers:
What were the major steps needed in this universe to get a feature from ideation to production?
Which teams were needed to do work that the RE Team could not complete themselves to ship features (i.e., virtual tours, extended lifecycle)?
What was the implicit optimisation goal the Listing Team and the RE Team were thinking of when they decided to introduce virtual tour attributes in the Listing domain?
What was the primary cost introduced by the optimisation goal mentioned in question 3, and where did it show up in the feature lead time?
What would happen if Bazaarly started spinning up more specialised category teams? What if they decided to go the other way round - merge verticalized teams back into one? How would these reorganisations affect the feature lead time?
Suppose Bazaarly started using AI tools and, as a result, cut implementation and debugging time by 50%. In addition, they had access to AI-powered monitoring and observability tools, which made understanding complex production systems 50% easier. What would happen to the end-to-end feature lead time in this case?
What if the AI tools increased the speed of writing code by 3x? Would Bazaarly produce 3x more features in the same amount of time?
Coming Next
In the next post, I will present another universe where the architecture looks quite different than what we have seen here. Find the post here.