For years my job was to jump into troubled projects and lead them until fixed. Fixing almost always meant a series of patches and band-aids to turn a horrible and unusable system into something that marginally met the user’s needs. Anything past that was going to take more time than we had, because the root problem always went back to early design decisions. The projects all looked vastly different from the outside, but their problems had a common cause. Again and again, the development teams had tightly coupled the UI and data storage tools, and had distributed the business rules throughout the UI. Often the same rule was written in two, three or more places.
Right about now you might be thinking: “Hey, I’ve done that lots of times and it works just fine!” I know you have, and so have I. This approach is often the exact right way to build small systems. But with complex systems, it will fall apart eventually. If you are creating something that has more than a handful of users or more than a few pieces of data, you are going to need to create a middle tier.
You’ve likely come across this term before. It gets thrown around a lot and can intend different things. Here I mean a group of APIs and microservices that sit between the presentation layer and data storage. But there is a very subtle aspect, often missed, to middle tiers. They are the one place that controls the data. They are not simply a pass-through. The middle tier makes all the decisions when it comes to data.
But I’ve gotten ahead of myself. Let’s back up and start as though we’re still in the design phase. A good starting place for architecting a solution is separating presentation, the storage and data access tools, and the data. This separation is because the goal for each of these is distinctly different. Presentation’s primary focus is the user. Storage and data access tools mostly focus on answering requests as quickly as possible. The data sits between these two domains. It uses storage and access tools to organize and prepare for questions from the presentation layer.
As I alluded to earlier, it is common for these three concepts to get muddled early on in the life of an application. Limited resources and time make it tempting to have the UI make a call directly to the database. As long as the application remains small, there is no apparent penalty for this approach. However, once its functionality grows, release cycles take longer because now business logic must be moved and changed along with the presentation controls. Further, as the size and volume of data grow, the need for more compute power increases for both the UI and database.
Placing a middle tier between the presentation layer and the database solves many of these issues. Microservices are perfect when data needs to be shuffled around behind the scenes but it’s the APIs that bring the real value here. So, let’s segue for a moment and talk about them in more detail. Most people have a view of an API as a means of getting and putting info into a black box. That isn’t wrong, but from an architecture perspective, an API is about control and boundaries. A well-conceived API manages incoming data, preps that data before sending it to be stored, and also optimizes retrieval. The API understands why it is receiving data and the need for that data in future. Another way to say this is that the API encapsulates the business logic. This why middle tiers are often called business tiers.
Having these processes separate from the presentation layer and data access tools provides a single point to address data issues. The natural boundary created by the API methods also allows for significant restructuring without the need to rewrite the presentation layer alongside. If you do make a few design mistakes, they are now much easier to fix.
The how-tos of creating a well-architected API are beyond the scope of this article. But, as you start to create your APIs, remember that the API controls the data, but it does so to optimize the presentation layer. Start by building your middle tier APIs’ endpoints. Those should feel like a natural extension of the presentation layer. Then let those endpoints drive what the API does with data. Organize the data so that the endpoints can answer requests from the presentation layer as fast as possible. The endpoints are where business logic is also embedded. If there are rules about how and when data can be used or shown, the API needs to control that. Doing this means that the logic is grouped in a single location and managed the most efficient way possible.
Often the data for an API will originate from the presentation layer. If that data is seldom written but often read, let the API preprocess that data. Better to do the work once on a write than for each read. If you have data that is coming from outside sources, it probably still makes sense to have the middle tier handle that data. By letting the middle tier lift that data from its source, it can apply the same preprocess and business rules. This way the middle tier is still controlling that data.
One last point: At some point, once you’ve implemented a middle tier, you will be tempted to let some data skip it. Resist, my friend. Now is the time you should allow your obsessive nature to rule out. There are times when it is ok to have the UI and database talk directly, but you will never be able to spot them ahead of time. You’ve spent the time to create a good architecture, so stick with it.
