Foreword – Why standardize?

Aren’t you tired of it already? Aren’t you bored of writing the same integration over and over again, on project after project, for client after client? We even write the same integration over and over again for the same client! One for every application, another set every time we replatform or rebuild one of their applications. It shouldn’t matter if the solution integrates with Contentful today and is being replatformed onto Kontent.ai tomorrow. One system manages content, the other system manages content, and the client doesn’t even want to change the end-user experience. Shouldn’t that be a quick task of migrating the data and changing out connection credentials? I’m so tired of having to rebuild entire portions of the solution or entire integrations just because I want to make what should be a simple change! Every single web-connected solution since the dawn of the web has been built repeating the same integrations again and again! It’s supremely wasteful and entirely unsustainable. Billions, if not trillions, of dollars have been wasted on repetitive development of code and integrations that only had to be rebuilt again and again because no standards were ever produced to provide an alternative way of working for the industry.

It’s well-past time for us to make a change.

In this article, I describe at a high-level my approach, which I refer to as “Requirement Data Model” (RDM) design, for producing API standards, whether for an enterprise at scale or for an entire industry.

Obstacles to Standardization – Your CMS is actually a Commerce Engine. Prove me wrong!

It’s absolutely INCREDIBLE that after over 30 years of development and innovation on the web we don’t have any standards for communication between similar systems, but that’s not even the worst of it: we don’t even have standards defining what different systems are! We label systems as Content Management Systems, Commerce Engines and so on, but these labels don’t actually carry any tangible meaning! In other words, there’s no way for you to objectively prove that Contentstack is not a Commerce Engine, or that Adobe Experience Manager is not a Product Information Management system. There’s no way for you to prove that Commercetools is not a Digital Asset Management System, or that Aprimo DAM isn’t a Customer Data Platform.

Go ahead and scour the web! You’ll find definition after definition of what a Content Management System is, a Digital Experience Platform, a Customer Data Platform, a Product Information Management system, a Commerce Engine, and so on. And yet, not a single one of those definitions is an accepted standard defining the minimum requirements of the respective system type!

Why is this a problem, you ask? Imagine that you want to standardize communication for Content Management Systems. The first thing that you would need to know is what a CMS is; what defines a CMS. Each platform is going to have some of its own unique features and IP, for sure, but you have to know what the minimum requirements are for something to be considered a CMS; you have to know the least common denominators. You would need to know these things because these least common denominators would be the things that you would actually standardize and everything else would remain platform-specific. But what if there are no least common denominators? Everything would be platform-specific and nothing would be standardized, which brings us back to square one!

And so I ask you: what actually defines a CMS or a Commerce Engine or a PIM, and so on? How do you actually know that a given platform is, in fact, a CMS? Is the answer really “because the vendor said it was”? I think we all know that if a platform vendor thought that labeling their platform as an “Eggplant System” would sell more licenses then they would do it, and I’m confident that we can all agree that labeling it as such wouldn’t make it so.

There is no valid answer to what defines one of these systems. It took years of searching for me to accept that. There is no valid answer because the terminology wasn’t defined with the intention of standardizing similar systems, but rather to sell licenses. Terms like “Content Management System” and “Commerce Engine” were created as marketing terms to describe the business problems that a given platform helps to solve, as a means of ensuring that the platform would be findable and considered in a prospect’s procurement process.

Why, then, do we see consistency in the terms? Because customers face similar business problems that relatively similar platforms can each solve in their own way. In fact, the business problems, themselves, are actually more similar than the platforms that address them.

Standardize business needs, not platforms

I spent years trying to find a scalable and maintainable approach that would enable us to standardize similar digital platforms – CMS, PIM, DAM, etc -, but it was a futile endeavor. We cannot standardize similar digital platforms without a definition that minimally describes the requirements that must be met for two platforms to be considered similar. We could build giant canonical data models (CDMs) – something I’m very much opposed to – and end up with a working solution, but it would be one that we could never maintain. As soon as a new platform comes out in a given category, the CDM would have to be reviewed and updated. Platform standardization, without requirements, is infeasible.

With that said, after having spent well over a decade working with digital platforms, what is clear is that clients share similar business needs, and having seen more than my fair share of backlogs over the years I can also say that clients tend to share many of the same requirements at a high-level. For example, just about every client looking for a Content Management System has a requirement to be able to get content out of the system – sometimes to be delivered to a web or mobile experience, a social marketing platform, an aggregation engine and so on. Many share a requirement to be able to push content into the system via an API endpoint called from another system. And so on.

Not every client has every requirement and not every system satisfies every requirement. However, the requirements themselves, at this level of abstraction, are consistent whenever they’re relevant. In other words, every client with a requirement to be able to “get content” shares the exact same “get content” requirement: an API call that returns content in a logical format for some specified subset of the content in the system. Given this, it’s fair to say that at this level abstraction we can reasonably standardize on the requirements, leaving it to platform vendors to decide which requirements they support and clients to decide which requirements they need to subscribe to.

Overview of the Requirement Data Model (RDM) design approach

The Requirements Data Model (RDM) approach, itself, isn’t exactly new. It’s a synthesis of Domain-Driven Design (DDD), API-first, and Black Box Programming that’s been applied to achieve the creation of open standards – the key word being “open”. This is, after all, what API Architects do for a living, only traditionally at relatively smaller scale, focusing in on a specific product, business unit or division, or a single enterprise or organization. The RDM design concept scales this focus up to the industry level.

To use an enterprise as an example, today when an API architect defines the APIs for an organization, they generally try to follow a similar pattern by making APIs that satisfy requirements laid out by consumers. In order to satisfy those requirements, they need to get data from or push data to a set of upstream or downstream systems, respectively, that are obscured to the consumer by the API – a black box approach. To achieve this, the API architect must become a consumer by approaching the product owners for the mentioned set of upstream or downstream systems (platforms) to determine what APIs are available and to request the creation of new platform endpoints, as needed to fill gaps.

This same pattern can be applied similarly at the industry level. However, all necessary roles must be filled and accepted standards for requirements and API schema definition must be created, maintained and enforced at the industry level. The necessary roles are the consumers, the API standards body to own schemas and governance, and the platform product teams.

Following RDM, the OSIRIS Alliance acts as the API standards body, in a role similar to that of the API Architect in the enterprise example. The OSIRIS Alliance defines the standardized, requirement-based API endpoints to be made available to the various consumers and owns the standards for how schemas will be defined and structured. The consumers, themselves, are the client businesses or entities who leverage the standardized platforms via OSIRIS APIs. Meanwhile, the function of the platform product teams is supported by the individual platform vendors in the marketplace.

It’s worth noting that, with the RDM approach, API middleware typically still exists and that a black box approach is still recommended. In cases where a consumer subscribes to multiple systems that satisfy the same requirements, business logic in the API middleware is used to aggregate, calculate or select the appropriate data based on the business’s own needs.

RDM accelerates solution development without hindering platform innovation

It’s critical to understand that wit the RDM approach, it is expected that and appropriate for solutions to leverage custom, non-standardized API endpoints to facilitate more unique requests and logical operations. In other words, each platform vendor may have its own custom APIs that provide unique value capabilities of the specific platform, based on its own IP. This also means that platform vendors have absolute discretion and freedom to innovate and produce new capabilities at their own pace without dependence on the API standards body keeping up with changes to avoid delays in bringing innovation to market. This sits is in stark contrast to canonical data models (CDMs), which would need to be continuously reconciled and updated in order to maintain order and standards compliance.

Additionally, because RDM APIs developed at an industry level are developed at a high level of abstraction, support some custom data is supported within the schemas themselves, to empower platform vendors and solution developers to self-select how the data should be enriched without deviating from the overarching standards. Analogously, this concept is similar to the concept of generalization, or interface implementation, in object-oriented programming, with the schemas being the interfaces and the vendor implementations of the APIs being the class-implementations.

Benefits of the Requirement Data Model design approach

The RDM design approach is not exclusive to the OSIRIS Alliance, and can be followed by any organization or industry containing many similar systems or overlapping systems with similar capabilities, looking to standardize APIs at scale.

There are many benefits of RDM, many of which have far-reaching positive impacts on organizations and operations. Some of the key benefits are highlighted below:

• Accidental complexity and technical debt – read: operational expenditure (OpEx) for maintenance and support effort for API integration work – are greatly reduced, as the repetitive integration complexity is removed, leaving only the custom features and connections needed by the solution.
• Time to market for developing solutions also decreases, also resulting in a corresponding reduction in implementation costs, as integrations are more predictable and thus faster to wire up.
• Connectors and integrations become simpler as well, as a single connector or integration built against the RDM APIs would work for integrating with any platform that supports the requirement or requirements that the connector or integration subscribes to. For example, imagine an integration, named “ContentIntegrator”, was built against the RDM “Get Content” API schema defined by the OSIRIS Alliance’s CDIS. Further imagine that six platforms – Contentstack, Contentful, BloomReach, Kontent.ai, Adobe Experience Manager and Sitecore – all support the CDIS “Get Content” API schema. As a solution developer, you could use “ContentIntegrator” to integrate your solution with any of the six platforms that support the CDIS “Get Content” API.
• Swapping systems becomes much faster and less expensive, as the level of code-reuse increases. In circumstances where custom APIs and data enrichment feature use is minimized, swapping a system could require little to no effort other than pointing the APIs at the new system and supplying new authentication credentials. Continuing with the previous example involving the fictitious “ContentIntegrator” integration, imagine that you’re a solution developer who has used “ContentIntegrator” to integrate Contentstack into your solution. Further imagine that you did not use any custom APIs or data enrichment features of the schema. Now imagine that your organization has decided to replace Contentstack with Kontent.ai. Migration of data between platforms notwithstanding, because Kontent.ai also supports the same CDIS “Get Content” API schema as Contentstack, all you would need to do to update your solution to support Kontent.ai is replace the Contentstack API URL and authentication credentials in the connector with the your new Kontent.ai API URL and authentication credentials.

Collectively, these benefits provide a noticeable decrease in repetitive development costs, maintenance spend and overall OpEx, thus enabling organizations and the industry as a whole to focus spend on Capital Expenditure (CapEx) and innovation instead of on repetitively building the same integrations again and again. The result is a more sustainable, more interoperable and connected web industry that has far more resources at its disposal for innovation and advancement.

Platform vendor responsibilities

Today, there are over 11,000 marketing technology platforms in the world, and that number is continuously growing at an incredible rate. Even if we simply narrow to the couple hundred technology platforms in the digital marketing space, that number too is continuously growing at a similar pace. What this means is that over the long-term there is no viable model for developing and maintaining standardized API endpoints without platform vendors taking accountability for their own implementations of the API endpoints that they choose to support.

This is no different from the model currently used today when defining the API endpoints for an API product within an organization. Look back at the example in the previous section that described how this process works at a high-level within an enterprise and note that the API architect does not participate in the development and maintenance effort necessary to create new platform API endpoints required by the API middleware – that job rests with the platform product teams. Similarly, to be viable with a high degree of scale, the RDM design approach requires each platform vendor to take accountability for their own implementations of the API endpoints that they wish to support.

The role of the OSIRIS Alliance

As previously mentioned, it’s important to note that between the Requirement Data Model design concept is not exclusive to the OSIRIS Alliance, but rather that the OSIRIS Alliance defines and maintains standards that follow RDM. Within this model, OSIRIS defines, maintains and enforces RDM API schemas and the and necessary API governance standards for them. These schemas are them published and made available to platform vendors, who in turn are responsible for implementing the contracts.

Incentive, certification and membership models for the OSIRIS Alliance will be discussed in greater detail in future posts, but for now suffice it to say that appropriate incentive must and will be provided to ensure the support of the platform vendors in the marketplace. This same reality holds true within an enterprise, and is typically satisfied by the structure of the API services model or authority and directives from senior leadership.