Back to News Archive

A beginner’s guide to IT system integration. Part 1: P2P, ESB and API-led connectivity

Posted on: August 9, 2019

Why Integration?

An inevitable step towards business growth

Nowadays, every business essentially becomes an IT business; with a plenitude of SaaS, mobile, IoT and other technologies even traditional industries, such as manufacturing, hospitality, automotive, logistics go through a digital transformation and build new business models around IT. Regardless of the industry, enterprises end up owning a great number of systems: either built internally or acquired from third parties; cloud-based or on-premise. According to the latest MuleSoft Connectivity Benchmark report, a big international enterprise on average has 900 different applications in use.

When choosing a new system, one of the obvious criteria is how can it serve a particular business purpose, or department; and very little thought is usually given to how this system will fit within an existing digital landscape across the enterprise. However eventually, when the necessity for business transactions across an enterprise emerge, IT teams across departments have to find a way to make their apps start “talking” to one another, and system integration becomes necessary. If your organisation has more than 10 such systems (let alone 900!) – this exercise would be quite a challenge.

Although each company has unique processes and challenges, in Ricston’s experience, there are several universal integration requirements that can be found in most of the businesses, regardless of their industry.

system integration illustration


4 most common integration requirements and use-cases

1. Information Portals

When more than one system has to “answer” a query or perform a business function & information from multiple systems has to be aggregated into one view.

To illustrate what is meant better, let’s take a look at an example use case of Ricston’s client in the Digital Media industry.

Example

Our client wanted to get a better understanding of their income, commissions and forecasted revenues for their online digital advertising and in order to do so, they had to aggregate information from different sources. Their data was distributed across multiple applications:

  • Supply Side Platforms to manage their advertising space inventory
  • Opportunities data in Salesforce CRM
  • Proposals, orders and ad-units in Google DoubleClick for Publishers (currently Google Ad Manager)

To achieve the goal, Ricston designed an integration solution to migrate the data to a common warehouse on a daily basis, where it was aggregated and consumed by an analytics tool (such as Tableau, or QlikView and QlikSense) to produce necessary reports and forecasts for the company.

2. Data Replication

When two different systems have to remain synchronised with respect to their data.

For example, another Ricston’s client, a leading online fashion store, had a requirement to automate the synchronisation of customers’ and orders’ data in their eCommerce systems.

Example

The Client used two different business management systems to manage sales & order requests:

  • Enterprise resource planning (ERP) software NetSuite &
  • E-commerce platform Magento

The process of synchronizing sales orders between the two systems was very time consuming when done manually. The client was scaling the business rapidly and wished to have an automated process that would sync information from one system to another while reconciling any data conflicts that might arise throughout the process.

In this case, our consultant has built an integration that ensured synchronization of data flows between NetSuite and Magento at predefined intervals on specific days without any human intervention.

3. Distributed Business Processes

When a single business transaction may involve steps affecting different systems & the transaction have to be orchestrated by some other “agent”.

One of our clients in the FinTech industry, for instance, was looking to unlock their global data and business processes for all company entities across the globe to use. One of such business processes was “revenue & cost management”, that involved connecting multiple financial applications into one Source-to-Pay system.

Example

In order to optimise their business revenue & cost management process and reporting the company decided to integrate their global cloud-based business spend management system – Coupa with their major ERP software – SAP and other backend systems and audiences into one Source-to-Pay system (S2P). This business process then was made available to all other regional branches via API’s.

This allowed replacing manual processes that due to business expansion couldn’t be maintained.

4. Business to Business

When a business needs to expose a certain functionality to 3rd parties (clients) & (or) consume data from partners.

This is a very common scenario, for instance whenever you are adding a car hire to your flight ticket – a b2b integration is in play. Ricston has helped to build a similar system for the leading premium transport service provider in the UK.

Example

The Client wanted to design a solution which would give their customers, partners and affiliates the opportunity to integrate with their cab booking system.

Our consultants have implemented the requisite integration logic and this enables Client’s partners (hotels and flight aggregators) to offer a taxi service as part of their journey planner. This has drastically increased the company’s addressable market.


The above are just 4 examples out of more than 400 different integration projects we have implemented so far. But while each business use case is specific, from the technical point of view, there are 3 main approaches to IT system integration.

Let us dive into tech and show the “evolution” of enterprise systems integration.



Evolution of enterprise systems integration

Integration yesterday: Point-to-Point

The first solution that comes to mind in order to connect systems and apps is a point-to-point (P2P) integration – connecting A to B, or A to C. The main idea behind it is that each pair of applications or systems that need to communicate must have a tailor-made connection designed specifically for handling data transformation and other messaging related services between them.

system integration - P2P illustration

While the P2P model is being used extensively today as well, it only works well with very small infrastructures, where two or three systems need to be integrated. However, every time when additional components are added to an infrastructure, the number of unique connections increases drastically. Since every pair of apps or systems require custom coding and maintenance – the IT architecture becomes complex and stiff.

Eventually, P2P practises lead to scalability problems, such as:

  • Tight coupling – replacing one of the systems would affect all the others that are communicating with it.
  • Code bloat – every pair of systems would need to have its own connection which requires to have engineers specialising in all the connected technologies.
  • No separation of concerns – every application would need to know all the details about its pair app, thus every update within the app itself would require more updates of its P2P connections to follow.

To sum up, using P2P for more than a few apps will result in an IT maze built on a ‘spaghetti’ type of code, which is almost impossible to debug and maintain.

system integration - P2P integration complexity

It doesn’t take long to recognise the downsides and limitations of the P2P approach, and there is an alternative solution – an Enterprise Service Bus (ESB).

Integration today: ESB

The core concept of the ESB architecture is to integrate different applications by putting a bus-like component between them and enable each application to talk to each other through that bus. It allows to decouple systems from each other: they can communicate consistently without being dependent, or knowing much about other systems on the bus. Moreover, you won’t have to repeatedly code the same things over and over again.

In a nutshell, having the ESB as the backbone of an IT infrastructure allows:

  • Reducing the code bloat
  • Loose coupling (applications no longer need to know about one another)
  • Being platform-neutral (applications need to know only how to communicate with the ESB)

system integration - ESB illustration

And there are more benefits at play here.

Routing – an application does not need to know what the target is

Integration logic takes care of the messages’ routing to the right system. So if you need to disconnect a target system and put a new one, you just adjust the routing logic and your original application is not affected.

Transformation – an application does not need to convert data formats

Without the transformation component, every system will be concerned with the format which the target system will use. But having the transformation capability on the ESB allows sending the data in any native format.

Filtering – an application does not need to perform validation

The ESB can also have a validation component. So if you have messages which are “illegal” for the target system, the ESB will filter them out.

Asynchronous Processing – the source and target do not need to be “alive” at the same time

A target system might not always be ready to receive your messages. So you might need to queue them somehow and implement a new mechanism each and every time. With asynchronous processing, the messages are being stored until the target system is ready to consume them in its own time.

Transactional Processing – transactions across different systems

This feature is very helpful in a situation where a single transaction affects multiple systems. In case something goes wrong, it allows you to roll back.

Although the ESB approach has a strong reputation with a proven record of handling various enterprise architecture requirements, it is in the very nature of technology to keep constantly improving and adapting to the ever-changing business needs.

The gap between what business is asking IT teams to deliver and what can actually be accomplished – keeps widening. Therefore modern businesses have to focus not just on delivering projects to service immediate needs, but on building a composable IT architecture within their organisation that supports both current and future requirements.

That’s where an API led connectivity approach emerges. An API (application programming interface) isn’t a new technology as such, however, the application of an API-led connectivity approach to software architecture is innovative. It moves far beyond the idea of P2P and ESB architecture and keeps its main focus on reusability.

Integration tomorrow: API-led connectivity

According to MuleSoft’s methodology – API-led connectivity is a methodical way to connect data to applications through reusable and purposeful APIs. It enables enterprises to build an internal Applications Network.

system integration - API-led connectivity illustration

API-led connectivity moves beyond the idea of P2P and traditional ESB architecture – instead, you build layers of API’s that are responsible for certain systems, processes and end consumer experiences.

It is realised by introducing layers of abstraction and control between the mission-critical back-end systems, enterprise systems that contain business data and the front-end exposed to developers, unlocking corporate data and assets.

The key point about the API led approach is that you are designing your systems in a reusable manner upfront; this speeds up the app launch and modification cycles, the release of new products to market, makes it easier to secure and manage access, and ultimately enable companies to do more and faster with less.


We will discuss the topic of APIs and API-led connectivity in more detail in the 2nd part of the article. If you wish to receive more updates like this in the future, please subscribe to our monthly newsletter (in the footer).


Learn more:

  • The concept of modern API’s. Watch video
  • API-led Connectivity with MuleSoft’s Anypoint Platform. Read article
  • Only 36% of IT leaders report that their organisation offers a completely connected customer experience. Read article

Contact Us

Ricston Ltd.
Triq G.F. Agius De Soldanis,
Birkirkara, BKR 4850,
Malta
MT: +356 2133 4457
UK: +44 (0)2071935107

Send Ryan a message

Ryan Delia
Business Development Manager