Integration "connectors" "middleware" and "API's"

Integration "connectors" "middleware" and "API's"

Probably a far from exhaustive assessment, but as there is really no such thing a simple list of “common integration API’s” because integration compatibility is far more complex than that, my review is necessarily long-winded.

Legend

Blue text is common integration terminology

Green text and lines marked as …

       are some of the more common integration “connectors”, “adapters”, “API’s” or "types" of integration you may want to consider for your requirement set.

Integration occurs usually in one of three common ways:

·         At the Data Level

Backend data stores of the relevant application are integrated, and can be either push or pull based, often using SQL calls on another applications database tables.

When the application that needs to be integrated does not provide any APIs or client interfaces, you would use data-level integration.

Typical interfaces here might include

       SQL

       CSV/TSV

       Custom ETL solutions

·         Through API’s

Considered the best way forward for application integration, and it uses the integrated application’s integration frameworks and APIs.

It is good to use, since it is transparent to the integrated application and it preserves the application’s data integrity.

API examples include:

       Siebel's Java DataBeans (JDB)

       SAP's JCA (J2EE Connector Architecture).

       Remote Procedure Call (RPC)

       Distributed COM (DCOM)

·         Via Client Interfaces

In cases where direct access to the database is not easy or possible, or when the business logic is embedded in the user interface, this is the correct integration method to use.

Mainframe and client/server applications are often good candidates for this.

Used as a last resort.

Typical interfaces here might include

       Screen scraping

       Scripting

       Exports to CSV/TSV

       Print to PDF

Enterprise Application Integration (EAI), aka “middleware,” provides the infrastructure to connect information sources, acting as a go-between for applications and their business processes.

At its core, EAI is focused on allowing multiple disparate applications to work together, sharing data, processes, and other critical functionality across the enterprise.

At first, enterprise software vendors did not include connectors to integrate their products with other packages, creating a market for independent software developers to introduce EAI middleware to facilitate this integration.

There are three major challenges to catering for Integration:

·         There is a dichotomy of modern application architectures, with developers following Sun’s Java 2 Enterprise Edition (J2EE) and Microsoft’s Dot Net (.NET)

·         There is a trend within IT to purchase “packaged” applications to satisfy niche business requirements

·         The development of wrappers for legacy applications, which may have no integration interface at all, is required to facilitate an integrated architecture

When envisioning an overall EAI solution, an organization must investigate and focus on the overall business integration needs and goals as they relate to both internal and external constituents.

In doing so, the organization will need to view integration from application-to-application (A2A) and business-to-business (B2B) perspectives.

An A2A integration approach has a more narrow internal focus based on its goal of creating a single enterprise model that comprises all of the technology assets within the boundaries of the enterprise

The A2A approach typically includes a common application architecture and security model, enabling a uniform approach to application development and integration.

In addition, A2A integration incorporates centrally controlled access schemas, again creating a common operation and support model for all applications within the organization.

These common elements, policies, and procedures can be developed and enforced efficiently, as all applications in questions are controlled by the organization itself.

With B2B integration, developing a single technology model is usually impossible due to the fact that multiple organizations, each with their own technology model, are involved in the overall architecture.

In designing B2B technology architectures, organizations must differentiate between public and private components.

This requirement along with inter-organization security and tightly controlled information and application access schemas are imperative to developing a secure inter-enterprise architecture.

Additionally, parties included in any B2B endeavor should include within their internal architectures support for common protocols.

Making up the foundation of the EAI functional model is Core Middleware.

Middleware represents the plumbing that allows multiple methods of application-to-application communications.

Core middleware techniques and methods can be incorporated directly into the applications that need to communicate with one another.

Types of Integration Middleware

Database Access

The most primitive and widely implemented manifestation of EAI is database integration.

This type of integration provides access to multiple, disparate, geographically separated databases.

Database access middleware may also provide a common API, allowing multiple applications to utilize the same data access methods.

Examples of database access middleware include:

       Open Data-base Connectivity (ODBC) and

       Java Database Connectivity (JDBC).

Database protocols include COM+/DCOM, CORBA, EDI, JavaRMI, and XML

Message Oriented Middleware

In its basic form, messaging allows two entities to communicate by sending and receiving messages.

In general, Message Oriented Middleware (MOM) products pass in-formation in a message from one application to one or more other applications.

These solutions can be configured in a point-to-point or multipoint topology such as a publish-and-subscribe model and provide fault tolerance and delivery guarantee.

The introduction of message queues allows the producer and consumer to work at different paces. For example, a purchase order generated by an inventory system can be placed on the queue

that is served by a vendor’s fulfillment system even if the fulfillment system is down for backup.

Another common component of MOM based solutions is an Application Program Interface (API) that abstracts the network implementation of the application from the developer.

IBM and TIBCO have popular message oriented middleware solutions on the market, both using proprietary components to accomplish similar tasks.

·         Websphere MQ (IBM)

·         Enterprise Messaging Service or Renezvous (TIBCO)  

       However, the Java Message Service (JMS) has been introduced as a specification within the J2EE framework.

JMS is an API for messaging that addresses many features typically found in stand-alone message oriented middleware products.

·         Microsoft’s Message Queue (MSMQ)

Transaction Processing Middleware

Born out of the mainframe application model, Transaction Processing middleware (AKA “TP Monitor”) handles resource management and transaction coordination in a distributed system.

ISO & ITU-T DTP standard (ISO TP)

Distributed Transaction Processing (DTP)

CORBA Object Transaction Service (OTS)

Java Transaction Suite (JTS)

Web Services define another approach aimed at solving the same fundamental problems outlined in this paper.

Web Services expand on this idea by doing away with middle-tier message brokers and instead exposing application functionality as services.

These services are made available from other applications for synchronous invocation, similar to API calls.

In order to achieve this, an organization must focus on creating a service-oriented architecture (SOA).

An SOA is simple in concept, but complex in reality.

Fundamentally, SOAs are focused on wrapping existing applications with a well-defined interface that transforms the application into a set of services accessible by other applications.

In order for an SOA to meet this goal it must address a few key components.

These components include:

·         A common, universal interface language used to wrap the applications

·         A ubiquitous and universal information transport mechanism

·         Flexible, evolving interfaces

·         Standard mechanisms to search for, locate, and execute services

By defining standards based on XML, web services are well suited to meet the overall goals of a SOA.

Distributed Object Middleware

These standards include:

       SOAP

       WSDL

       UDDI

When used in the context of web development, an API is typically defined as a set of Hypertext Transfer Protocol (HTTP) request messages, along with a definition of the structure of response messages, which is usually in an

       Extensible Markup Language (XML) or

       JavaScript Object Notation (JSON) format.

While "web API" historically has been virtually synonymous for web service, the recent trend (so-called Web 2.0) has been moving away from Simple Object Access Protocol (SOAP) based web services and service-oriented architecture (SOA) towards more direct

       representational state transfer (REST)

Integration Brokers

Integration brokers, also known as message brokers, act like a combination of a postal service and a translation service within application communication channels.

Integration brokers  support legacy EDI data formats and XML data exchange schemas such as:

       Rosetta NET

       cXML

       BizTalk

       XEDI

       ebXM

Application Servers

by supporting the Java Connector Architecture (JCA), application servers provide a common model upon which connectors may be built that allow connectivity and integration into any JCA supporting application or system

       Database - JDBC

       Messaging - JMS

       Transaction Processing - JTS

       RPC - RMI

       DOF - .NET

       DOF – EJB

XML

Although the EAI market is in a fragmented state, some emerging standards are beginning to drive towards a common plat-form capable of true enterprise-wide application integration.

XML is a standards-based language that deals solely with physical data and its structure.

XML provides a common data exchange format allowing disparate applications to exchange information without each application knowing inherent logic and data types utilized by each participant application.

One critical factor in the use of XML-based data exchange is the adoption of standard vocabularies to be used for all XML messages.

As with many standards-based approaches, organizations will have to make some decisions when choosing the appropriate vocabulary.

In particular, competing technology providers will continue to create separate business-specific XML vocabularies that alone will not be inherently compatible.

Organizations must choose one standard, or incur the costs of designing systems that can use multiple standards.

XML alone cannot solve all enterprise integration issues.

Connectors/Adapters

Most enterprises turn to one of three types of connectors:

·         SAP Web Services,

·         vendor-supplied enterprise service buses (ESBs) and, increasingly,

·         cloud connector and data integrator products.

o    Mule ESB

o    Boomi

Note: Boomi would not work for enterprises that need to do on-premises extract, transform and load (ETL) in the magnitude of terabytes per hour

… to avoid tracking and managing point-solution connectors the likes of:

SAP

JIRA

ORACLE

LinkedIn

Hadoop

Twitter

Eloqua

Workday

Amazon RedShift

Microsoft Dynamics CRM

Chatter

NetSuite

Zuora

MongoDB

Oracle CRM On Demand

Twitter

SharePoint

Concur

Teradata

Microsoft Dynamics AX

Microsoft Great Plains

Workday-NetSuite

Salsa Connector

NetSuite OpenAir

… and a thousand others some of which can be seen on the TIBCO Supported Adapters page