Salesforce Integration Methods When and Why? Part 2

Continuing our series of blog posts on Salesforce Integration Methods: When and Why?, which focuses on various aspects of CRM system integration, let's examine middleware in more detail. 

What is Middleware?  

Middleware is software that lies between two systems, allowing communication, data exchange, and synchronization between them. Middleware acts as a bridge connecting Salesforce with external systems within the Salesforce ecosystem. In the education sector, middleware often integrates Salesforce with enterprise applications such as Google Workspace or Microsoft 365 for managing student accounts, accounting systems for syncing financial data, attendance tracking systems for importing and exporting student attendance records, and SEVIS for handling international student visa and exchange program requests. (insert SEVIS blog link?), and more.   

What does middleware do?  

Middleware, in the context of a Salesforce integration, is responsible for a few key operations. One is Event Handling. Event handling involves the collection of any published events (Platform Events in Salesforce), managing the distribution of those events to any endpoints, and ultimately taking any appropriate action in response to receiving a published event. From the Salesforce side, event publishing can be accomplished via the event bus or a Flow that works by publishing Platform Events or creating new Platform Event records. On the middleware side, event handling relies on an active listener that 'listens' for Platform events from Salesforce and can then perform any appropriate action depending on the specifics of the event received, including forwarding those events to any subscribers, performing additional integration actions such as querying Salesforce, distribution of information to an external system(s), or creating records. Event handling becomes particularly useful when you need to control when an integration kicks off by having an event listener act as the start of an integration process or action. In this way, we can publish platform events from Salesforce, and an integration process can trigger as soon as the middleware platform receives that event.  

Middleware is also helpful for data transformation and translation. Transformation is mapping or changing data from one format to another while maintaining data integrity and ensuring interoperability between whichever systems you are trying to integrate. Commonly, the transformation and translation process involves taking in data in one format from one system, be it CSV data, for example, and transforming it to match a native format for another system, such as JSON or XML. This is extremely useful when integrating two systems that take in or output different data formats, which is almost always experienced in any complex integration. Although middleware isn't explicitly required to perform transformations due to things like Dataweave 2.0 in Apex, in Salesforce's words, "...Transformation of data can be coded in Apex, but we don't recommend it due to maintenance and performance considerations."  

Middleware platforms also offer centralized monitoring capabilities, providing tools to oversee the health of all integrations from a single dashboard. Although a smaller volume of less complex integrations can be monitored through debug logs in Apex or custom objects in Salesforce, middleware platforms such as Mulesoft allow for a central platform to check the health status of multiple integrations at once in an organized and efficient manner. You can think of this as a special dashboard that allows you to see what each integration is doing, if it is running correctly, see any errors as they come through, and, with some middleware platforms, allow you to make some changes to the integration directly from the user interface. Middleware platforms are useful for real-time monitoring of integrations as they might display the status of an integration or any latency, for example. Additionally, they may provide automated alerts for any failed integration transactions or system outages. Finally, these platforms are often built with logging in mind and, as such, allow for detailed logs to help debug issues specific to an integration, which may likely be faster than sifting through Salesforce debug logs at large.  

When do I need to use middleware?  

You may have noticed that the listed middleware functionalities above aren't always required to write a solid integration. However, there are use cases in which middleware may be the best choice to connect your external systems with Salesforce.   

Complexity and Transformations  

If the integration you are writing is simple, say integrating a single system without any or minimal data transformation, built-in Salesforce tools such as Apex and/or Flows can suffice. However, if you have multiple external systems, heavy data transformations, or on-premise systems that are not exposed via an API, you will likely need to implement the integration with the help of middleware.   

Data Volume and Throughput  

As a multitenant solution, Salesforce enforces strict governor limits across the platform to ensure that resources are allocated fairly and efficiently across all of its tenants. This ensures stability and performance across all organs running on the Salesforce platform. However, this can cause some headaches when dealing with a very large volume of data. For example, Salesforce has limits on Apex CPU time and heap size. Complex workflows or large datasets can potentially exceed these limits, warranting the use of middleware that can help offload intensive processing and data transformations to the middleware layer.