API SOLUTIONS: Big Data & Data Silos

{api:dab}: Bringing the power of APIs to micro, small and medium-sized enterprises

FACT: Humankind generates roughly 3 quintillion bytes of data every day...

Big Data Landscape

In many respects, data drives the modern commercial world, with your business generating more data today than it ever has before. For example, in 2017 the global community generated 2.51 quintillion bytes of data every single day, and that is set to grow exponentially in the coming decades.

The breadth and depth of data available to our business today is huge. Data analytics can provide meaningful insight into the smallest of detail or allow high-level overviews to paint the vista of your commercial landscape. A better understanding of how your organisation works can let you streamline, automating tasks to reduce costs and improve efficiency. So the more vectors you have to collect your data, the better, right? Wrong! Data silos — the result of relying on different platforms and applications to generate, collect, and store data — don’t help your efforts. They actively hinder them.

Having detailed information about your business processes is good, but having that data stored in separate places (or data silos) means you have multiple fragmented pictures of your operation. You get glimpses of the wider picture, but no cohesive overall vision. Let's look at a few of the most common ways data silos are hampering the ability of organisations to make data-driven decisions and changes and what an API strategy can do to mitigate:


Data Siloed Between Departments

This is a common way for data silos to emerge. Most small-to-medium sized businesses are made up of different departments and each of these departments wil have its own needs, goals, and priorities. No problem so far - that is a standard operational model. But, the data problems arise when those departments all procure and use separate software solutions, based upon their own particular commercial needs.

For example, staff on the operations side of your business might be dealing with production, while the sales team interacts directly with customers, with the accounting staff processing the numbers without any direct link to either. If each of these units chooses software specific to their department, there will be no way to generate a complete picture of the organization without manually integrating all those data silos together, or relying on the frequent export of the data, to a central repository.


Internet of Things (IoT)

Data from IoT devices can contain some deep and interesting client metrics, but much of this information is not currently being leveraged by the suppliers of the devices. When you consider that the majority of data produced by and for businesses is created outside of a traditional data centre environment:

  • Video Surveillance
  • Output of IoT Devices
  • User Generated Documents
  • Mobile Devices (Chat etc)
  • Payment (EFT) Devices

We can see the depth and breadth of the potentially available information. This is set to increase as businesses go through their technology growth and adoption cycles.


Manually De-Siloing Introduces Potential Problems for your Data

It is likely that your senior management are already aware of (and frustrated by) the data silos in your business. It may well be the case that you have developed a strategy for manually combining data from different sources to obtain the full picture of your operation - but there is a problem with this approach:

Every time a member of staff has to manually interact with a data source, even to move it from one system to another, objectivity decreases, cost increases and potential for error is introduced. It is impossible to have a single source of truth for your business if you’re working with data silos and a single source of truth is something that every business would benefit from. These could include quite granular customer touch-points (company email subscription, support call frequency) - Centralised data gives you real insight into trends occurring within your business and your industry. Regardless of what business sector you are engaged in, it can be very helpful to eliminate data silos if you want to leverage the power of the information you’re generating and collecting.


How Can {api:dab} APIs Help?

The {api:dab} API development platform allows non-IT technical staff to quickly define and deploy with full CRUD functionality. This results in a framework allowing disparate systems/devices, both internally or externally, to write or display data for the benefit of a central data warehousing and management information system, in real-time. Most modern platforms will support API posts and where legacy systems are concerned, we have the fall-back of automated export jobs from the host system, with automatic imports to the API system, via our dedicated SFTP workflow server.

{api:dab}: Publish data via API in minutes!