The big data trend continues, and more and more companies are hopping on the bandwagon. While many organizations assume they need big data’s wisdom, often the “small” operational data they already have will do just fine.
Operational data is internal data, such as the data that gives Uber its ability to dispatch cars. Big data, in comparison, is information collected in high volume and at high velocity. It’s occasionally collected internally, but purchasing it remains a more common practice.
If you don’t truly need big data, embedded business intelligence company Exago explains why you may regret pursuing it: “The trouble is, big data is notoriously difficult to wrangle on account of its size and complexity. Setting aside for the moment that many enterprises have to purchase access to big data they don’t produce themselves, the process of grooming that data for reporting and analysis can be prohibitively expensive.”
In addition, just because a company purchases fancy new analytics tools and huge volumes of data to go along with them doesn’t mean they have a clue about how to extract the pearls of insight from the oysters. Mining data for actionable information requires attentive management, accurate analysis, and continuous adjustment, and buying software and raw data doesn’t provide companies with the skills necessary to master these processes overnight.
This year, the way organizations gather and use data has been under something of a microscope. Facebook has taken most of the heat, but others have been scrutinized as well. Despite this criticism, big data does still offer invaluable insights for some situations and challenges — what you need to figure out is whether yours are among them.
If you’re thinking about investing in big data for your organization, take the following considerations into account to ensure you’re truly working toward the goal you seek.
1. Needing a lot of data doesn’t always mean you need big data.
Before jumping in, make sure the problem you’re trying to solve or the goal you’re hoping to achieve actually requires big data rather than just a lot of data. As Jim Gallo, national director of business analytics at ICC, explains. “Just because you have a lot of data doesn’t mean it should be considered big data.” Although the term seems to emphasize volume over anything else, “big data” actually just describes a quantity of data that requires new tools to process it. Typically, big data utilizes multiple physical or virtual machines working together.
If you’re merely storing and retrieving large volumes of files in and from a data warehouse, you’re facing a different kind of challenge. Huge data sets are an issue that many organizations have been dealing with for years, and the quantity of data by no means indicates a “big data” problem.
2. Even with big data, operational data remains critical.
It’s a common misconception that organizations must choose between big data and small operational data. In fact, a complete big data solution could depend on combining them.
Big data is most commonly used retrospectively, and analytical big data technologies such as Hadoop can generate valuable insights after data has been collected. However, operational big data systems are still responsible for importing and storing data via real-time workloads. Incorporating both types of data will ensure your data efforts produce the most effective results.
3. The payout from big data requires big changes.
The hype surrounding big data has inflated expectations, in many cases well beyond what’s reasonable. Gaining a competitive advantage from big data can also require enormous changes that are impossible or impractical for many organizations to make. For instance, big data helped a retailer see that by keeping items on the showroom floor for a longer period, both before and after discounting them, it could increase its profits significantly. Unfortunately, this change had far-reaching supply chain implications, and the company was unable to put it into practice.
The insights generated through big data analytics can be easy to replicate, so it’s possible that consultants in your industry might already provide the services you’re looking to glean from big data. Be sure to do your homework before you spend the money on a big data initiative.
Although big data is everywhere you look, it may not actually be the right solution for your organization. Big data can be insightful, but these insights are distilled after the data has been collected and analyzed. Ultimately, before you go chasing big data, you might want to focus on better using the operational data you already have. Even if you end up needing big data after all, you’ll be better prepared for it after you get a handle on your in-house data.
