Big Data seems to be generating a lot of hype for the past couple of years. Millions of dollars of VC investments in Big Data companies, lots of PR & content floating around about how Big data will change the face of your market. However, not everything you hear about this industry may be accurate. In fact, some of it, if accepted without evaluation, can even hurt your business.
1. Big Data has the answer to all your business questions
This is one of the most common coffee table conversations around Big Data. Big Data is certainly helpful but it won’t give you the answer to every business question. It may potentially help you find answers to complicated and cryptic business questions but surely not everything. Things which can be done using an Excel Spreadsheet or Databases, should be done on a spreadsheet or using a database. Most of the times, you’ll find that small data, applied properly, can help you find answers much sooner than Big Data. So, it’s important to set your expectations about what you’ll get from a Big Data Strategy.
2. Only a Data Scientist can handle Big Data
Every now and then, we come across people who claim to be data scientists who are driving the Big Data Strategy in their company. I’m sure they are doing an important job of delivering insights from data, but Big Data works with or without Data Scientists. It begins with the Developer who wrote the application to capture your data, goes all the way to the data analyst who built the analytics & reporting framework, to the Business Executive who reviews the report or presentation and takes action. Data Scientists simply help steer the insights from data, most of their work being done by intuitive, automated and well-documented tools & applications. Most companies make the mistake of waiting to hire a data scientist before they even look at their data. So, don’t wait on data scientists before you start prepping for Big Data.
3. Big Data is rocket science
The growing paychecks of Data Scientists might make you think that Big Data is complicated and only a few people with highly specific skill sets can understand & tame it. Big Data is just more volume, velocity and variety of data. It need not be complicated. In fact, you’ll find that the business rules that apply to Small data also apply to Big Data; business rules are always independent of the size of the data. In fact, a well-designed big-data system is often simple, scalable and fast.
4. The more data you have the better
Data is only as valuable as the insights it provides. There is a debate on how much data is effective and whether more data is better. One school of thought suggests that more data you have the more you could learn from it. However, the need for Big Data heavily depends on the nature & size of your business. For example, an industry like online advertising would have a lot of data as businesses need to record each user click, every second. Similarly, a big e-commerce business like Amazon would have tons of data because of the sheer number of daily users and activity on their site. However, if your business doesn’t qualify these criteria, you need to seriously ask if you need Big Data, in the first place. It’s always better to store only as much data as needed; keep gathering data till you start gaining applicable insights from it, not more.
5. Big Data is all hype
Big Data is getting a lot of media coverage and PR time. It is partly because there is hype, but partly because the big data tools have shown the capability to handle unmanageable amount of data and derive insights using commodity hardware. The hype is due to the misconception that any business that builds technology to analyze lots of data will blaze away with ground breaking insights. This is based on the mistaken premise that more data equals more insights. In reality, Big data is not just hype but a real shift of technological capabilities on how businesses start to look at their data.
6. Big Data is unstructured
If you have been into big data domain, you must have heard the opinion that Big Data is unstructured data. It is not true. As mentioned earlier, big data is just data that has more Volume, Velocity and Variety. It could be structured or unstructured, it’s just bigger, accumulates a lot faster and arrives from various types of sources.
7. Data eliminates uncertainty
Data surely helps convey more information about a business question but it won’t pin point you directly to the right answer and eliminate decision-making. It still requires you to evaluate your options, make tradeoffs and come up with the best way to move forward. Future data is as uncertain as the market condition. Uncertainty can be due to various uncontrollable areas, competitive landscape, customer experience, market conditions, and other business dependent conditions. It’s important to remember that Data doesn’t eliminate uncertainty but surely minimizes it.
8. Big Data systems are expensive to build and maintain
This is one of the most common misconceptions among businesses. It is one of the major hurdles to adoption of Big Data among SMEs. Typically, large enterprises try to ride the Big Data wave by gathering as much as data as they can and investing heavily on storage technologies to accommodate all the data they’ve gathered. They wait while they figure out how to use all this data. SMEs can’t afford this approach and therefore tend to stay away from Big Data. Businesses need to understand that Big data is popular mainly because commodity hardware could now be used to tackle big data. In fact, you don’t even have to build or own storage solutions for it. Cloud Services like Amazon’s EC2 allow you to create your own analytic sandboxes and pay as you use it. You can easily fire up an environment to conduct the initial analysis and turn it off when you’re done, paying only for the time you’ve used it. Big data systems are not expensive anymore, their cost has been falling every year. So, cost should never be a deterrent for indulging in big data project.
9. Big Data is for Big Corporations Only
Unfortunately, this misconception is a side effect of the previous one. Big data tools are inexpensive. They are accessible to not only big corporations but also Small/Mid size companies. SMEs can easily dabble with Big Data without spending a fortune. Leverage Cloud to create simple data experiments. If your initial experiment is successful, you can scale up your environment to handle the full load of Big Data. When your project is completed, you can simply turn it off. This way, the total cost for your project goes down drastically. Also, you can use a fail-fast model to explore many possibilities for your analysis before deciding to fund an initiative.
Big Data landscape is filled with a lot of Myths, so make sure you try it out yourself, experiment with a little bit of “Big Data” before you make up your mind.