As enterprise looks up to solving their complexities in dealing with problems and challenges, Big Data techniques are increasingly getting popular and as experts opine, Big Data is surely here to stay
In today’s complex world of enterprise IT, the biggest challenge comes in the form of demystifying complex things with appropriate usage of IT, coming as a saviour in such a scenario, Big Data and its related applications are helping the enterprise community to get better results out of their existing data loads and create more value.
Primarily driven by the exponential evolution of the Web, today’s Big Data requirement is redefining the way information is being generated and stored as well. This is prompting a lot of enterprise-class applicationsto find a unique way to handle big amounts of data and generate informed decision making intelligence capabilities.
Barun Lala, Director Storage, HP India feels, “Today, devices which generate data are everywhere, for example: RFID sensor networks; social data, telecom related data, scientific data, medical records and data, surveillance data; photography and videos and e-commerce data. This is resulting in data sets whose size is larger than before and beyond the ability of currently available tools to capture, manage and process. While this may be a complex environment, this also presents organizations opportunities to derive significant value out of the existing data. This value that can be derived out of this data through real time analysis is Big Data.”
Driving their profitability ahead, most enterprises today want to find out more about the relevance of Big Data and the value-add it brings to their organization from a business perspective.
For decades, companies have been making business decisions based on traditional and transactional data stored in relational databases. Beyond that critical data, however, is a potential treasure trove of non-traditional, less/multi structured data like weblogs, social media, email, sensors, device logs and images that can be mined for useful information. This is exactly where big data comes into play.
Sheshagiri Anegondi, Vice President, Technology, Oracle India opines, “There is a strong latent demand for Big Data and Analytics platforms in the Indian market. The reason I say that is because most companies have their business processes online today. They have the ERP systems and so forth. But increasingly, you have a lot more data that’s come online that hadn’t been available for analysis before. You have social interactions, location data, and other types of sensor data, and this is all now coming within the grasp of enterprises to start to analyze.”
“In addition, enterprises are looking for the right tools to handle Big Data and to derive real business value from big data. They need to overcome various technical hurdles around scalability, around the complexity of the data and around the velocity or rate at which the data is coming from the sensors and managing data latency. They also have the challenges around some of the softer issues such as the quality, privacy and security of the data. We’re working to solve all these technical requirements,” Anegondi added.
As far as the enterprise community is concerned, today most of the data available is in an unstructured format coming from different sources at great speed. According to industry analysts, only 5% of information created in the world is structured. The remaining 95% is multi-structured data and is increasing at an exponential rate. In such a situation, organizations want to do deeper and more sophisticated analysis of this data because when this multi-structured Big Data is distilled and analysed, and meshed with traditional enterprise data, organizations can develop a more thorough and insightful understanding of distinct aspects of their business.
However, the exploration in data volume is affecting not only the big enterprises but small organisations as well. Anegondi believes, “It’s pertinent for SMEs, especially e-commerce companies to adopt Big Data and Analytics to maintain a competitive edge in the struggling ecosystem of mid segment market. For example, analyzing inventory data from a smart vending machine in combination with the events calendar for the venue in which the vending machine is located, will dictate the optimal product mix and replenishment schedule for the vending machine. But SMEs have budgetary constraints and resource crunch and hence they have to be extra careful while implementing Big Data. They should follow an enterprise architecture approach and properly align and prioritize big data implementation with the business drivers. They should look at big data investment as an extension to their existing information architecture.”
Big Data Adoptions
Globally Big Data is being adopted across all verticals, and for every vertical the use case is different, the solutions are different. Verticals like BFSI, Telecom, Retail, Healthcare, Media, and government sectors are at the forefront of adopting Big Data in India owing to the enormous data flow in these sectors and the wealth of opportunities the technology provides for these businesses.
Says Lala, “HP provides customers with a powerful solution for managing big data, especially when considering the volume and types of data generated by businesses in today’s information economy. By unifying technologies from acquisitions of Autonomy and Vertica, along with HP’s strong portfolio of Information Optimization solutions, HP is enabling enterprises to harness maximum information, generating insight, intuition and ideas to drive growth.”
The relevance of Big Data is cutting across industry verticals and organizations size. So, for small web 2.0 companies having a small volume of data but having the need to do real time analysis of the data, Big Data would be important. Similarly for a large telcos, retailers or BFSI organizations, there is huge amount of data lying in their system. If they are equipped with the right tools to do real time analysis, this gives them an excellent opportunity to understand their customer and operations far better.
Oracle plays an important role in the Big Data market with its end-to-end solutions to address the full spectrum of enterprise Big Data requirements. “In continuation with our strategy to deliver pre-integrated, pre-tested engineered systems for data management requirements, we have an engineered system called the Oracle Big Data Appliance. It combines optimized hardware with the most comprehensive software stack featuring specialized solutions developed by Oracle to deliver a complete, easy-to-deploy solution for acquiring, organizing and analyzing big data. It is designed to deliver extreme analytics on all data types, with enterprise-class performance, availability, manageability and security,” informs Anegondi.
On the other hand, HP Cloud provides the underlying infrastructure required to process big data. “We partner with third party solution providers who enable enterprises to better configure, manage, manipulate, and analyze their data affordably. For structured and unstructured data, using HP products and also using open source line Hadoop, Hive, Pig, Flume, etc.,” chuckles Lala.
The future of Big Data
With data related analytics and insights coming in to play in almost all the big decision making capabilities for the enterprise community, experts close to the industry do feel that big data holds great potential in the coming years to comes.
With huge opportunity lying in the market, analysts are predicting 30%-40% y-o-y growth in this space. Today enterprises are not only looking for storing data but also wants to know how to get the best result out of it.
Big Data can be defined as:
1) The speed at which information flows into these primary online systems
2) The number of customers a company must deal with
3) The acceptable interval between the time that data first enters a system, and its transformation into information that can be analyzed to make key business decisions
4) The kind of data that needs to be handled and tracked
Also working well within the enterprise-class requirements, big data techniques work within the following mentioned parameters.
1. Velocity – defines how fast the data is coming into the system.
2. Variety – it comprises of all types of data that are being captured that is – structured, semi-structured, unstructured data as well.
3. Volume – this is the potential of terabytes to petabytes of space that data will be taking to get stored.
4. Complexity – this involves everything from moving operational data into big data platforms and the difficulty in managing the data across multiple data redundancy sites and geographies.