Why Is Something So Good, But So Hard To Do.....Big Data

By Patrick Pang, Head of Solution Consulting, JOS

Patrick Pang, Head of Solution Consulting, JOS

Big Data; When people see this buzzword, they see business opportunities to expand and make profit. Digital transformation has become a top priority for businesses as they understand the long-term benefits of Big Data. Hadoop, HDFS, Machine Learning; we are so familiar with these terms. However, people have begun to realize how difficult it is to make effective use of the data they possess.

The common problem is that they have no clue on where to start.

From a local system integrator’s perspective, I would like to share the current challenges of big data that are predominant in the IT industry. Gathering data has become easy but being able to derive new market insights from massive amount of raw data is challenging. Here are some pain points to consider when dealing with big data: (1) Finding the Right People (2) Setting Up the Infrastructure and (3) Encountering Knowledge Barriers.

Finding the Right People

It is crucial to find people who are familiar with utilizing Big Data. According to our research, we need to wait at least two years for students studying under Big Data-related programs in Hong Kong universities to become Big Data experts. For instance, hiring a Hadoop administrator could be a headache as Hadoop was established in 2006, meaning that it does not carry a very long history.

"Organizations that seek to rely on Big Data must be aware that it is not easy to justify the Return on Investment, in other words understanding the monetary benefits received from implementing Big Data.”

Setting up the Infrastructure

Even if we have the right people, setting up a solid infrastructure like Hadoop, is another challenge as it requires a lot of resource implementation and maintenance support. Organizations that seek to rely on Big Data must be aware that it is not easy to justify the Return on Investment, in other words understanding the monetary benefits received from implementing Big Data.

Encountering Knowledge Barrier

Since there is a shortage of Big Data talents in the market, organizations are experiencing a limit to their ability on dealing with Big Data challenges. Due to the lack of expertise and resources, market participants are starting to encounter knowledge barrier. Thereby, there has been an increasing demand among enterprises for a collaborative platform where they can conduct analytics in one single place. In previous years, this may have been impossible, but now JOS has the solution that can resolve this knowledge barrier – Bamboo.

A Way to Start Big Data in a Fast Track

In order to capture useful information from raw data, we got to import it, conduct analytics jobs and create visualized results. With Bamboo, a Big Data solution provider, these three business intelligence tools are all merged in one platform which can minimize time and cost of deployment.

In the banking industry, many banks put a lot of effort in making use of Big Data to achieve better surveillance such as anti-money laundering. However, the process of scaling out and increasing the accuracy is becoming another major issue for them as data volume goes up.

With Bamboo, business analysts at banks could easily create their own hypotheses of what they want to track down from the aggregated data and collaborate with data scientists who can execute the hypotheses by creating a workflow. This simple platform offers a stepping stone of building a digital channel for organizations to adopt and fill in the performance gaps caused by the Big Data revolution.

In no doubt, Big Data has significantly become vital to businesses in aspect of creating a competitive edge to keep up with the constantly changing market trend. We can witness that companies are becoming more data-driven by the increasing demand for data scientists. In regards to the pain points of dealing with Big Data, we must start leveraging the available technologies and identifying the right partnership so that we achieve the synergy effect of maximizing our business opportunities with Big Data.