Feb 18, 2015
This article was first published on bicorner.com
Data is an asset. There is no question about that. The economics of data are based on the idea that the value of data can be extracted through analytics. However, contrarily to commodities, it is believed that the value of data doesn't grow as a proportion of its volume (1). These assumptions make data a special type of asset. Although this economical view of data is correct, it doesn't account for the fact that increasing volumes and variety of data create more opportunities to extract added value. Being able to capture structured, semi-structured and unstructured data has modified this set of assumptions. Big Data is changing the way analytics were commonly viewed, from data mining to Advanced Analytics.
Surveys conducted in the past 12 months (2) consistently show that 10 to 25% of companies surveyed have managed to successfully implement Big Data initiatives. Also, 50 to 70% have plans to implement or are implementing Big Data initiatives.What is not shown however is the percentage of these that have successfully managed to create value out of data. Primarily because attention has been placed on the technical side of Big Data.
This article doesn’t focus on Big Data from a volume and therefore purely technological point of view. The reason for this is that the Volume part of Big Data isn't what allows extracting the value of data. It allows to store and process data in more efficient ways than before, hence supporting the process of value creation but not delivering it. The key to data value creation is Big Data Analytics.
There is no magic recipe to successfully implement Big Data Analytics in an organisation. It is a combination of skills, people and processes, like it is the case for any project or strategic initiative. There is however a series of challenges that need to be clearly understood or addressed to ensure maximising the chances of success.
Confusing Big Data Analytics and Business Intelligence
Don’t be mistaken, they are related. But they aren’t the same. Big Data Analytics isn’t another BI initiative. It builds upon it, that’s a fact. But thinking that BDA is about transforming data into information through dashboards and reports would be a serious mistake. BDA is about extracting valuable insight from data. It is about empowering decision makers with analytics that will eradicate gut feel decision-making and enable the Data Driven organization of the future.
For years BI vendors have promised organisations they would help them leverage the value of their data through fancy dashboards and reports. BI has been a serious pocket of investment for many companies, many of whom are still trying to achieve the promised return on it. BI is helping organisations access their data in a meaningful way. One can’t deny that. But it is not creating value from data, or maybe just marginal value.
So please, don’t position BDA as another BI investment. It is much more than that. It is actually what BI vendors have always dreamed to achieve. This is therefore why many of those, if not all of them, are proposing Big Data solutions. Whether they are addressing this market adequately is another debate that won’t be discussed here. What’s for sure is that they haven’t fully grasped the importance of the analytics part of Big Data, apart from adding another application to their stack. The good news is that others have which is the reason why we see new companies specialised in analytics entering the market in recent years.
The reason I’m putting this forward is because, unlike BI initiatives, BDA can bring practical results quite rapidly without significant investments. With BDA, the most successful approach is a gradual one: “Start small, Think big”. Start with a limited IT infrastructure, leveraging the open source solutions available on the market. Start with a limited team ensuring you are supported by the right people. Start with a small use case, no need to look for high complexity from day one. “Start small, Think big”.
Overestimating the analytics maturity of the organisation
Not all companies are data driven and not all companies fully comprehend the benefits of analytics. Needless to say that as a consequence, not all companies fully understand what Big Data Analytics is about. Starting with the assumption that everybody is fully knowledgeable with regard to BDA would seriously endanger the success of your BDA initiative.
For this reason, any BDA initiative requires a change management approach which includes an extensive communication effort. Communication is necessary to educate, inform, explain and sell BDA. Even before the start of an implementation, you will need to act as an evangelist, a marketer, a salesman, a lobbyist, a journalist, a teacher and a professor. You will need to spend a considerable amount of your time introducing the concepts of BDA and their benefits to your peers, business stakeholders, management, IT teams and executive sponsors.
A typical organisation will first be very hesitant regarding BDA as the only known thing about it will be the buzz accompanying it. This is normal and nothing to be afraid of. Then comes the start of the journey, the moment when some people in the organisation will make a link to their activity and identify some potential use cases. The most important step is the next one. The one where a business use case has been worked on, delivered and resulted in a meaningful business outcome. This is the moment that an understanding of BDA will start growing within the organisation. Why? Because it will be easier to relate to it. Seeing a concrete example, in your domain of activity, with your data, is what triggers understanding. The rest will depend on each organisation. But supported properly, BDA should then become part of the management and organizational decision-making toolkit.
So, make sure you clearly understand the level of maturity of your organisation when it comes to understanding Big Data Analytics, and take it into consideration as you start on the implementation journey.
Finding the right use cases
Often, Big Data Analytics are presented as “Finding a needle in a haystack”. Although the image is interesting, it can be very misleading. First, it isn’t about looking for a needle but more for a diamond. Then, it is necessary to know not only in which haystack to look for, but also in which farm, which state, which country. The point here is that, if you look for something somewhere, you’ll likely find something. But the value of that something might not be related to your intended business objective.
Too often, companies are lured into thinking that running analytics on a very large set of data is BDA. They think that it is the way to create value from data. Sure enough, they will find correlations. Some might even have a true business meaning, what is referred to as causality. But BDA is not only about correlations. It is far more than that. And if you want to experience the full benefit of BDA, the best way is starting with a concrete and meaningful business use case.
This use case doesn’t have to be excessively complex from the start. On the contrary, start with a simple one. Remember “Start small, Think big”. Also, depending on the level of maturity of your organisation with analytics, starting with a low level of complexity is the safest approach.
Obviously, a low level of complexity for BDA might already mean an expert level of analytics. What is most important here is finding a case with the highest chances of success in terms of business outcome. So, we’re looking here at a use case where:
the variety of data will be limited i.e. ideally, internal structured data,
the data is known by the business users,
the volume of data doesn’t overwhelm your new Big Data infrastructure,
the business has a clear view on the outcome they are seeking,
the business outcome will effectively be useful,
the business is available to spend some time with the BDA team.
From these requirements, it is obvious that the success of your BDA initiative is strongly influenced by the business users. The IT and analytics dimensions can be worked out, one way or another. But the role of the business can’t be eluded. The end product of analytics should be to the main benefit of your business stakeholders. Not only will it bring the most value to the organisation, but it will also secure the long-term positioning of BDA.
Using Agile Big Data Analytics
As previously stated, the role of business stakeholders is key to the success of Big Data Analytics within an organisation. They are key to the development of the initiative in their ability to identify the right use cases and also in delivering successful outcomes. Delivering a BDA use case can be compared to delivering a project. But in this case, the standard waterfall approach to project management should be avoided.
Coming back to the maturity on analytics, you will notice that this maturity also evolves significantly during the lifecycle of a BDA use case. Business users have a general idea of the scope when the analytics process is launched, but they need the BDA team to guide them in defining a detailed scope. Once the data is extracted, the exploration process provides more insights to the users and helps them to further refine their understanding of the desired outcome. Then, in the core analytics phase, the various stages of results allow users to engage with the upcoming outcome.
A traditional project management approach wouldn’t allow for a sufficient level of interactions between the users and the BDA team. Doing so would result in an analytics deliverable that wouldn’t fit business needs. For this reason, applying an Agile project management methodology, SCRUM-like, addresses this weakness.
The aim here is to work in a series of iterations involving business users and the BDA team along the various phases of the analytics process: from scope definition, requirements gathering, data extraction, data exploration, analytics phases, to delivery. Working in small iterations and in close collaboration with the users ensures the delivery of a meaningful business outcome (3). One to which the business users can relate because they will have been involved in the full lifecycle.
Trusting the results of the analytics
Applying agile principles to Big Data Analytics will ensure the engagement of those users involved in the process. In most instances, they will be the first users of the analytics outcome. But in others, users of the analytics outcome won’t be the ones involved in the BDA process. In this case, there is a significant possibility that the level of trust in the analytics outcome will be limited. Why? Because for many, it is new. But mostly because it will be difficult for them to understand how data they know can generate such insights.
To prevent this, the first step is to ensure that the business users involved in the analytics process fully engage with the results and share their engagement with their peers. Communication therefore needs to be promoted through specific interest sessions, presentations, one-to-one discussions, etc.
The process to engage users on the results is the exact same process used in any change management initiative. Communication is key. Communication on the rationale, the approach, the benefits and the details of the analytics process. BDA will appear to many as something new, even though it is using data they have been working with for years. This is normal and shouldn’t be overlooked because the success of the BDA initiative will depend on its adoption by the business users.
Getting the technology right
As mentioned in the introduction, this article isn’t about Technology. This being said, there are a few points that need special attention. The Big Data technology space is still in its infancy which means the number of actors in the market is still quite high and evolving pretty fast. The choice of solutions is very large and can therefore fit many different needs. Obviously, it makes use of all technology options such as SAAS, cloud, virtualisation, mobile, etc.
As a consequence, it can be very easy to get lost in this jungle of possibilities.Best advice? Start small, Think big. Investing massively from the start would be detrimental to your business case.
One note for later: visual analytics. Mark my word. It’s not only a nice graphical trend. It’s the future of analytics.
Finding the right resources and skills
Last but not least, building the right team with the right skills. By now, everybody has heard extensively about Data Scientists (4). It is definitely a great job and the resources aren’t infinite. Although not everybody agrees with the exact definition of their job, there is a common agreement that they should possess some key skills such as statistics, business analysis, communication, creativity and an understanding of BI.
But there are other roles & skills that are equally important to the success of a Big Data Analytics initiative:
Analytics Architect: acting as business analyst, use case lead, assessing re-usability of analytics, determining the analytics roadmap for a specific domain;
Data Engineer: data guru, expert in data warehousing, data extraction, data cleansing;
Graphic Designer: not yet mainstream but will soon be in the BDA field. Those profiles will allow leveraging the full capacity of visual analytics derived through BDA.
Those profiles are not common profiles to be found easily. It is therefore advised to leverage existing resources to fill some of these roles where possible.
Obviously, sponsoring, steering and managing this team is essential. But these roles don’t ensure the delivery. So, focus your attention on the ones who will effectively do the job.
As stated in the introduction, there is no magic recipe to successfully implement Big Data Analytics in an organisation. Addressing the 7 challenges highlighted in this article form a strong basis to start any implementation. What they don’t address is whether or not your organisation is ready to begin the Big Data Analytics journey.
So, are you ready?
Sr. Analytics Architect/Sr. Mgr at Electronic Arts (EA)
well said Laurent...you've articulated the key points very much effectively..engaging the business partners at the early stage is an effective way for any BDA intiatives...