Top 10 big data and analytics predictions for 2017

IT leaders forecast big data and analytics trends. What to watch? Spending on tools increases, data jobs evolve, and entire industries are disrupted as companies compete using advanced analytics.

You know the saying: There are three things that are certain in life: death, taxes, and change. Change is mandatory, but progress is optional. If there’s another thing that is certain in 2017, it’s that more data will be generated, which means more opportunities for more data to be analyzed. Other predictions involve how teams will work together, and the underlying technology that will improve productivity and drive businesses forward.

Things to watch, in no particular order:

1. Tool investment grows

An increasing number of enterprises will invest in analytics tools, while others will hone their efforts further and dive into the next generation of analytics, doing more niche data analysis. In short, data analytics will become more sophisticated, and enterprises will make more actionable business decisions.

IDC forecasts worldwide revenue from big data and business analytics will reach $187 billion in 2019, more than $55 billion of that from software. "Organizations able to take advantage of the new generation of business analytics solutions can leverage digital transformation to adapt to disruptive changes and to create competitive differentiation in their markets," says Dan Vesset, group vice president, analytics and information management, at IDC. "These organizations don't just automate existing processes—they treat data and information as they would any valued asset by using a focused approach to extracting and developing the value and utility of information."

Forward-thinking businesses will conduct descriptive analytics, which interpret events that have already happened. Or they will go a step further and deploy predictive analytics, which tells them what will happen if they stay the course. Prescriptive analytics will tell executive teams what steps they can and should take to improve key performance indicators. Behavioral analytics will provide the ability to target customers’ wants and offer them the right products and services. 

2. Collaborative analytics takes hold

Data analytics is a team sport, and successful data science initiatives leverage high-performance team collaboration, says James Kobielus, big data evangelist at IBM, in a blog post. Furthermore, the best collaboration environments enable data science professionals to flexibly share feedback, guidance, ideas, models, requests, samples, and templates on diverse projects. Static PDFs and PowerPoints will eventually be replaced, and a team’s ability to stay connected to data through data-driven alerts, share relevant data and best practices, and build upon one another’s work will drive businesses forward.

3. Analytics becomes decentralized

As the next generation of data analytics kicks in, centralized analytics will be replaced by a decentralized approach to help meet more specific business needs, according to research and advisory firm International Institute for Analytics (IIA). While a certain level of central coordination is useful, when central analytics groups grow large, they might become targets of budget cuts and other pressures, which could drive decentralization, the IIA's "2017 Analytics Predictions and Priorities" report says. 

4. More sophisticated platforms emerge

The IIA also expects organizations to adopt analytics platforms with capabilities to start them on the path toward deploying artificial intelligence (AI) as the next progression of analytics. Forrester concurs, noting in its 2017 customer predictions report that AI investments will triple as firms begin tapping into complex systems, advanced analytics, and machine learning technology.

5. Analytics move to the cloud

For all the compelling reasons organizations are moving processes to the cloud, observers expect—and advise—enterprises to look to cloud platforms for conducting analytics in 2017. Both open source and proprietary analytics methods are becoming more readily available, the IIA report says, resulting in cloud vendors becoming "serious competitors" to traditional analytics firms. “Cloud platforms such as AWS and Azure are also providing robust analytics capabilities, from basic reporting to advanced optimization and machine learning, that make analytics in the cloud a viable solution for both exploratory work and production work,’’ the IIA report states.

6. CDO role gains prominence

While the analytics function continues to be a work in progress in some instances, more organizations are expected to formalize the role of analytics professionals. “In 2017, we can expect the role of the chief data officer (CDO) to move from ‘bad guy’ in the enterprise to be the steward of significant initiatives," says Bruno Aziza, chief marketing officer (CMO) at Hadoop tool maker AtScale, in an interview. As more organizations bring in CDOs, the role will mature as they overcome some of the initial challenges they faced when it was new to the enterprise, he predicts. "We can expect CDOs to enable access.”

The trend toward CDOs gaining more prominence in 2017 “represents a much deeper change occurring throughout most organizations,’’ observes Ted Friedman, a research vice president and distinguished analyst at Gartner. Analytics practitioners need to work more closely with others to realize the benefits of using data and analytics to capture transformative business opportunities and mitigate risks, Friedman adds.

7. CMOs must evolve

CMOs have traditionally held the keys to the data analytics castle. But if they don’t evolve, they could be out. For some, that evolution will be a tall order, as they will be tasked with both designing more satisfying customer experiences, while also mastering technology and analytics to deliver contextually rich experiences, according to Forrester's 2017 customer predictions report. 

Some brand-focused CMOs have been uncomfortable taking on the full mantle of technology, analytics, and customer experience (CX) programs, the report states. Likewise, analytics-focused CMOs have been uncomfortable developing engaging customer experiences. Maintaining the status quo is risky. “We predict that CEOs will exit at least 30 percent of their CMOs for not mustering the blended skill set needed to drive digital business transformation, design exceptional personalized experiences, and propel growth,” according to the Forrester report.

8. Data analytics go vertical

Five industries accounted for nearly 50 percent of worldwide big data and business analytics investments in 2016, and will remain on top through 2020, according to IDC. Specific verticals that are driving the growth are banking, discrete manufacturing, process manufacturing, federal/central government, and professional services. “Within banking, many of these efforts are focused on risk management, fraud prevention, and compliance-related activities," says Jessica Goepfert, program director for customer insights and analysis.

“Data analytics will go vertical, and companies that build vertical solutions will dominate the market, Ihab Ilyas, professor of computer science at the University of Waterloo, and co-founder of Tamr, predicts in a Forbes article. “General-purpose data analytics companies will start disappearing. Vertical data analytics start-ups will develop their own full-stack solutions to data collection, preparation, and analytics.”

9. Data scientist role evolves

The well-documented shortage of data scientists and analysts skilled at statistical and predictive analytics will continue in most industries. IDC predicts that by 2018, businesses in the U.S. will need 181,000 people with deep analytical skills and five times that number of people with data management and interpretation skills. The data scientist job also becomes increasingly complex as they work within integrated, multidisciplinary cloud-based development environments, as Kobielus describes in his post on the next-generation data scientist.

10. Hadoop no longer dominates big data

Hadoop will remain a mainstay of unstructured data acquisition, transformation, cleansing, and queryable archiving. But when it comes to some of Hadoop’s core components, such as MapReduce for massively parallel data processing and Hadoop Distributed File System (HDFS) for data storage, Kobielus and others believe the trend will be toward using Apache Spark and distributed object storage. Kobielus states that the trend is "most visible in IBM’s Watson Data Platform (WDP), which uses Apache Spark instead of MapReduce and distributed object storage in lieu of HDFS." While WDP and other next-generation cloud data platforms will source data from Hadoop clusters, Spark is the centerpiece of the new cloud data services platform, he adds.

Several technologies were developed in tandem with the big data wave to fulfill the need for analytics on Hadoop, says Dan Kogan, director of product marketing at Tableau Software, a company that produces interactive data visualization products. But enterprises that have complex, heterogeneous IT environments will require both Hadoop and non-Hadoop sources to find answers to questions buried in sources ranging from systems of record to cloud warehouses, to structured and unstructured data. “In 2017, customers will demand analytics on all data," he says. "Platforms that are data- and source-agnostic will thrive, while those that are purpose-built for Hadoop and fail to deploy across use cases will fall by the wayside. The exit of Platfora serves as an early indicator of this trend.”

“Prior to 2017, machine learning and visualization tools to analyze large amounts of data haven't really existed,’’ observes David Barton, analytics division head at Innovation Enterprise. “Now, they are improving in quality and the cost is decreasing, so it is expected more will be deployed and more companies will make unstructured data a higher priority.”

What’s the bottom line when it comes to analytics in the new year?

Develop comprehensive data analytics techniques and bring in advanced tools to more finely analyze the data being produced. If you’re not doing data analytics yet, start. Take action. Don’t find yourself behind the analytics eight-ball. “The journey toward digital business is, at its core, a drive to better collect, manage, and exploit data assets and apply analytics for richer insights,’’ says Gartner’s Friedman. “By 2018, Gartner predicts that over half of large organizations will compete using advanced analytics and proprietary algorithms, disrupting entire industries.”

Data and analytics: Lessons for leaders

  • Companies will compete using advanced analytics and proprietary algorithms, disrupting entire industries.
  • Cloud platforms will provide robust analytics capabilities, making the cloud a viable option for exploratory and production work.
  • High-performance team collaboration will be critical to successful data science initiatives.
  • Analytics professionals’ roles will become more formalized. CDOs will be more prominent, and CMOs will be pushed to evolve.
  • Data scientists' jobs will become increasingly complex.