Finding the Key to Big Data Success

By RP Siegel
Credit: photoexplorer via

There is an old saying, “Give a man a hammer and soon everything looks like a nail.” The man soon finds satisfaction in all that his hammer can do, and, impressed with its capabilities, he begins seeking out opportunities to extend its usefulness. The same can be said for information technology.

Surely, we have all been impressed with all that IT can accomplish. But, when it comes to using IT in what is surely its most ambitious undertaking, Big Data projects, companies shouldn’t let the IT department do the driving, according to a number of experts who presented at last month’s Data Warehouse Institute (TDWI) World Conference. That would be like allowing the carpenters to design the house.

That's not to say that the role of the IT department is insignificant. But it should be a supporting role, according to Ed Silva, worldwide practice lead with HP's Information Management and Analytics group. Thomas Rathburn, a senior consultant with The Modeling Agency, added that Big Data, analytics, and predictive modeling are business performance topics. Technology can help enhance business performance but it's just a means to an end.

It’s really a classic example of a cross-functional team, with all of the potential that involves, and all of the challenges associated with having all of that specialized knowledge around the table, much of which is not shared across the enterprise. It requires trust and a willingness to give up control in those areas where others know more.

But is it a self-directed team, or is there a leader? Chris Mills, director of engineering at ADP Dealer Services Co., told TDWI Conference attendees that business owners need a clear vision of the objectives that the development team can get excited about.

IT experts may well have elegant and sophisticated implementation schemes that truly make the most of the equipment being used. But it may not be justified if it doesn’t serve the ultimate business vision and solve the problem that project was developed to address. If the development can be done without being overly rigid, allowing room for all team members to be creative, that’s even better. But it will likely need a firm hand on the tiller to keep the project on course.

Mills spoke in terms of agile development. “We work directly with the product owner,” he said. “The closer the connection to the business, the better we're able to do our jobs.”

Being agile is the ability to adjust business goals as market conditions and customer needs change.

Silva shared a story about a customer who said, “I'm looking for a golf lesson, but the software vendors are always trying to sell me a driver.” Companies that are selling Big Data and analytics tools need to learn to speak the language of their customers. Business heads don’t want to be consumers. They want to be partners.

At the end of the day, most businesses want to find meaning in the data, which isn’t always easy. It takes skill, good tools, and a keen sense on the part of the provider of knowing what the customer, or the project leader, is looking for.

Data services company Acquia offers examples of Big Data projects on its website. Among them are:

  • Consumer product companies and retail organizations monitoring social media like Facebook and Twitter to get an unprecedented view into customer behavior, preferences, and product perception.
  • Manufacturers monitoring vibration data from their equipment, which changes slightly as it wears down, to predict the optimal time to replace or maintain.
  • Manufacturers monitoring social networks to detect aftermarket support issues before a warranty failure becomes publicly detrimental.
  • Sports teams using data for tracking ticket sales and even for tracking team strategies.

The desired outcomes for companies using Big Data are often expressed in terms of velocity and value. They want to be able to quickly convert raw data into meaningful information, and they want to be able to refine the data into useful and actionable insights.

Ahmed Noor wrote in Mechanical Engineering about Raytheon’s use of Big Data in its manufacturing execution software that monitors the number of turns given to a screw in a critical adjustment for a missile. If the screw is not given the correct number of turns, an error message flashes to alert the assembler. He went on to describe how quality control information from across the factory is being integrated towards the eventual goal of zero-incident, zero-emission, zero-defect manufacturing.

Noor described the development of cybernetic assistants using tools like predictive search to sort through Raytheon's continuous blizzard of data for a few special snowflakes or a revelatory pattern. One example of these kinds of tools is Mindmeld by Expect Labs, a smartphone app that listens to phone conversations and searches for material on the Internet based on its eavesdropping capability. I think it’s safe to say that we can expect to see more applications of this sort in the future.

It could be that the moral of the story is not so much the data after all but the way it is presented. That was the takeaway for Taylor Erickson after spending years as a consultant specializing in fixing failed SAP implementations. He told conference attendees that customers were generally unimpressed with vanilla sales reports, no matter how good the data might have been.

It was only when he turned to an iPad app by Roambi Analytics that takes data from any data source and transforms it into visual and easy-to-understand insights that he saw his customers’ faces light up. Now, Erickson said, no matter how complex the underlying data is, the user experience needs to be intuitive and engaging. That could be easier said than done. But the ability to perform that kind of data comprehension, consolidation, and presentation, could be the knowledge engineering of the future.

So when a business starts its next Big Data project and begins assembling a team, it needs to make sure there is a visionary leader who is grounded in the realities of the business, the voice of the customer, the IT people, and someone who can perform the kind of data wizardry that brings smiles to stakeholders’ faces.

Post a Comment

comments powered by Disqus

Newsletter Signup


From Around The Web Recent