Oracle has released an intriguing study that includes a survey of utility executives who collectively say they are not doing so well when it comes to Big Data – i.e., the challenge of making sense of large volumes of complex data that can be transformed to help improve business operations and customer service.  In fact, the utility executives rank themselves among the least prepared to handle the data deluge, something we at Pike Research have noted as a major challenge for the industry as it transforms itself with smart grid technologies.

Here is the relevant scorecard, according to Oracle’s study:

• Public sector (government), health care and utility industries are the least prepared, with 41% of public sector executives, 40% of health care executives, and 39% of utility executives giving themselves a grade of “D” or “F” for preparedness
• Overall, 29% of the C-level executives surveyed from 11 different industries in North America give their firms a “D” or “F” for preparedness
• By contrast, communications industry executives claim to be the most prepared to handle the data deluge, with 20% giving themselves an “A”

Practically all the executives surveyed are concerned about the near- to mid-term data challenges, with 97% saying their firms must improve their use of data over the next two years.  Thus utilities are not alone, just behind the curve compared to others.

So why are utilities flunking out?  One reason is a lack of experience.  Few utilities have ever seen such a huge volume of data generated from utility processes.  In the past, utilities read a meter once a month and sent a bill.  That’s roughly 12 data points per year per meter.  With the latest technology, that volume expands exponentially, with the potential number of meter reads increasing to more than 35,000 per year per meter (based on 15-minute intervals).  That’s an increase of 291,900%!

And that’s just the meter read.  Some advanced meters also send and receive other data related to pricing, pre-paid options, load control, tamper detection, and temperature, which can amount to hundreds of additional attributes that must be processed and correlated.  In addition, the grid itself is being outfitted with a plethora of new two-way communicating devices that report their measurements even more frequently than most smart meters.  These additional types of data create new challenges that many utilities are just now trying to comprehend, and leverage to their benefit and that of their customers.

Moreover, the utility industry lacks skilled data analytics experts.  With just about everyone across the corporate spectrum seeking these people, utilities have to compete for scarce talent.  Most people with these skills are gobbled up by sexier industries, like communications or web-based businesses.  As a result, utilities are turning to third-party vendors for help with data analytics, and that will help ease the burden for the near-term.  However, utilities that want to leverage the data in their own way and perhaps save the cost of outsourcing will need to bring this expertise in-house.

The data deluge is a growing, long-term issue that will test utilities for years to come.  The message from the Oracle survey does have an upside, however: executives know they are failing, which is the first step in making the necessary changes to improving their performance.

The Edison Foundation announced in May, 2012 that as many as one in three households in the United States now have smart meters. Yet, utilities stand to know even less than they did before about some of their operations once they deploy an advanced metering infrastructure.  That is, if they don’t find a way to turn smart meter data into actionable information.  With electro-mechanical meter reading, meter readers do more for the utility than just read consumption values off of the meters; they also look for signs of meter tampering and fraud, and inspect the meters and utility assets for their condition.  Without these eyes in the field, utilities may experience problems identifying theft and knowing when other field assets may be in need of repair.  Smart grid data analytics can help address these problems, but given the volume and velocity of the incoming data, utilities must first define the business challenges they are trying to overcome.  Getting and storing the data is not the issue; making sense of it is.

In a field whose practitioners are said to be “reading the electronic tea leaves,“ many vendors focus on technology solutions to solve the smart grid big data problems.  And perhaps rightly so, as in the discipline of data analytics, issues like data quality, usefulness of the algorithms, scalability, and performance are key factors to deployment success.  However, in choosing the right solution, successful utilities consider not only their most pressing issues like meter-to-cash operations, demand response and customer service, but also how to leverage their smart grid to introduce operational efficiencies, such as voltage optimization and asset protection.

Figuring out how to do this effectively requires defining the business problems where a data analytics solution can deliver a promising return on investment (ROI).  Not all solutions will.

Smart grid data analytics (stand by for a Pike Research update of this report in 2012) is part of the process towards achieving a fully optimized utility in a modernized electricity delivery framework.  The optimized utility has learned how to leverage data analytics for meter operations, grid optimization, asset control, and renewables integration for the benefit of both the utility and the consumer who desires to save money or use energy more efficiently.  Vendors may offer a variety of strategies towards achieving these goals, including managed services and applications, enterprise applications, platforms, and visualization tools.  It’s likely that not one solution will fit all and a flexible approach – achieved through partnerships and open systems – will be the most powerful.

As smart meters and other grid sensors make their way at a rapid pace into the field, a strategic, business-focused, viewpoint from utility stakeholders will go a long way toward ensuring solid decisions that support the evolutionary nature of the smart grid and all the information that can be gleaned from it.