Close this search box.

We are creating some awesome events for you. Kindly bear with us.

Sharing real-time relevant data with utility consumers – Saving costs while helping the planet

Sharing real-time relevant data with utility consumers - Saving costs while helping the planet

Changing consumer behavior with timely relevant information
and the right incentives has the potential to significantly reduce electricity
consumption, leading to a win-win situation for everyone- cost savings for
consumers, helping utility companies offset the intensifying energy demands on
their aging infrastructures and reducing environmental impact.

Opower (acquired
by Oracle in 2016) is a leading provider of customer engagement and energy
efficiency cloud services to utilities. Its clients numbering over 85, include eight
of the US’ 10 largest utilities. The company’s consumer engagement platform
which pairs behavioral science with big data analytics reaches more than 18
million homes across North America, Europe, and Asia.

The platform presents insights into household energy use
through informative dashboards, alerts, incentives, and household comparisons.
The information delivered through mail, email, text messages, and web generates
heightened awareness of the energy consumption and impact on consumers’ wallets
and thereby incentivises consumers to reduce energy consumption.

For instance, customers alter their behavior to use energy
during off-peak times, swap out older and inefficient appliances, and anticipate
factors such as weather in order to save energy and money.

However, achieving these benefits is not easy.

Dealing with escalating
data volumes

The primary focus area for utility companies is their
physical infrastructure. Managing the data that is collected from that
infrastructure is secondary.

Questions such as “How do I manage my flow of electricity? How
do I make sure the lights are on?” are what utilities are concerned about,
noted Eric Chang, Opower’s technical lead for data services at the time (as of 2013).

Advanced metering infrastructure (AMI) technologies such as
smart meters, thermostats, and other in-home devices lead to an
escalation in data volume, further complicating the situation. These devices generate
and send providers frequent voltage and temperature readings. For example, a
smart meter may generate 100 readings per day, per household. When that is multiplied
by thousands of customer households for just one utility provider, the volumes
of data are enormous. Data size is further compounded by related data streams
including demographics data, smart appliances and sensors, weather data,
consumer behavior information, and social media data. Utilities are usually not
equipped to manage these kinds of volumes of data.

Data is only valuable if is processed and analysed. Opower
recognised that processing and analysing this information would provide the
utilities with much-needed insights into customer’s energy consumption that
could be used to drive energy savings, reduce demand at peak times, adopt new
rate structures, support smart meter roll outs, and lower cost to serve.

Shifting to a
scalable data platform

As utilities are unable to cope with the vast volumes of
smart meter data, Opower’s service model seeks to deliver that scalability,
with the cost scaling based on use.

Opower’s first data management platform was a MySQL open
source relational database management system (RDBMS). It was adequate with a
small number of utilities clients.

As Opower’s client base and consumption data grew and its
products offered more functionality, the company needed to adopt a new
architecture –one that could scale to process and store large amounts of data
without the cost and operational overhead of its relational databases.

Mr Chang said, “For us that new architecture included Apache
Hadoop and [Apache] HBase.” In 2011, the company began moving to a distributed
services-oriented architecture.

Opower deployed Cloudera
as the enterprise data hub to expand its big data capabilities
and to store, transform, and query time series and other data. Key components
of Opower’s architecture include CDH
and HBase.
Apache Hive serves as Opower’s data

The enterprise data hub improved processing speed and
efficiency, and the ability to easily pull data in parallel from multiple
sources. In addition, HBase enabled Opower to deliver insights at scale, in
real time, to any user.

HBase serves as a highly scalable data store that is built
on top of Hadoop and offers fast, random read/write access to users and
applications. Raw energy consumption data is batched regularly and imported
into HBase, analytics are performed, and the results are stored there.

Immediate consumers of the data stored in HBase include web
front ends that directly forward queries from end users and other internal
processes, for example, processes that generate customer emails or paper mailings.

Opower manages its Hadoop cluster through Cloudera Manager.

“Cloudera Manager provides a lot of tools that enable us to
know exactly what the cluster is doing at a given time,” said Tim Luo, a former software engineer (as of 2013) on the Opower data services team. The tool allows Opower’s
team to respond to alerts quickly, manage and tweak configurations, restart
servers, and ensure that overall cluster health is good.

Chang explains the closed loop system that makes it easy for
Opower to correlate the impact of those efforts.

“The utilities provide us with meter reads, we perform
analysis, we target a certain number of customers to deliver that analysis to,
the customers act on that information and change their behavior, the utilities provide
new meter reads, we compare those to the original reads, and we measure the actual
impact in energy savings in that targeted population.”

Minute changes in the behavior of individual consumers,
leading to say two to three percent energy savings, can add up and make a big
difference when they are driven across several millions of households.


Cost savings

The company has helped its utility partners save energy
consumers more than US$320 million in utility bills – and generated nearly
three terawatt hours in energy savings. That is enough energy to power every
household in Salt Lake City and St. Louis for an entire year.

Improved sustainability

Today, Opower is using big data to engage customers and
motivate them to save energy and money. Increasingly, utilities will use big
data to fuel insights that will drive new opportunities for innovation and
efficiency in their energy delivery.

Opower does not only influence consumer energy savings; it
imposes a mindfulness on itself.

The Hadoop platform infrastructure allows Opower to be much
smarter about how it utilise its resources across the platform. Opower also
worked closely with Dell on acquisition of hardware to ensure lower power
consumption drives.

Continuous innovation

Opower continuously evolves its products so that as devices,
communication channels and preferences, and customer behaviors change, its clients
can respond effectively. Opower is constantly innovating, measuring the
outcomes, and using that information to improve the platform.

Cloudera plays an important role in enabling this continuous

A key advantage of Cloudera is its open source aspect and
the new functionalities which come out of that. Cloudera focuses not just on understanding
the software, but understanding how to use it and how to interact with the open
source community to get the most value from it.

Mr Drew Hylbert, Opower’s former vice president of
technology and infrastructure (as of 2013), said about Cloudera, “They are an excellent
partner both in keeping the systems going and delivering new technology on top
of HBase. As we continue to scale, we know we can rely on Cloudera to help us understand
how to grow – the impact on our cluster, how we need to provision, and what configurations
need to be tweaked to accommodate that additional scale.”

Content from Cloudera customer success story, dated November 2013, on  


Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private company, Qlik offers real-time data integration and analytics solutions, powered by Qlik Cloud, to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik serves more than 38,000 active customers in over 100 countries.


As a Titanium Black Partner of Dell Technologies, CTC Global Singapore boasts unparalleled access to resources.

Established in 1972, we bring 52 years of experience to the table, solidifying our position as a leading IT solutions provider in Singapore. With over 300 qualified IT professionals, we are dedicated to delivering integrated solutions that empower your organization in key areas such as Automation & AI, Cyber Security, App Modernization & Data Analytics, Enterprise Cloud Infrastructure, Workplace Modernization and Professional Services.

Renowned for our consulting expertise and delivering expert IT solutions, CTC Global Singapore has become the preferred IT outsourcing partner for businesses across Singapore.


Planview has one mission: to build the future of connected work. Our solutions enable organizations to connect the business from ideas to impact, empowering companies to accelerate the achievement of what matters most. Planview’s full spectrum of Portfolio Management and Work Management solutions creates an organizational focus on the strategic outcomes that matter and empowers teams to deliver their best work, no matter how they work. The comprehensive Planview platform and enterprise success model enables customers to deliver innovative, competitive products, services, and customer experiences. Headquartered in Austin, Texas, with locations around the world, Planview has more than 1,300 employees supporting 4,500 customers and 2.6 million users worldwide. For more information, visit


SIRIM is a premier industrial research and technology organisation in Malaysia, wholly-owned by the Minister​ of Finance Incorporated. With over forty years of experience and expertise, SIRIM is mandated as the machinery for research and technology development, and the national champion of quality. SIRIM has always played a major role in the development of the country’s private sector. By tapping into our expertise and knowledge base, we focus on developing new technologies and improvements in the manufacturing, technology and services sectors. We nurture Small Medium Enterprises (SME) growth with solutions for technology penetration and upgrading, making it an ideal technology partner for SMEs.


HashiCorp provides infrastructure automation software for multi-cloud environments, enabling enterprises to unlock a common cloud operating model to provision, secure, connect, and run any application on any infrastructure. HashiCorp tools allow organizations to deliver applications faster by helping enterprises transition from manual processes and ITIL practices to self-service automation and DevOps practices. 


IBM is a leading global hybrid cloud and AI, and business services provider. We help clients in more than 175 countries capitalize on insights from their data, streamline business processes, reduce costs and gain the competitive edge in their industries. Nearly 3,000 government and corporate entities in critical infrastructure areas such as financial services, telecommunications and healthcare rely on IBM’s hybrid cloud platform and Red Hat OpenShift to affect their digital transformations quickly, efficiently and securely. IBM’s breakthrough innovations in AI, quantum computing, industry-specific cloud solutions and business services deliver open and flexible options to our clients. All of this is backed by IBM’s legendary commitment to trust, transparency, responsibility, inclusivity and service.