February 29, 2024

We are creating some awesome events for you. Kindly bear with us.

EXCLUSIVE – Building a scalable data analytics platform for government

OpenGov Asia had a chance to speak with Dr Ole Nielsen again. This time, Dr Nielsen dons a new role – Director of Data and Analytics for the Chief Minister, Treasury and Economic Development Directorate in the ACT Government. Previously, he held the title of Director of Scientific Computing and Systems Engineering, Geoscience Australia.

Speaking to Dr Nielsen, his knowledge and wealth of experiences are evident. More importantly, Dr Nielsen is passionate about data. To this end, he is willing to explore new frontiers.

Agility in Government

This year, his team established the Australian Capital Territory Data Analytics Centre. The Centre aims to build data literacy across the ACT Government. With better data management and analytics capabilities, the government can ultimately deliver better policies and services for citizens. A team of eleven data scientists, analysts and engineers spearhead the Centre’s projects.

In the Centre, agility is key. Dr Nielsen’s team works in bursts of twelve weeks for any given initiative – one of the first of its kind in the ACT Government. The Centre takes on problem statements from other directorates in the ACT Government. If too little has been achieved by the end of the cycle for whatever reason, the governing body can choose to stop and ask the Centre to work on something else.

“We may not finish, but we will have something of value within that twelve weeks – or we may have uncovered issues e.g. with data access that prevents and initiative from progressing” explains Dr Nielsen.

Periods of intensive focus like these hold immense value. Realtime monitoring of water quality in rivers and lakes, payroll tax compliance, mapping businesses by type and location to inform potential investors, and even applications to promote the safety of victims to domestic violence, are just a few of the initiatives being tackled by the agile mindset.

Data Analytics Project for Government

Of these initiatives, the one he’s most excited about, is the enterprise grade data analytics platform his team developed in an impressive sixteen weeks based on learnings from a minimal viable product which took much longer. The cloud-based platform serves to promote better use of data by providing access and sharing of data between directorates in the ACT Government. At present, government data in Australia is neither stored centrally nor in a uniform manner. Every organisation houses their own data and it is stored either on paper or digitally in a range of systems. Additionally, under the traditional arrangement, security concerns arise when access to a system is granted to a third party. What could be an innocent act of extracting data could result in breaching the system.

“Traditionally, what we’ve seen is silo mindsets. Everyone works really well, but they’re not working together… and that does not always lead to the [desired] outcome,” Dr Nielsen explains. “This platform allows each agency [in the ACT Government] to have their own tenancy, their own area in this data platform, where they can ingest data from their own systems and make it available for sharing with other parts of the government, in a controlled, privacy-centric way. That way data can be shared without allowing third parties access to individual systems. This is a subtle distinction – access to the source system is very different from access to the data”

A DevOps team and agility were key to delivering a successful outcome. The extended team worked together towards a common goal, they automated pretty much everything and every two weeks, the team evaluated the project’s progress and assessed room for improvement. Consequently, this seriously slashed what could have been a sixteen-month project into sixteen weeks.

It’s a People Problem, Not Data

Despite the success of such models, there was some scepticism and not everyone were keen to jump on the bandwagon. After all, this model was novel and unfamiliar. People want detailed plans on how and when the outcome will be delivered – something the team’s modus operandi did not provide. To help getting buy-in for the vision and build trust, bouts of negotiation and relationship management were required.

“It’s difficult to change the status quo,” he said. “People are not afraid of change… people are only afraid of losing something or being blamed for failure.”

For example, taking a gamble on winning a million dollars from the lottery. While this could actually be very dangerous people are not afraid of such a change. On a more serious note, this is Dr Nielsen’s point: People have to take risks from time to time to achieve outcomes in a rapidly changing, ambiguous and complex environment. However, traditionally there is widespread fear of being seen to fail so deviations are often frowned upon. One wrong move could mar their reputation. No one wants a poor performance review.

Nevertheless, during the implementation process, the team remained very respectful of the existing ICT processes which helped build the trust that is required. They made it a point to have face-to-face interactions, rather than communicating over an ebb and flow of emails. For Dr Nielsen, this was a game changer. Both teams could understand each other’s point of view with immediacy and roll out a mutually agreed solution. In his words, the collaboration fostered on the ground was the biggest change that was achieved.

Dr Nielsen opines, “It takes time, it takes convincing, and it takes a great deal of compassion.”

What Goes Down Must Come Up

More can be expected from the enterprise grade data analytics platform. The platform will consistently be upgraded to meet everchanging demands. One such demand is a growing volume of data which is why the platform is built to be scalable; Another is the relentless need for new versions to be implemented.

As a real test of the platform’s scalability, the team would tear the platform down entirely and build it back up every day through automation. For the team, its cyclical rebirth, and indeed its ability to do so repeatedly without failure, is a litmus test of confidence in the product. This was inspired by the way Netflix and Australia Post are working, for example.

At this point, Dr Nielsen’s understanding of scalable shocked me. But he assures, “But that is how we need to work in the future.”

Once a change has been made, the automation script keeps a record. A new version can be rebuilt with the change in mind instead of revamping the existing system. Affectionately calling it a software product rather than a project, this process allows for a complete automated recreation of the platform whilst incorporating changes. If ever the system were to go down, an identical platform can be built from scratch in a matter of hours, Because of what automation can afford in terms of velocity, reproducibility and transparency., the successful regeneration process is a great source of motivation for the team. Especially in times of disaster, they are confident in mitigating risks.

How Data Continues to Challenge Government

The data analytics platform is a good instance of how governments can use technology for better services and policies. However, there continues to be challenges in realising that. Dr Nielsen shares what he thinks are some fundamental issues he’s seen.

First are systems which are incompatible with the demands of the times. Data is sitting in a hodgepodge of different systems, with different modes of governance. Accessing the data is difficult and the quality of data is also varied.

Furthermore, legislation which governs data in the past might conflict with what is needed today, evidenced by Australia’s new proposed Data Sharing and Release Bill which aims to set out more legislative clarity to promote better use of data

Third, there is a reluctance to share data within government. Dr Nielsen attributes this to culture. Officials are uncertain about another agency’s data quality, and hence might be misinterpreted. Questions about the ownership of the data linger. Not only do people fear a loss of control, they are also afraid about doing the wrong thing since there is a lack of clarity about what is within reasonable and legitimate bounds.

“There’s a whole range of issues. But they all boil down to uncertainty and fear. We need to have a lot of clarity, psychological safety and transparency around this,” said Dr Nielsen.

The condition is not limited within the realms of the government. Around Australia, Dr Nielsen notes a lack of public trust in government. There might be more trust in government if people saw more transparency and engagement from government organisations.

He says, “We as government need to earn the trust. You can’t tell people to trust you. You earn the trust by being transparent and delivering digital services of value to people.”

Dr Nielsen concludes, “It is a very exciting time to live right now. We have digital technology opening up, where everyone is interconnected, and you can get real-time services which are personalised. Although it is going to be difficult to navigate, it is also very exciting.”

PARTNER

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private company, Qlik offers real-time data integration and analytics solutions, powered by Qlik Cloud, to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik serves more than 38,000 active customers in over 100 countries.

PARTNER

CTC Global Singapore, a premier end-to-end IT solutions provider, is a fully owned subsidiary of ITOCHU Techno-Solutions Corporation (CTC) and ITOCHU Corporation.

Since 1972, CTC has established itself as one of the country’s top IT solutions providers. With 50 years of experience, headed by an experienced management team and staffed by over 200 qualified IT professionals, we support organizations with integrated IT solutions expertise in Autonomous IT, Cyber Security, Digital Transformation, Enterprise Cloud Infrastructure, Workplace Modernization and Professional Services.

Well-known for our strengths in system integration and consultation, CTC Global proves to be the preferred IT outsourcing destination for organizations all over Singapore today.

PARTNER

Planview has one mission: to build the future of connected work. Our solutions enable organizations to connect the business from ideas to impact, empowering companies to accelerate the achievement of what matters most. Planview’s full spectrum of Portfolio Management and Work Management solutions creates an organizational focus on the strategic outcomes that matter and empowers teams to deliver their best work, no matter how they work. The comprehensive Planview platform and enterprise success model enables customers to deliver innovative, competitive products, services, and customer experiences. Headquartered in Austin, Texas, with locations around the world, Planview has more than 1,300 employees supporting 4,500 customers and 2.6 million users worldwide. For more information, visit www.planview.com.

SUPPORTING ORGANISATION

SIRIM is a premier industrial research and technology organisation in Malaysia, wholly-owned by the Minister​ of Finance Incorporated. With over forty years of experience and expertise, SIRIM is mandated as the machinery for research and technology development, and the national champion of quality. SIRIM has always played a major role in the development of the country’s private sector. By tapping into our expertise and knowledge base, we focus on developing new technologies and improvements in the manufacturing, technology and services sectors. We nurture Small Medium Enterprises (SME) growth with solutions for technology penetration and upgrading, making it an ideal technology partner for SMEs.

PARTNER

HashiCorp provides infrastructure automation software for multi-cloud environments, enabling enterprises to unlock a common cloud operating model to provision, secure, connect, and run any application on any infrastructure. HashiCorp tools allow organizations to deliver applications faster by helping enterprises transition from manual processes and ITIL practices to self-service automation and DevOps practices. 

PARTNER

IBM is a leading global hybrid cloud and AI, and business services provider. We help clients in more than 175 countries capitalize on insights from their data, streamline business processes, reduce costs and gain the competitive edge in their industries. Nearly 3,000 government and corporate entities in critical infrastructure areas such as financial services, telecommunications and healthcare rely on IBM’s hybrid cloud platform and Red Hat OpenShift to affect their digital transformations quickly, efficiently and securely. IBM’s breakthrough innovations in AI, quantum computing, industry-specific cloud solutions and business services deliver open and flexible options to our clients. All of this is backed by IBM’s legendary commitment to trust, transparency, responsibility, inclusivity and service.

Send this to a friend