How open source drives innovation in the UK public sector

Although it may seem counterintuitive for a business model that involves making source code freely available for redistribution and modification, open source is everywhere. The history of how it arrived here: a belle-epoque of collaboration buoyed by serious investment from juggernauts like Amazon, Google and Facebook has been covered before. Less attention has been drawn to how integral open source has been to the thriving innovation currently transforming the UK public sector

Innovation in the public sector

In the last decade or so, the UK public sector has accelerated through a sea change of tech-driven modernisation under the banner of digital transformation. A report released last week revealed that mobiles and tablets now account for the majority of internet traffic to GOV.UK services.

Mobile access has been non-negotiable for most modern businesses for a number of years. Yet the figure truly crystallises the progress made in a segment not known for being on the pulse of tech trends, and one often satirised for lagging remarkably behind wider industry. Last July, in a dish seemingly readymade for farce-hungry twitter trolls, the Government’s new Digital Secretary was revealed to have not tweeted since 2015.

To say that Government ministers are overflowing with enthusiasm for solutions that improve the speed, efficiency and quality of their services would be to put it mildly. The fervour is best exemplified by the new UK Health Secretary, Matt Hancock, who since taking on the role has embarked on a PR stroke evangelical mission to eulogise how AI and other technologies can transform the NHS once and for all.

Apart from more conventional measures like ditching fax machines, Hancock has promised to scrap hand-written prescriptions and is a proud member of GP at Hand, Babylon AI’s controversial digital GP service.

The transformation “process” often involves enlisting specialist firms such as Kainos or ASI Data Science to deliver on digital transformation goals that leverage AI, machine learning and data-driven making, and that, most importantly, are “cloud-first”.

Compared to startups, where innovation tends to be focused on doing something completely new and disruptive, external consultancy and delivery teams usually find that public sector employees are less welcoming to change. After all, they are often working in low-paid, pressurised and uniquely scrutinised environments where the onus is to not make a headline-worthy cock up.

“It was only once I started working more directly with more traditional areas such as healthcare organisations or the public sector that I got a flavour of what it was like to work with more organisations where change is not exactly popular,” says Dr Austin Tanney, Head of AI at Kainos.

“To be honest though, I’m not sure if any of us really want a massive government department or a health trust to try and revolutionise things, totally disrupt what they do and be willing to gamble everything,” Austin adds. “Innovation in the public sector, maybe, by necessity, a little less dramatic.”

Model mania

Machine learning is the technology driving most of the disruption in the public sector. Departments gripped by model-mania are desperate to deploy algorithms in every nook and cranny of the state’s infrastructure. The Environmental Agency is building models to predict areas at risk of flooding; the Home Office to alert them to suspicious passport applications, and the Department of Work and Pensions to detect fraudulent applications.

Kainos has helped DEFRA, DVSA, DWP and others deliver many digital transformation projects, many of which are multiyear and ongoing. In an early stage machine learning project, it’s helping the DVSA to identify fraudulent activity in MOT garages – a project that won the UKIT award for machine learning and AI project of the year.

“By using AI, we have the potential to develop solutions that can improve efficiencies, deliver better services and reduce costs,” Austin says.

ASI also works directly with government departments to evolve their AI capabilities. Jean Innes, Director of Transformation and Strategy at ASI, tells me that their first step is to train and build data science units within departments, before designing the models that identify the hitherto hidden patterns contained in the truckloads of data citizens provide the government.

“There is an increasing desire within the public sector to leverage the power of artificial intelligence, machine learning, and data-driven decision making to enhance public life,” says Jean.

ASI also has a track record of contributing projects for the public good. When the Prime Minister challenged Big Tech to remove radicalising content from their platforms, Facebook and others said the technology wasn’t ready. ASI thought otherwise and set about building a sophisticated ensemble machine learning model for the Home Office, which they made available for free to all media platforms.

Open Source

Jean identifies a number of key ingredients required for AI innovation to succeed in the public sector: from identifying a “well-defined problem or opportunity”, to cultural changes (encouraging in house data-leads to cooperate with experts and adopt new technical approaches), to developing high-trust relationships between external collaborators to “ensure that there are no surprises.”

A final indispensable component is open source software: it keeps the project within budget in the short-term, prevents costly and prohibitive vendor lock-in further down the road, and give scientists more freedom to experiment.

“It also allows you to make sure you are making the right technical choices to meet the outcome for the user in the early stage of delivery,” adds Austin.

Jean tells me that the UK’s strongest data scientists come from academic backgrounds. 

“You can understand what your algorithm is doing all the way through because you can view the source code of libraries that you use. If that is insufficient, you can modify the algorithm to make it the best it can be, rather than relying on a half-baked pre-packaged implementations,” she says.

Culture of collaboration

The ethos of open source demands that data scientists give a bit in return for the take and contribute innovations back to the libraries they use; building positive sum games in which everyone benefits.

The Lens library, developed by ASI consultants and engineers, automates the initial step of data exploration, cutting out several hours of drudge work that used to stultify new projects. And many of ASI employees are responsible for innovative contributions to Jupyter – the open source ecosystem dedicated to data analysis.

Fostering collaboration in software engineering is a cause that the Government also has thrown its full weight behind. Part of its Digital Service Standard states that all new source code developed is made opened and published under an open source license. Kainos’s deputy CTO, Rory Hanratty tells me that the standard has helped to build better practices around software engineering, expedited collaboration and sped up innovation and bug-fixes.

“Being part of an open source software community that is free to both use and to contribute code and code changes also brings with it a positive working culture, based around openness and transparency, encouraging learning and sharing across deliveries,” adds Austin.

But Austin also cautions that the rapidly evolving development frameworks at the top of the stack can mean changes are sometimes brittle. Additionally proving the “supply chain” of dependencies can be tricky from a security standpoint.

His advice?: “Make sure you have a solid approach to making technology selections, and make maturity and security part of your selection criteria.”

The main threat to this decentralised approach to innovation is that the collaborative spirit won’t persist. The products of open source are often greater than the sum of its parts, stimulating innovation that wouldn’t otherwise happen. But without the community scaffolding, progress would equally come to a grinding halt.

That’s not to say this will happen any time soon, or at all. High-profile advocates at the UK’s leading software companies just need to remember to consistently advocate for open-source and not forget to return the favour.

• Jean Innes is Director of Transformation and Strategy at ASI
• Dr Austin Tanney is Head of AI at Kainos
• Rory Hanratty is Deputy CTO of Kainos