News Hub

Nation-state threat actor suspected of Cloudflare hack

Written by Wed 7 Feb 2024

Cloudflare said it suspects a nation-state threat actor is responsible for a breach of its Atlassian systems.

At 4pm on 23 November, Cloudflare’s security team was alerted to the threat actor’s presence and deactivated the Smartsheet service account 35 minutes later. After another 48 minutes, the user account created by the threat actor was identified and deactivated.

Cloudflare said its security team immediately began an investigation and cut off the threat actor’s access. On 26 November, Cloudflare brought in Crowdstrike’s forensic team to perform their independent analysis.

Whilst the threat actor was not named by the company, Cloudflare said that based on its collaboration with industry colleagues and the Government, the security incident involved a ‘sophisticated actor, likely a nation-state’ who operated thoughtfully and methodically.

The operational impact of the incident was described as ‘extremely limited’. However, Cloudflare said it took the incident seriously as the threat actor used stolen credentials to access the server, obtaining ‘some documentation and a limited amount of source code’.

Cloudflare stressed no customer data or systems were impacted by the event. The company added because of its access controls, firewall rules, and use of hard security keys enforced using its Zero Trust tools, the threat actor’s ability to move laterally was limited.

As a result, no services were implicated and no changes were made to its global network systems or configuration.

“This is the promise of a Zero Trust architecture: it is like bulkheads in a ship where a compromise in one system is limited from compromising the whole organisation,” said Cloudflare.

Cloudflare’s ‘Code Red’ Mitigation Project

From 27 November, Cloudflare redirected the efforts of the Cloudflare technical staff to work on a project called ‘Code Red’. The company’s focus was to strengthen, validate, and remediate any control in its environment to ensure Cloudflare was secure against future intrusion.

“Analysing the wiki pages they accessed, bug database issues, and source code repositories, it appears they were looking for information about the architecture, security, and management of our global network; no doubt with an eye on gaining a deeper foothold,” said Cloudflare.

Therefore, Cloudflare decided to intensify its security protocols to prevent any potential oversights in its log files by threat actors.

“Our aim was to prevent the attacker from using the technical information about the operations of our network as a way to get back in,” said Cloudflare.

Cloudflare rotated more than 5,000 individual credentials, segment test and staging systems, performed forensic triages on 4,893 systems, and rebooted every machine in its global network. This included all systems the threat actor accessed and all Atlassian products like Jira, Confluence, and Bitbucket.

The threat actor also attempted to access a control server in Cloudflare’s new data centre in São Paulo. All attempts to gain access to the control server were unsuccessful. To secure the equipment in the Brazil data centre, the equipment was returned to the manufacturer.

Cloudflare also looked for software packages that had not been updated, user accounts that may have been created, and unused active employee accounts. The company also scanned for any sensitive information in Jira tickets or source code, purged all HAR files, and securely uploaded them to the wiki to mitigate any potential token exposures.

“Whenever in doubt, we assumed the worst and made changes to ensure anything the threat actor was able to access would no longer be in use and therefore no longer be valuable to them,” said Cloudflare.

On 5 January the immediate ‘Code Red’ effort ended. Cloudflare said work continues across the company around credential management, software hardening, vulnerability management, additional alerting, and more.

How did the Threat Actor Gain Entry?

On 18 October, Cloudflare was the victim of a compromise to Okta’s systems, which allowed the threat actor to gain access to a set of credentials previously meant to be rotated.

Cloudflare failed to rotate one service token and three service accounts of credentials that were leaked during the Okta compromise. These tokens were not rotated as Cloudflare mistakenly believed them to be unused.

On November 14, the threat actor initiated reconnaissance on Cloudflare’s systems, aiming to exploit credentials. The actor attempted unauthorised access to the Okta instance and Cloudflare Dashboard but was denied entry in both cases.

A day later, the threat actor used the Smartsheet credential to establish an Atlassian account resembling a typical Cloudflare user. The threat actor then included the user in certain Atlassian groups to maintain continuous access to the Atlassian environment, in case the Smartsheet service account was removed.

On 15 November, the threat actor accessed Atlassian Jira and Confluence using the Moveworks service token to authenticate through Cloudflare’s gateway. They then leveraged the Smartsheet service account, which had administrative privileges on Atlassian Jira, to install the Sliver Adversary Emulation Framework.

Sliver is a commonly used tool by red teams and attackers to establish command and control connectivity, allowing them to gain persistent access to the computer where it is installed.  A red team evaluates an organisation’s security by simulating real attackers’ tactics, techniques, and procedures (TTPs) with authorisation from the organisation.

The installation of Sliver was facilitated through the ScriptRunner for the Jira plugin. This allowed the threat actor continuous access to the Atlassian server and they used this to attempt lateral movement. 

Over the next day, the threat actor viewed 120 code repositories out of 11,904. Of the 120, the threat actor used the Atlassian Bitbucket git archive feature on 76 repositories to download them to the Atlassian server.

The 76 source code repositories primarily dealt with backup operations, global network configuration and management, Cloudflare’s identity system, remote access protocols, and the use of Terraform and Kubernetes.

Join Cloud & Cyber Security Expo

6-7 March 2024, ExCeL London

Cloud & Cyber Security Expo is one of the largest IT security events in Europe.

Don’t miss the chance to build partnerships and discover solutions to protect your business.

Written by Wed 7 Feb 2024

Send us a correction Send us a news tip