Researchers have discovered a security risk via a malicious package spreading technique called “AI Package Hallucination.” This technique exploits generative AI platforms, such as ChatGPT, that can generate hallucinated sources, links, blogs, and statistics. This can include questionable fixes to Common Vulnerabilities and Exposures (CVE) and links to coding libraries that don’t actually exist.
An attacker can use this technique by formulating a question asking ChatGPT for a package that will solve a coding problem. If ChatGPT fabricates a code library (package), the attacker can create a malicious package to replace the “fake” package recommended by ChatGPT. This could trick a victim into downloading and using the malicious package.
The report mentioned that an attacker could ask ChatGPT for a package that will fix a vulnerability in a popular piece of software. If ChatGPT generates a fake package that claims to fix the vulnerability, the attacker could then create a malicious package that looks identical to the fake package. The victim would then be tricked into downloading and installing the malicious package, which could then be used to exploit the vulnerability.
AI package hallucination could cause a security risk to most industries including data centers as it could potentially steal data, install malicious software and disrupt operations.
AI and automation has revolutionised certain industries, and the data center market is no exception. With the ability to increase efficiency, reduce critical incidents, and improve customer experience, it has become increasingly important for operators to look at AI and automation as a solution. As some data center operators consider hiring automation engineers to work around coding to cater to requirements of data centers, this could pose as a security concern.
Among the steps mentioned to reduce the risk of AI package hallucination is to avoid the download and execution of a code that has not been tested and to obtain open source codes from a trusted and verified source.
The World Economic Forum (WEF) – The Global Risks Report 2022, identified the number one risk for Australia, New Zealand and United Kingdom as the “failure of cybersecurity measures”, which was defined as “business, government and household cybersecurity infrastructure and/or measures outstripped or rendered obsolete by increasingly sophisticated and frequent cybercrimes, resulting in economic disruption, financial loss, geopolitical tensions and/or social instability in those countries.”
Bahrain, Indonesia, Japan, Singapore, Taiwan and United Arab Emirates were among the list of other countries where failure of cybersecurity measures was ranked within the top-5 risk bracket.
