Written by Astitwa Mishra, a B.A. LL.B. (Hons.) student, and Suraj Kamti, a B.B.A. LL.B. (Hons.) student at the University Institute of Legal Studies, Chandigarh University.

This finance guy got on video call with me and several colleagues in Arup’s Hong Kong office behind the Sydney Opera House based engineering company in London in January 2024. He wasn’t only suspicious of a message he received prior to requesting a confidential transfer – the call was supposed to assuage his doubts. Yes, the faces were known, the voices were clear and the little mannerisms were there. He authorized 15 wire transfers in the next couple of hours that amounted to approximately USD 25.6 million sent to five Hong Kong bank accounts. All of those on that call, including the CFO, were artificial intelligence ‘constructs’, created by video and audio of Arup’s own executives which was publicly available. There was no password, or pin, which could be deciphered. Arup’s servers were not used for any malicious purposes. But, despite this, the company incurred more losses than most traditional ransomware are likely to ask for.

The Arup incident is just a good stepping stone to a broader debate on the status of cybercrime law today. Computer misuse criminal law is defined as the unauthorized use of a computer and is shared by the Computer Fraud and Abuse Act (CFAA) of the United States, the Information Technology Act, 2000 in India and the Budapest Convention as the rest of the world. Anyone who is not authorized to use a system or uses it beyond the limits of the authority he or she has been given is guilty of an offence. In all three cases, it is being used without permission, and for all those reasons, it’s been pretty effective at thwarting hackers, viruses and, most recently, ransomware.

But AI, particularly the advent of large language models used as autonomous or semi-autonomous agents, ruptures this tether, one that manifests itself in a variety of ways. If an attacker is able to create an input which bypasses the instructions that he/she programmed into the AI, and instead follows their instructions, it is not accessed but rather the AI was already doing whatever it was designed to do. It does something that it was meant to do, in a sense: follow instructions in its context, and the AI reads the words it was written to read. The issue is not that it circumvents a technical barrier, it’s that it takes advantage of the fact that the model is not robust enough to tell the difference between a trusted instruction and an embedded one. This is another form of harm, for which the law has yet to determine a name, or even a course of action for punishment.

This piece is in 5 parts. The authors of Part II explore the evolution of current cybercrime concerns (phishing, ransomware, impersonation fraud) instead of their replacement, and how existing laws can cope with the evolution. Prompt injection is then explored in detail in Part III, where you will find explanations about why, conceptually, it is different from the traditional intrusion, and why security researchers now say it’s not a patchable bug but an unsolved problem. The tension this has created in the current U.S., India and international cybercrime legislations is discussed in Part IV. Part V explores current regulatory responses, such as the EU AI Act, NIST AI Risk Management Framework and ISO/IEC 42001, as well as details a legal agenda that is particularly relevant to AI-native cyberattacks such as India’s data protection regime.

II. The Changing Anatomy of Cybercrime: AI as a Force Multiplier

A. AI-Enhanced Phishing and Social Engineering

The phishing attack has always been based on a very simple asymmetry: the attacker knows that the message is forgery, the victim does not. When it comes to phishing, the issue is the amount of work, but when it comes to LLM’s, they crunch that work. In a generative model, the native speaker does not have to know the context of the target organisation and dedicate time to crafting well-thought out text in the target language, but can instead have the text generated in a few seconds that is grammatically correct and contextually suitable, followed by the personalisation of the text with data from social media, leaked databases, or a company’s own public disclosures. Offence committed – cheating by personation, or fraudulent inducement to transfer of funds or credentials – is not new. The cost of committing it has remained the same, and so have the number of times it can be attempted, except when it is done very well. The correlation between AI-powered phishing and AI-powered attacks on the systems processing the legitimate content (e.g. email, tickets, documents) is no accident, it’s about the same trust factor that automated systems (human or machine) have in any legitimate content.

B. Deepfakes and the Collapse of Verification

The other undesirable development that can be illustrated with the Arup case is the reduction in the significance and value of real-time verification as defence. Even though it’s easy to configure a video call with a trusted contact, this is a great foundation for establishing identity, compared to an email, at least. The underlying deception is easily covered by the provisions of corporate fraud in most jurisdictions, such as the Bharatiya Nyaya Sanhita (Indian Penal Code) provisions on cheating and forgery or the similar federal wire fraud statute in the United States. Where they were not designed to look: the fraud model that is played with the victim’s own eyes and ears, and is rendered “ordinarily diligent” to deceive.Where they don’t belong: one type of fraud model, synthetic media is placed in victim’s eyes and ears, which the victim uses and fools.

C. AI in Ransomware Operations

Ransomware is also a virus that is well known in the doctrines, but operates quickly in practice. The advantages of AI for the ransomware lifecycle are automated targeting network reconnaissance, AI-aided ransom notes and negotiation scripts for the victims’ perceived ransom payability, and ultimately, negotiation bots operating on the ransomware side. All of this has no impact on the ingredients of the underlying offence. It does make the average, underfunded hospital, school district or city/town hall a more economic target than it was before, because it is easier to run an extortion on a specific target, than it is on an average one.

Legislatures have been working on the ransomware problem, but not in the context of an update to the cybercrime laws as they are, but with specific and ad hoc measures. When legislatures have decided they can’t handle the cyber threat, they have tended to work around it, not through it, and it is likely that when the cyber threat becomes more AI-based, like prompt injection, a prohibition on paying the ransom, if not communicating with the ransomware actors, will be added to the state budget in 2021.

D. So why is there need for new law here?

The conclusion that often follows is that there is a doctrinal vacuum which has allowed the creation of phishing and ransomware using AI.Taking the above as a fact, it’s easy to draw a doctrinal conclusion that AI-powered phishing and ransomware is the result of a doctrinal void. They don’t, primarily. In all instances, the crime committed is un-authorised access, fraudulent inducement, extortion, but using more sophisticated methods. The tricky parts of the law are, however, in the scenario where an attack occurs without circumventing a technical barrier, but by exploiting a system’s own design, with the instructions that the system was built to follow as in a deep fake voice call or an AI generated phishing email.

III. Prompt Injection: a problem in the Prompt Channel.

A. What is the Difference between Prompt Injection?

In cyber attack, the system is made to do something that it was not designed to do. This could be a SQL Injection, buffer overflow or credential stuffing attack. One of the few technicalities in which the large language model was designed to be able to handle is to follow instructions in its input is what is exploited in prompt injection. This weakness is not a problem with the access control list, but a property of these types of models of language. A stream of tokens that are a mixture of a system prompt read aloud to the user, content extracted from a document, and a user’s message. For instance, the attacker makes an instruction (text) for that stream, in this case, an email, the model will most likely be given a task to summarise the content of that email, and/or, if it is a support ticket, it will most likely be asked to do a task on that ticket, such as triage it. The Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications (OWASP) 2025 is a top 10 list of the worst attacks against production deployments of LLM applications, and the OWASP documentation admits that though many of these methods reduce the risk, there is no guarantee they eliminate the risk altogether because LLM applications are probabilistic.

B. Direct and Indirect Injection.

Typically, there will be someone available to tell you what is the difference between two types of the attack. When the users enter bad prompts directly in the chat window, which is an attempt to bypass the safety measures or malicious exploitation of it, it is called direct prompt injection. The more serious variant for enterprise deployments is indirect prompt injection: The malicious instruction is part of third-party content that the AI system is supposed to act on a third party’s instructions: a webpage it is asked to summarise, a calendar invitation it is asked to read, a resume it is asked to read about for a hiring pipeline, an email in a shared inbox. The attacker also doesn’t need to communicate with the target organisation as there’s no mechanism in the model that says that the content is untrusted according to the attacker’s instructions.

There are documented instances of real life occurrence that illustrate the pattern – not just theoretical. Single-click data exfiltrations in consumer AI assistants, activated by a malicious URL parameter, and another group of vulnerabilities in AI coding and productivity assistants and copilots that required vendors to implement basic sanitisation of the output of these tools have been reported both in the past year and in the wild, increasingly in production systems, according to analysts tracking the trend.

C. Why it is not needed to be patched.

A traditional software vulnerability can be solved in principle: Determine vulnerable part of a software, modify it and test. Prompt injection is not something this model is capable of, since the ability it relies on (the ability to follow instructions expressed in natural language) is not a bug in parallel to the model’s purpose. It’s mostly synonymous with the purpose of the model. An effective filter for injected code can also be effective for legitimate code, and the attack surface is growing with the use of models with more external tools: e-mail clients, retrieval augmented search, code execution environments, and third-party plugins through open protocols. A security perspective: As the experts in the technical security community concur, according to OWASP in their guidance, NIST in their AI Risk Management Framework’s Generative AI Profile and the newer Agentic AI Profile (NIST Internal Report 8596): There is no one-size-fits-all approach to securing AI agents – layered defenses are needed.

D. From Chatbots to Agents: Why the Stakes Are Rising

The number of AI systems that just answer questions has decreased while the number of systems that actually take action has increased – and it’s only getting more urgent and more urgent: writing and running code, asking questions of enterprise databases, sending emails and making financial transactions. In the case of the Arup fraud, while a deepfake video isn’t required, it’s the same kind of damage that such a response would have caused on a chatbot — an agent with access to a company’s email account, payment system or production codebase can wreak this kind of havoc.

IV. The Doctrinal Gap: existing cybercrime law has fallen behind the curve when it comes to the AI-native threats.

A. The Authorisation Paradigm and its limits

All cybercrime prosecutions in the United States that involve an AI system will ultimately have to grapple with whether the accused used it “without authorization” or “exceeded authorized access” and both of those require that the defendant have access to the AI system, albeit for different purposes, to some extent. This is an assumption that doesn’t necessarily meet the prompt injection. In a traditional sense, if a user sends an antagonistic prompt to a publicly accessible chatbot that user has not broken any rules regarding access to the chatbot because he could have sent messages to it anyway, it is the message he sent that harmed the model.

Opinions are divided on the extent to which this should go. The uncertainty is not theoretical, however, because this Federal trade secret case was brought in 2025 and has been brought to court, for which it raises an interesting question about the meaning of the statute, and how it was interpreted and applied in this case: Is it a lawful technique to reverse engineer using a public interface, or a violation of the CFAA unauthorised access provision?

B. Van Buren and the Narrowing of the CFAA

Large language models, on the other hand, have no innate module boundaries—the same weights are applied to a user’s query as well as to a manipulated query, and the same context window is used to generate the response for each.Large language models do not have distinct parts to perform different functions, the weights are identical and the context window is identical as opposed to a file system or a database. Today, Van Buren is in even more shaky doctrinal waters in 2021 thanks to the statute’s effect of narrowing the scope of the doctrine.

C. India’s IT Act: Unauthorised Access as the Organising Concept

Unauthorized access is also similar to that of the Information Technology Act, 2000 of India. Thirdly, a prompt injection attack on an AI system that’s been deployed by an Indian business is not so easy to pull off: If an attacker accesses an AI without the permission of the person controlling it, they may be liable for Section 43, but if they haven’t interacted with a public facing chatbot, or an entered document is not the kind the system was designed to accept, the crime is more about the deception than the method itself.

This gap is worse after the advent of the Digital Personal Data Protection Act, 2023 along with the IT Act. While the wrongful use of an AI agent and the exfiltration of personal data might not clearly be an offence under the IT Act, the damage done by such an AI agent using/exfiltrating personal data will certainly be a personal data breach under the provisions of the DPDPA on breach notification (Part V). The Indian laws on cybercrime and data-protection were two separate pieces of legislation, not designed to be related to this specific case.

D. International Law: The Budapest Convention and the UN Cybercrime Convention

At this moment, there is no specific provision in the Convention on Cybercrime on AI-enabled crimes and if the draft Convention on Cybercrime will eventually become an international instrument, there is no explicit provision in the draft Convention on Cybercrime on international cooperation on AI-enabled cybercrimes. The provisions concerning international cooperation in the Convention are limited to those offences which can be prosecuted with a minimum sentence of four years imprisonment, which may not necessarily include some offences of manipulation relating to AI, depending on the interpretation given in domestic legislatures.

V. Comparative Regulatory Responses: Building New Guardrails

A. The EU AI Act’s Article 15 Approach

As if by magic, the most advanced – and still somewhat underdeveloped – regulatory answer yet is not from the cybercrime part of the world, but instead from product security law. This, indeed, is the first legislation to explicitly mandate a provider of a high-risk AI system to make the system resilient to a third party exploiting its vulnerabilities to change its use, outputs or performance, rather than leaving this to the criminal law which would come into effect after a fact.

The provision of Article 15 is indeed a novel doctrinal development which should prove useful in future comparative studies on the topic – it takes a step directly toward the manipulability of a system, not just as a measure of criminal deterrence for the party who could exploit such manipulability, but also as compliance obligations for the party who has built or deployed it.

B. Standards as Soft Law: NIST AI RMF, ISO/IEC 42001, and OWASP

Technical standards have filled in the gap in most jurisdictions, and are becoming more and more de facto compliance markers. With no comprehensive legislation regarding artificial intelligence in India, this multi-layered approach to standards is likely to be the de facto baseline vocabulary for compliance with the requirements of Article 15 or other standards that have similar obligations in a jurisdiction like India, and may be the evidentiary language for determining compliance and other AI obligations in the future if a due-diligence or a negligence standard is applied to an AI-manipulation incident.

C. India’s Data Protection Route: The DPDPA and the DPDP Rules, 2025

The most direct form of regulation that India faces on AI-native cyberattacks is not through its cybercrime law, it’s through its comparatively new data-protection law. It provides a much-needed (but limited) regulatory remedy – If a prompt injection attack bypasses the AI agent, and personal data is shared with an unauthorized third party, then this is a data breach under the DPDPA and the loss of control of the data is severe, even if a criminal can be named and brought to justice under the IT Act. However, it does not create any crime with regard to the person who actually does the manipulation, which has yet to be determined by a “non-binding interpretation” of the IT Act.

New Zealand’s State responses to ransomware attacks are in line with those of other countries.In the case of response by the states of the EfSui, ransomware Payment Bans are in effect.State responses to EfSui: Ransomware Payment Bans.

The payment bans in North Carolina and Florida that were covered in Part II are a blueprint for what may be necessary in the future for AI-native attacks, but without requiring criminal law enforcement. The question of whether payment bans are good policy or not does not exist, but even in light of the particular facts and data available today to the payment ban in North Carolina, one wonders how much of an impact this decision would have if the law was enacted in the future, and the approach of designing a narrow and specific measure with its own logic that is predicated on the particular mechanics of the new attack — rather than trying to shoestring the new attack into the existing doctrinal toolbox — is instructive for how legislatures might eventually approach prompt injection and AI system manipulation more generally.

VI. Towards Legality of AI-powered Cyberattacks.

As analysed above, the existing laws and regulations on cybercrime and cyber fraud are well suited to regulate phishing and ransomware attacks involving AI, in a sense allowing for room for interpretation. There’s another issue though, and that’s the prompt injection and manipulation of the AI system. It is not likely that the judicial interpretation of the statutes, whose purpose was to regulate a different time and technology era, will be able to fill the gap in understanding in the above. Its suggestions for a legislature – or, for the meantime, a well-funded regulator – are five proposals.

A. Define “AI System Manipulation” as a Distinct Statutory Concept

A Distinct Statutory Concept (DSC) is a statutory concept which is clear in the syllabus.

Rather than adding the concept of unauthorised access to the vocabulary of prompt injections, legislatures should create a new crime of the kind of “adversarial prompt injection” that occurs when someone uploads instructions that get the AI system to perform actions that it is supposed to prevent. This would eliminate the entire line-drawing problem, and question whether the defendant intended to make the system disregard the “lines” set up by the system deployer for the system. Severity could be graded, as already provided in the EU AI Act with the different levels of treatment that are provided for high-risk systems – as the actual harm caused and the sensitivity of the system being manipulated would differ.

The EU AI Act’s concept of protection against manipulation is one that should be embraced not only in the EU, but with any regulation of AI anywhere, but it is not just the fines that will be effective in the EU AI Act. As with other types of AI systems, high-risk or high-exposure systems would need to be documented for adversarial testing and would be strong evidence of negligence in civil suit if there is no such testing and the owner or operator of the AI system suffers a loss from manipulation of the system by a prompt. Thus an existing voluntary technical practice is turned into a kind of fire-code inspection system as are the basis of negligence in regard to physical safety: a system of documentation, which a court can use to decide whether reasonable precautions have been taken.

B. Allocate Liability Along the AI Value Chain

With an AI-native incident, there are at least three parties involved: the developer of the foundation model, the enterprise that deployed and configured it, and the attacker that created the input into the model that would be used to try to manipulate it. Existing law was drafted for the developer-deployer model or the liability that would go to any one of the individuals who did not keep a perimeter in place was not intended for this situation. Additionally, a statutory apportionment scheme similar to the product liability doctrine could also exist: the developer of the foundation model (who did not implement some of the recommended defence measures according to the prevailing technical standards) would not be liable, the enterprise that gave its agent an excessive privilege to use the use case would be liable. Such a distinction is already implied by the NIST Agentic AI Profile, which is about “least-privilege tooling.

C. Update International Cybercrime Instruments

However, the Budapest Convention, as well as the newer UN Convention against Cybercrime, were negotiated without any consideration of the presence of AI manipulation. Proposals to modify and/or expand the Budapest Convention, which have been made by scholars, should be taken up at the treaty level, not in the narrow sphere of the academic writing and commentary, and in the context of the slow pace of treaty negotiations. It is important to consider an interim step of an informal ‘non-binding model law’ or guidance note, as the Council of Europe has done in the past to explain existing provisions in the Budapest Convention, as a method to assist states in interpreting the provisions; such notes should explicitly cover offences to be covered by the addition of a concept of AI manipulation before the formal amendment process would occur.

D.Effectively apply Safeguards for High Stakes Deployments to a Sector of the Field.

Finally, organizations that are already subject to significant impact on their decisions and operations by AI agents – like Healthcare, Financial Services, and Critical Infrastructure – must be protected with sensitivity to their risks and not be one-size-fits-all horizontally. Specific oversight of the risk of AI manipulation is yet to be clearly identified, and the financial sector regulators (Reserve Bank of India) and the new provisions on health data in the DPDPA Rules could be qualified as a potential lead agency.

VII. Conclusion

In this piece, I argued that the path that this type of offenses has taken (from AI-driven phishing, to the demise of real time verification as in the case of Arup, to the new structural type of threat known as prompt injection) represents an evolution in cybercrime that the current law has failed to adequately capture. In some cases, AI is also providing a tool to aid in the commission of the crime or using it to make the crime more plausible; for example, in phishing, ransomware, the principles of the crime were basically still the same, with the only difference being the interpretation and execution skills. But where AI isn’t hacked, hackers can still get away with it, exploiting the gullibility of a system to follow instructions, as Van Buren’s limitation of the CFAA, the unresolved debate among American law scholars, and the lack of any specific section in the Indian IT Act indicate, albeit in different ways.

The EU AI Act, along with the technical standards environment that are likely to be established around NIST, ISO/IEC 42001, and OWASP, is the most stringent to date in terms of directly regulating it, being able to manipulate is a compliance risk – it’s not the only one – and therefore a compliance liability for the AI system developers. The current answer to the problem in India is by way of breach-notifying regime in the DPDPA where the incentive is to create better designs to defend. It offers little clarity, however, as to who would be held responsible for the manipulation – the individual who has committed it or the person who has been manipulated. The final step – the last gap – is not in the distant future and can be accomplished through the addition of a separate statutory definition of AI system manipulation based on the robustness of the AI system, liability allocation in the value chain, updating of international instruments and sectoral specific safeguards. This space will only become more costly as more and more AI agents do the answering of questions, writing code or managing infrastructure, and the next Arup-scale incident won’t wait for the law to catch up.