Police and national security bodies in the EU will be banned from using real-time biometric data driven by artificial intelligence in most circumstances without having judicial authorisation, it has emerged.
The measure was part of a historic agreement reached between the European parliament and EU member states on Friday after three days of negotiations. However, officials only revealed the operational details on Monday, as the final text will not be published until “a cleaning-up process” is complete.
The ban on “Big Brother” surveillance will apply in public and private places, ranging from parks to sports grounds, except in the event of specified serious crimes, a terrorist threat or urgent searches for victims. Even then, police will require approval from a judge or independent administrative authority first.
Only in the most exceptional circumstances, such as a live terrorist threat, will police be able to switch on AI biometric tools without a judge giving the green light.
However, under the new rules, they must still obtain authorisation within 24 hours and provide the appropriate authority with a “prior fundamental rights impact assessment”, according to an EU official with knowledge of the text.
They must also notify the relevant market surveillance authority and the data protection authority. If permission from the judge or authority is not forthcoming, the AI tool must be switched off and all data extracted on a suspect or suspects deleted immediately, the official said.
According to officials, these safeguards will avert what has been termed “predictive policing”, which MEPs feared would be used alongside racial profiling to discriminate against individuals. One EU official said it would mean that law enforcement authorities would not be allowed to pursue someone as a suspect “just because an algorithm says you are a criminal”.
In total, the EU and MEPs have agreed a specific list of 16 types of serious crime where this exception to the surveillance ban may apply.
The exceptions were terrorism, murder, rape, organised or armed robbery, grievous bodily injury, child sexual abuse, kidnapping, hostage taking, crimes under the jurisdiction of the international criminal court, unlawfully seizing aircraft or ships, sabotage, trafficking human beings, body parts, illegal drugs, weapons or radioactive material, and participation in a criminal organisation involved in one of those offences.
Each country must be compliant with these AI prohibitions within six months of the laws going on the EU statute books or in the EU’s gazette of record, the Official Journal. The new laws governing artificial intelligence in the EU will also include other bans to guard against the risks that AI poses to society and life.
They include AI systems or applications that “manipulate human behaviour to circumvent users’ free will, such as toys using voice assistance encouraging dangerous behaviour of minors”. They also include a ban on AI systems that allow “social scoring” by governments or companies, similar to China’s system of “social credit”.
after newsletter promotion
The EU insisted on this in the final text amid fears that such AI-derived data would be used to exploit the vulnerabilities of people with respect to their age, disability, or social or economic situation. It might have meant, for instance, that certain customers could be banned from restaurants based on their “social score” or status, or that certain employees would be ruled out of jobs.
Emotional recognition artificial intelligence, which can read people’s facial expressions in real time to assess stress or fatigue, will also be prohibited in the workplace.
MEPs leading the fight for prohibitions were determined to ensure that the EU did not turn into a surveillance state like China, where traffic police can intervene, for example, if they think a lorry driver is tired.
