BCU and Covatic develop new UK AI defense to counter cyber attacks on black-boxes

Research outlines approach to improve resilience of AI models used in high-risk environments

Birmingham City University (BCU), based in the United Kingdom, has published new research in collaboration with UK tech company Covatic on improving AI system defenses against black-box attacks.

The study was released in the journal Expert Systems with Applications.

AI models used in sectors such as healthcare and autonomous vehicles are often targeted through subtle changes to input data. In black-box attacks, adversaries repeatedly test a model to reverse-engineer how it works, allowing them to feed altered inputs that can lead to incorrect outputs, such as misidentifying a stop sign or misinterpreting medical images.

The research proposes a method that applies basic image alterations, rotations or resizing, before data enters the model. This pre-processing step is designed to disrupt adversarial inputs and improve resistance to such attacks.

Testing showed a 21% increase in performance compared to standard models without defenses, and a 2.3% to 4.6% improvement over alternative strategies such as random noise defense, depending on the system used.

Applied partnership with integration plans underway

The study is the result of an existing partnership between BCU and Birmingham-based Covatic. The company provides audience targeting tools for digital advertisers using edge computing and privacy-preserving technology. Covatic plans to apply the findings within its machine learning infrastructure.

Atif Azad, Professor of Artificial Intelligence at BCU, said: “As AI plays a bigger role in critical areas like healthcare and self-driving technology, addressing security risks is essential. This research takes an important step toward making AI systems more resilient against cyber threats.”

Dr. Umar Daraz, Director of Innovation at BCU, said: “We are incredibly proud to be working with our partners from Covatic. It’s a relationship that has grown to encourage cutting-edge research on AI, data and cyber security solutions, and has helped to create new jobs in the West Midlands. This research will form part of Covatic’s own applications, which are used by millions of people already.

“The collaboration has been built on our common goals of innovation and supporting local companies by designing, developing and introducing new tools that were seeded locally, but could have international impacts.”

Research designed to inform product resilience

The project was also used to evaluate the robustness of Covatic’s own systems. The company says its advertising technology now runs on infrastructure informed by the testing protocols developed during the collaboration.

Nick Pinks, CEO at Covatic, said: “At Covatic, we are committed to staying at the forefront of technology. This breakthrough has helped us stress-test and better secure our machine learning algorithms from malicious attacks.

“Our CTO, James Holyhead, worked closely with the BCU team on this project, reflecting our dedication to advancing future technologies, particularly in the advertising industry.”