Researchers point to topology for deep-learning speed boost

Researchers at the Champalimaud Centre for the Unknown claim to have developed a new method for computer vision training which could dramatically improve performance - allowing a character-recognition system, as an example, to be trained on just 50 images instead of several thousand.

Deep learning is a hot topic at present: Using a corpus of existing data, a deep learning system figures out how to perform a task such as recognising shapes or faces or creating content of its own. To be accurate, the system needs a large corpus: Even a relatively simple deep learning algorithm needs to be trained on tens of thousands of images before it has a chance of making identifications of its own.

That could be set to change, however, with researchers at the Champalimaud Centre for the Unknown claiming to have found a way to dramatically reduce the number of images a deep learning system needs for its training data.

The secret: Topological Data Analysis (TDA), first developed by mathematician Patrizio Frosini in 1992. 'Topology is one of the purest forms of math,' claims Mattia Bergomi, lead author of the new study. 'And until recently, people thought that Topology would not be applied to anything concrete for a long time. Until TDA became famous in the last few years.'

Using Frosini's TDA, the team created a neural network which is capable of recognising hand-written numerals - a surprisingly difficult task, given the wide variance in human handwriting even between those who use the same written language. Without TDA, the network required a training data set comprised of thousands of images of each of the 10 target digits; with TDA, combined with a set of a priori meaningful features described as 'a set of "lenses" through which the network would "see" the digits,' the neural network required fewer than 50 images per digit with no drop in recognition quality.

'What we mathematically describe in our study is how to enforce certain symmetries,' Bergomi explains, 'and this provides a strategy to build machine learning agents that are able to learn salient features from a few examples, by taking advantage of the knowledge injected as constraints.' The goal of the research: A new neural network architecture, currently in development, which will allow human operators to quickly inject high-level knowledge 'lenses' in order to dramatically speed up their training.

The team's work has bee published in the journal Nature Machine Intelligence as Towards a Topological-Geometrical Theory of Group Equivariant Non-Expansive Operators for Data Analysis and Machine Learning.