IN FOCUS

Cutting Edge Face Recognition

Photography by Chino Sardea
Digital Imaging by Mohammad Izzadely
23 Feb 2018

Dr. Jiashi Feng works with dynamic artificial neural network – the restless brain of machines that are smart enough to perform complex tasks such as speech recognition, natural language processing, and computer vision. Together with his team, Dr. Feng has developed face recognition and analysis techniques that will enhance Singapore’s homeland security and contribute to its Smart Nation objectives

Dr. Jiashi Feng enables computers to grow learning ability with dynamic neural networks. Dr. Feng is currently an Assistant Professor with Department of Electrical & Computer Engineering of the National University of Singapore. He received his B.E. degree from the University of Science & Technology China in 2007, and PhD degree from NUS in 2014. From 2014 to 2015, he was a postdoctoral researcher at University of California, Berkeley. Dr. Feng has published over 100 research papers in machine learning, deep learning, object recognition and big data analysis. He received awards for ILSVRC2017 object localization, MS-Celeb-1M face recognition, and best paper award from TASK-CV with ICCV 2015. Dr. Feng’s current research interest focuses on AI, machine learning and computer vision.

Portfolio: How would you explain to a layman what you do, ie, enabling computers to grow learning ability with dynamic neural networks? 

Dr. Feng: Artificial Neural Network (ANN) is a computation model inspired by the biological neural networks that constitute animal brains. ANN has been widely applied in many artificial intelligence (AI) areas, like speech recognition (iPhone’s Siri), natural language processing (Google Translate) and computer vision (for example, face recognition). 

Such ANN models consist of multiple layers, and each layer processes the outputs from the previous layer and outputs to the following layer for further processing, similar to animal brain structure. The connections between adjacent layers for transferring the information are determined by machine learning algorithms, i.e., these connections are optimized on a collection of training examples. 

The learning ability of a neural network depends on the number of layers and complexity of connections between them. A network model with more layers or more connections between two layers has greater learning ability – it can learn to perform very complicated tasks as long as it has access to sufficiently many examples for training. However, over-complex network models also bring another risk – it may ‘over-fit’ unimportant clues from the training examples, e.g., it may learn to tell different faces by very detailed texture. Such clue is obviously not applicable to recognize new faces outside of the training examples.

Therefore, in the research and application of ANN, how to design its architecture (including number of layers, connections, etc.) such that the architecture is suitable for the concrete application problem is very important. The architecture cannot be too simple with insufficient learning ability or too complex to train well.

My work introduces dynamic ANN models. The architecture of such a model is dynamic and adaptive to the concrete applications. For a challenging application, like recognizing a large number of different animals, our dynamic ANN model will automatically grow the model complexity to learn to perform this task, including growing the number of layers and connections between layers. This is similar to human learning processing. When learning new knowledge, new connections between neurons will be established. When handling simple tasks (like recognizing handwritten digit numbers 0-9), the dynamic ANN model will prune unnecessary layers and connections, degenerating into a simpler model. Thus it can avoid the risk of overfitting to some trivial details.

Equipped with such dynamic ANN model, computers can automatically grow its learning ability by learning and modifying the model architectures without human interference.

From your (type of) research and field of enquiry, what exciting new things can we expect in the future? How will they help improve people’s everyday lives?

Broadly speaking, my research field is Artificial Intelligence (AI). In particular, I am working on machine learning (including deep learning) and computer vision. AI has made very impressive progress in recent years. For example, Google DeepMind’s AlphaGo defeated Ke Jie (ranking No.1 Go game player) in last year.

Go game is believed to be very challenging game for AI as it has a tremendously large search space. AlphaGo builds on a powerful neural network model to learn optimal policy for playing this game. Other emerging AI techniques that are reshaping our everyday lives include self-driving cars, an example application of computer vision techniques, AI for medical image analysis and diagnosis, such as cancer detection or tumour classification. Face recognition and video analytics have many useful applications in our daily lives: for surveillance; for unlocking your smart phone (faceID from iPhone X); or for verifying your identification at the immigration point in an airport.

What have you accomplished to date with your research? Has it been used or acquired by companies as underlying technology for apparatus, devices or products for people? Please give us a rundown with details.

My team and I, the Learning and Vision research group with Electrical and Computer Engineering at the National University of Singapore (ECE@NUS), have been working on AI, deep learning and computer vision for a decade.

Based on the deep learning models and machine learning approaches we developed, we have won multiple international challenges for image classification, object localization, and face recognition.

For example, we developed a new large-scale deep learning model which is capable for recognizing 1 million faces very fast. With this model, my team has won the MS-Celeb 1M large-scale face recognition challenge and low-shot learning challenge – our model had the highest accuracy in recognizing 1 million celebrity faces. In the past year, we also won the ImageNet 2017 object localization challenge, which is essentially the ‘Olympic Games’ in the computer vision community. The task in that competition was to localize objects from 1000 categories in images with complex background. At this juncture, I want to express my gratitude to my students and other group members for their efforts in securing these achievements.

Based on our deep learning techniques, the developed face recognition model will be adopted by Singapore’s Ministry of Home Affairs (MHA). We have finished a prototype model, and now we are planning to scale up the application to large systems, including surveillance cameras. Our face recognition and analysis techniques will enhance Singapore’s homeland security and contribute to its Smart Nation objectives. 

Our team has also developed the most accurate human pose estimation model, pedestrian detection and search model, video object segmentation model, and interactive image segmentation model, which is useful in intelligent photo editing software, like Photoshop.

How have we benefited from machine learning in general? What products or devices are we using now that have been the result of research in machine learning?

Machine learning has become very common in our daily lives and assists us in many aspects. An example is virtual personal assistants, like Siri, Alexa and Google Now. They can assist in finding information and Q&A by recognizing your question through your voice – this is applied machine learning for speech recognition. Machine learning is the core technique running these products. It helps collect and refine the information on the basis of your previous interaction with the software. 

Another application of machine learning is for big data analytics, like finance data and transportation data. For example, machine learning is applied in traffic prediction.

Recommendation systems between social media services and advertisements is also based on machine learning. Machine learning personalizes your news feed with more targeted ads, so social media platforms are using machine learning for their own and user benefits, like post recommendation, such as people you may know, similar pins.

Email spam and malware filtering also employ machine learning techniques to detect spams. Search engines such as Google, Bing or Yahoo, and online product recommendation sites like Amazon also apply machine learning. 

As mentioned above when referencing our own work, machine learning plays an important role in video surveillance and face recognition, finger print recognition, etcetera.

Dr Jiashi Feng defines following concepts – deep learning, robust learning, subspace learning, large-scale machine learning – and how they are used in computer vision, big data analytics and artificial intelligence

Deep Learning is a subfield of machine learning that is concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. It is a modern name for the artificial neural network (ANN). Deep Learning relies on many hierarchical layers to automatically extract useful data representations for target applications. For example, extracting useful information from face images to identify a person.

Robust learning is also a subfield of machine learning. Machine learning models have been applied in many products and have very close relationship with our daily lives. Your smart phone, face recognition, transportation, health trackers and apps, and even self-driving cars apply robust learning.

But machine learnings may be easily corrupted by malicious attacks. The target of robust learning is to build a machine learning model that can still output correct and desired predictions even in the presence of malicious attacks. Robust learning is very important for the safety of the application of machine learning when using such products daily.

Large-scale machine learning aims to increase the scalability of machine learning models to enable them to process and analyze large-scale data – that’s how our software can recognize a million faces.

Deep learning has been widely used in computer vision (face recognition, object detection, image retrieval, self-driving cars), and big data analytics (weather forecast, financial data analysis, fraud detection, transportation data analysis).