When you were a kid, you most probably went to a theme park with your parents. Remember how the cheerful entertainers photographed everyone and offered to buy the printed pics later? We cared little about privacy then. Were these pictures then transferred to Interpol and FR-ed by early face recognition prototypes? Or kept for future reference?..

Facial recognition is raising privacy concerns more and more around the globe. Especially with the technology being available on billions of smartphones, and the data going to many different private companies. In some places like San Francisco, it is even banned from use in public places. So why is FR chosen over other biometric solutions and other security techniques in general?

To answer this, we need to look at facial recognition through the lens of security theory and compare it to other options at hand.

Security Background

Essentially, authentication has three levels, each answering its own question:

1) What do you know?

2) What do you have?

3) What are you?

The first level is the simplest and the most vulnerable. It may be a password, a code, an answer to a security question - something that can be easily cracked, forgotten or socially engineered. Still, we use it in so many places, as it is primitive, and does not require any special equipment.

The second factor is typically a thing that you physically own. Like a key or a keycard, a token, or a phone - something you most likely use for 2-FA to log into your internet bank. Second level security is obviously better, but you can still lose the key, or someone may steal your SIM card - even without taking your phone.

Ultimately, the third tier is something that we see at the beginning of the Bourne Identity movie. An almost naked man remembers nothing of his life, yet he walks into the bank and is immediately identified and authenticated. (What would Jason do if they asked for a password, huh?) Substantially, this security level includes biometrics - unique human features that are hard to change - and is more complicated to implement and to fool. Some of the biometric methods require fancy equipment (like iris, palm, vein scanning). Others are simpler and can be added on top of most existing security systems. Analytics like face or voice recognition need sophisticated software but only require a routine hardware upgrade.

Practice shows, the most efficient way is to combine at least two security factors. For top-level security, use all three factors on different stages or for access to different resources.

Why FR

Of all third level techniques, facial recognition happens to be the most suitable option. Why?

• It is relatively easy to implement (e.g., compared to gait analysis, which is a favorite in spy movies yet is very challenging as of this moment)
• It does not require additional equipment (unlike a fingerprint, palm print, or retina scanning) and is compatible with existing CCTV systems
• FR does not require human interaction > quick, real-time, invisible, proactive, and also safe, which is especially important now, during the pandemic
• It makes identity theft very expensive and troublesome

While FR accuracy with modern algorithms is quite high, you can still deceive artificial intelligence. The techniques to fool FR are mostly complicated and therefore expensive - camouflage makeup, morphed portraits, specially designed masks - and FR engine manufacturers constantly invent countermeasures for these. If the false positives are unacceptable in the target scenario, e.g., for access control in critical locations, one should use FR in combination with other authentication methods.

What Makes a Good FR

So, how do you choose a good face recognition solution? Like with any other product: explore the market and ask experts.

Having compared many face recognition engine vendors and solution providers, we have collected some nuggets of wisdom. And, we have applied that information when developing our own face recognition module.

Choose Top-Notch Technologies

Early facial recognition algorithms tended to be based on facial features. New generation FR engines rely on neural networks(NN): these are faster, lighter, and more accurate at the same time. They also good on partial faces (which includes people wearing masks) and faces at big angles. Also, NNs are forgiving to changing facial expressions, glasses, different haircuts, and beards. Modern FR algorithms also refute the superstition about racial/gender bias.

Present FR technologies also put forth other approaches. Amongst them - 3D face modeling, which is light source independent and very good at large angles. Another very interesting FR technique uses thermal images instead of traditional video. Thermal cameras ignore visual interference like glasses or makeup and can operate at ultra-low light. The main drawback of this approach is the lack of thermal image databases, never mention the cost of a decent infrared imager. But maybe your next ID card will have both pictures!

Live FR events with details and estimators

Luxriot Vero Face Recognition handles traditional images and is based on neural networks, featuring all the benefits of the cutting-edge FR technologies. Here, at Luxriot, we have had experience with older algorithms with our first generation FR software. We do know the difference. One can compare it to exchanging your Honda for a Tesla. After trying the cool new thing, you will suffer each time you use the old one. And those who never tried should book a test drive!

Check Independent Sources

Each software manufacturer thinks his own geese swans, and there is little you can do about it. What you can do is do a bit of research. Several independent institutions test the face recognition algorithms. One of the most prominent and respected ones is NIST (National Institute of Standards and Technology of the United States) with their FRVT, which stands for Face Recognition Vendor Test. They test a lot of metrics and make the results public, so you can see for yourself how your chosen FR performs in different scenarios.

The face recognition engine behind Luxriot Vero FR is #2 in the NIST FRVT 1:1 verification leaderboard and delivers outstanding results in the 1:N test (one-to-many matching). You get a super low false-positive match rate, top performance, and sky-high accuracy.

Decide on Dependencies and Features

Consider software and hardware requirements. Some of the face recognition solutions only run on specific hardware or need a costly GPU unit. Other ones have a complicated licensing model that makes you pay for extra functionality. Imagine a separate fee for mobile application alerts, age estimation, or subject enrolment. Some solutions are completely standalone while others require a fully functional video management system.

Consider where the load will go. Some solutions offer the calculations to be done on edge (camera-side FR), some require specific equipment (e.g., powerful GPU), and some are happy to use a generic computer.

While the Luxriot facial recognition module is not independent and operates only in joint with the Luxriot Evo video management platform, it offers all the benefits of an integrated solution. Its license limits the number of faces in the subjects' database, allowing you to have as many channels as you want. Besides, Luxriot Vero FR works with both Evo S and Evo Global versions, meaning it will suit both small installations and large enterprises with branch offices.

FR event search based on image: high accuracy even at a big angle

Featuring a lightweight Web UI, Luxriot FR does not require a stack of video cards, solely relying on the CPU. You can re-use existing hardware for the face recognition software, and then also easily re-purpose the ex-FR server for other needs. Leave the graphics cards for gaming and mining bitcoins!

Last but not least: Evo works with thousands of devices, which means you can use almost any video source for running the facial recognition. Of course, there are certain requirements for the video quality and camera placement, but you will save some money by not buying specialized cameras.

Look for Extras

So you have narrowed your choice down to three or four options. Similar performance, good price, easy setup. But maybe one of them fits your use case better than the other ones? What is critical for your project? Perhaps it is the ability to work with different types of databases because the law enforcement will have to search the local, country-wide, and international DBs. Or, maybe, your client is a bank that wants customer analysis reports. Those by age and gender, additionally combined with people counting analytics. It is good to know what extra options your solution provider can offer to anticipate the customer's needs.

Live recognition, database search, all kinds of alerts and alarms, including those to/from paired access control suites - this is only a part of what you get with Luxriot Vero FR. Apart from the basic face identification and matching, it offers multiple estimators - age, gender, sentiment, - with mask detection coming next. An important feature in the pandemic and post-covid world is the combined reading of the face+temperature, with an option to search the temperature reading history.

Combined FR and temperature readings in Luxriot Evo Monitor

Through Evo Event & Action engine, the Luxriot FR solution offers a gateway to dozens of embedded and third-party events, actions, analytics, access control modules, allowing you to build a flexible solution for different verticals.

Where to Apply

A system with facial recognition indeed costs more compared to traditional CCTV. But "traditional video recorder" solutions quickly become obsolete because no one just sits there watching terabytes of video recordings. We need the data to make sense, hence all the analytics and event-driven approach.

There is a multitude of markets for FR. Basically, it comes useful wherever there is a need for automatic and instant identity verification:

• security, government, law enforcement: police investigations, border checks, voting, control access to sensitive areas, staff attendance systems
• smart city: find missing persons, runaway patients, criminals, people banned from entering facilities (gaming areas, airlines)
• retail and banking: know your customer, face pay, self check-out, personalized ads, rectify shoplifters, recognize VIPs at events, authorize mobile payments, ATM transactions 
• ports and airports: security, speed up check-in, boarding, and border control, monitor criminals and missing persons
• schools: identify dangerous individuals or expelled students, track attendance
• casinos: authorize staff login, control age, blacklist cheaters and members of voluntary exclusion lists
• and many more!

To Use or Not to Use

Of course, it is important to make FR work to support the legal system, not compete or beat it. We must support the protection of biometric data and ensure proper usage profiles. Otherwise, more and more places will support cities like Boston and San Diego in banning FR - even for law enforcement. We will be knocked sideways without facial recognition. While probably no one will miss targeted ads, we will certainly lack all the profit it can potentially bring us by making our lives safer and easier.

Facial recognition is like a sharp knife in your kitchen. It is a powerful tool, dangerous if misused. It must be kept away from children and laymen, and may only be used with adequate precautions. And you get tons of benefits by applying it reasonably.