FaceNet

Easy face recognition, verification, and clustering.

FaceNet is a deep convolutional network designed by Google, trained to solve face verification, recognition and clustering problem with efficiently at scale.

1. directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.
2. optimize the embedding face recognition performance using only 128-bytes per face.
3. achieves accuracy of 99.63% on Labeled Faces in the Wild (LFW) dataset, and 95.12% on YouTube Faces DB.

Example

TL;DR: Talk is cheap, show me the code!

import { Facenet, FaceImage } from 'facenet'

const facenet = new Facenet()

// Load image from file
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`
const image = new FaceImage(imageFile)

// Do Face Alignment, return faces
const faceList = await facenet.align(image)

for (const face of faceList) {
  // Calculate Face Embedding, return feature vector
  const embedding = await facenet.embedding(face)
  assert(face.embedding === embedding)

  console.log('bounding box:',  face.boundingBox)
  console.log('landmarks:',     face.facialLandmark)
  console.log('embedding:',     face.embedding)
}

Full source code can be found at here: https://github.com/zixia/facenet/blob/master/examples/demo.ts

Try it by run:

$ git clone [email protected]:zixia/facenet.git
$ cd facenet
$ npm install
$ npm run demo

Output

1. Console

TODO:

1. Files

X(file1)  <-- 0.5 --> X(file2)
   ^                    ^
   |
   1.1
   |
   v                    v
Y(file1)  <-- 0.4 --> Y(file2)

Requirement

OS

Now I'm developing under Ubuntu 17.04

Should support all platforms including Linux/Mac/Win32 in the future version.

Dependency

1. Node.js >= 6 (8 is recommend)
2. Tensorflow >= 1.2
3. Python >=3.5 (3.6 is recommend)

Ram

Neural Network Model	Task	Ram
MTCNN	Facenet#align()	100MB
Facenet	Facenet#embedding()	2GB

If you are dealing with very large images(like 3000x3000 pixels), there will need additional 1GB of memory.

So I believe that Facenet will need at least 2GB memory, and >=4GB is recommended.

API

1. Facenet

import { Facenet } from 'facenet'

const facenet = new Facenet()
facenet.quit()

1. Facenet#align(image: FaceImage): Promise<Face[]>

Do face alignment for the image, return a list of faces.

2. Facenet#embedding(face: Face): Promise

Get the embedding for a face.

2. Face

Get the 128 dim embedding vector for this face.(After alignment)

import { Face } from 'facenet'

console.log('bounding box:',  face.boundingBox)
console.log('landmarks:',     face.facialLandmark)
console.log('embedding:',     face.embedding)

face.image.save('/tmp/face.jpg')

3. FaceImage

import { FaceImage } from 'facenet'

const image = new FaceImage('tests/fixtures/two-faces.jpg')
image.resize(160, 160)
image.save('/tmp/image.jpg')

Environment Variables

FACENET_MODEL

FaceNet neural network model files, set to other version of model as you like.

Default is set to models/ directory inside project directory. The pre-trained models is come from 20170512-110547, 0.992, MS-Celeb-1M, Inception ResNet v1, which will be download & save automatically by postinstall script.

$ pwd
/home/zixia/git/facenet

$ ls models/
20170512-110547.pb
model-20170512-110547.ckpt-250000.index
model-20170512-110547.ckpt-250000.data-00000-of-00001
model-20170512-110547.meta

Develop

$ git clone [email protected]:zixia/facenet.git
$ cd facenet
$ npm install
$ npm test
$ npm run demo

Command Line Interface

align

Draw a rectangle with five landmarks on all faces in the input_image, save it to output_image.

$ ./node_modules/.bin/ts-node bin/align.ts input_image output_image

embedding

Output the 128 dim embedding vector of the face image.

$ ./node_modules/.bin/ts-node bin/embedding.ts face_image

Resources

Machine Learning

• Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning
• Face recognition using Tensorflow
• Google: Our new system for recognizing faces is the best one ever
• A tensorflow implementation of "Deep Convolutional Generative Adversarial Networks
• What does Locality Sensitive Hashing Forests do? · maheshakya/my_first_project Wiki
• Average Face : OpenCV ( C++ / Python ) Tutorial

Python

• Google Python Style Guide
• PyLint, PyChecker or PyFlakes?
• Useful Python Modules: Flake8
• PEP 8 - Style Guide for Python Code
• Python 3.6 venv — Creation of virtual environments

Typing

• Mypy syntax cheat sheet (Python 3)
• Python 3 Type Hints and Static Analysis
• typing — Support for type hints

NumJS

• Stackoverflow: numpy-like package for node
• Read/manipulate/display images using NumJs
• Numjs - Like NumPy, in JavaScript
• ndarray - Modular multidimensional arrays for JavaScript

Todo

1. NPM Module: facenet
2. Docker Image: zixia/facenet
3. TensorFlow Sereving
4. OpenAPI Specification(Swagger)
5. Examples

Inspiration

This repository is heavily inspired by the following implementations:

• FaceNet by David Sandberg @davidsandberg
• OpenFace by CMU Satya Lab @cmusatyalab

Credits

1. Face alignment using MTCNN: Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
2. Face embedding using FaceNet: FaceNet: A Unified Embedding for Face Recognition and Clustering
3. TensorFlow implementation of the face recognizer: Face recognition using Tensorflow

Author

Huan LI <[email protected]> (http://linkedin.com/in/zixia)

Copyright & License

• Code & Docs © 2017 Huan LI <[email protected]>
• Code released under the Apache-2.0 License
• Docs released under Creative Commons