C++ API

The APIs from all other languages (Python, Java, C#…) are generated from the C++ code using SWIG. It’s straightforward to use the API from different languages based on this documentation as threre are only #3 useful functions (init, process and deinit).

The header file to include is defined at https://github.com/DoubangoTelecom/ultimateFace-SDK/blob/master/cpp/ULTFACE-SDK-API-PUBLIC.h

enum UltFace::ULTFACE_SDK_IMAGE_TYPE

Defines the image types.

Values:

ULTFACE_SDK_IMAGE_TYPE_RGB24

Each pixel is stored on 3 bytes. Each channel (R, G, B) is stored with 8 bits of precision (256 possible values). Here is how the pixels are packed: 

const int pixel = (B & 0xff) << 16 | (G & 0xff) << 8 | (R & 0xff);

ULTFACE_SDK_IMAGE_TYPE_RGBA32

Each pixel is stored on 4 bytes. Each channel (R, G, B, A) is stored with 8 bits (1 byte) of precision (256 possible values). The R channel is stored at the lowest memory address followed by G, B then A channels. If you’re using Android then, this is the same as ARGB_8888. Here is how the pixels are packed: 

const int pixel = (A & 0xff) << 24 | (B & 0xff) << 16 | (G & 0xff) << 8 | (R & 0xff);

ULTFACE_SDK_IMAGE_TYPE_BGRA32

Each pixel is stored on 4 bytes. Each channel (B, G, R, A) is stored with 8 bits (1 byte) of precision (256 possible values). The B channel is stored at the lowest memory address followed by G, R then A channels. If you’re using iOS then, this is the same as kCVPixelFormatType_32BGRA. Here is how the pixels are packed: 

const int pixel = (A & 0xff) << 24 | (R & 0xff) << 16 | (G & 0xff) << 8 | (B & 0xff);

ULTFACE_SDK_IMAGE_TYPE_BGR24

Each pixel is stored on 3 bytes. Each channel (B, G, R) is stored with 8 bits (1 byte) of precision (256 possible values). The B channel is stored at the lowest memory address followed by G then R channels. If you’re using C# then, this is the same as PixelFormat.Format24bppRgb. Here is how the pixels are packed: 

const int pixel = (R & 0xff) << 16 | (G & 0xff) << 8 | (B & 0xff);

ULTFACE_SDK_IMAGE_TYPE_NV12

YUV 4:2:0 image with a plane of 8 bit Y samples followed by an interleaved U/V plane containing 8 bit 2x2 subsampled colour difference samples. More information at https://www.fourcc.org/pixel-format/yuv-nv12/

ULTFACE_SDK_IMAGE_TYPE_NV21

YUV 4:2:0 image with a plane of 8 bit Y samples followed by an interleaved V/U plane containing 8 bit 2x2 subsampled chroma samples. The same as ULTFACE_SDK_IMAGE_TYPE_NV12 except the interleave order of U and V is reversed. More information at https://www.fourcc.org/pixel-format/yuv-nv21/

ULTFACE_SDK_IMAGE_TYPE_YUV420P

These formats are identical to YV12 except that the U and V plane order is reversed. They comprise an NxM Y plane followed by (N/2)x(M/2) U and V planes. This is the format of choice for many software MPEG codecs. More information at https://www.fourcc.org/pixel-format/yuv-i420/

ULTFACE_SDK_IMAGE_TYPE_YVU420P

Same as ULTFACE_SDK_IMAGE_TYPE_YUV420P except the order of U and V is reversed. More information at https://www.fourcc.org/pixel-format/yuv-yv12/

ULTFACE_SDK_IMAGE_TYPE_YUV422P

YUV 4:2:2 image with an NxM Y plane followed by (N/2)x(M) V and U planes.

ULTFACE_SDK_IMAGE_TYPE_YUV444P

YUV 4:4:4 image with an NxM Y plane followed by NxM V and U planes.

ULTFACE_SDK_IMAGE_TYPE_Y

Grayscale image with single channel (luminance only). Each pixel is stored in single byte (8 bit Y samples).

struct UltFaceImageInfo

Abstract class representing image information.

Subclassed by UltFace::UltFaceImageInfoCompressed, UltFace::UltFaceImageInfoUncompressed

Public Types

enum Type

Image type

Values:

Compressed
Uncompressed

Public Functions

virtual bool isValid() const = 0

checks whether the image info is valid or not.

bool isCompressed() const

checks whether the image info represents compressed data or not.

Type type() const

the type of the image info.

struct UltFaceImageInfoCompressed : public UltFace::UltFaceImageInfo

Class representing compressed (png, jpeg…) image information.

See

UltFaceImageInfoUncompressed

Public Functions

UltFaceImageInfoCompressed(const void *data_ptr, const size_t &data_size)

Constructs compressed (jpeg, png…) image information.

Parameters

  • data_ptr: pointer to the compressed data.

  • data_size: size of the compressed data in bytes.

const void *data_ptr() const

pointer to the compressed data.

size_t data_size() const

size of the compressed data in bytes.

virtual bool isValid() const

checks whether the image info is valid or not.

struct UltFaceImageInfoUncompressed : public UltFace::UltFaceImageInfo

Abstract class representing uncompressed image information. An uncompressed image could be of RGB (UltFaceImageInfoRgbFamily) or YUV (UltFaceImageInfoYuvFamily) family. For compressed image (jpeg, png…), use UltFaceImageInfoCompressed

See

UltFaceImageInfoUncompressed

Subclassed by UltFace::UltFaceImageInfoRgbFamily, UltFace::UltFaceImageInfoYuvFamily

Public Functions

ULTFACE_SDK_IMAGE_TYPE chroma() const

image chroma

size_t widthInSamples() const

image width in samples

size_t heightInSamples() const

image height in samples

int exifOrientation() const

jpeg exif orientation (within [1,7])

virtual bool isValid() const

checks whether the image info is valid or not.

bool isRgbFamilyType() const

checks whether the image info represents RGB data or not.

struct UltFaceImageInfoRgbFamily : public UltFace::UltFaceImageInfoUncompressed

Class representing RGB-family (rgb, rgba, bgr…) image information

See

UltFaceImageInfoYuvFamily

Public Functions

UltFaceImageInfoRgbFamily(const ULTFACE_SDK_IMAGE_TYPE &chroma, const void *rgbPtr, const size_t &widthInSamples, const size_t &heightInSamples, const size_t &strideInSamples = 0, const int &exifOrientation = 1)

Constructs RGB (rgb, bgr, rgba…) image information.

Parameters

  • chroma: image chroma.

  • rgbPtr: image data pointer.

  • widthInSamples: image width in samples.

  • heightInSamples: image height in samples.

  • strideInSamples: image stride in samples. Should be zero unless your the data is strided.

  • exifOrientation: jpeg exif orientation (within [1,7]). The engine will rotate the image based on the orientation info. More info at https://jdhao.github.io/2019/07/31/image_rotation_exif_info.

size_t strideInSamples() const

image stride in samples

const void *rgbPtr() const

image data pointer

virtual bool isValid() const

checks whether the image info is valid or not.

struct UltFaceImageInfoYuvFamily : public UltFace::UltFaceImageInfoUncompressed

Class representing YUV-family (yuv420, nv12, nv21, y…) image information

See

UltFaceImageInfoRgbFamily

Public Functions

UltFaceImageInfoYuvFamily(const ULTFACE_SDK_IMAGE_TYPE &chroma, const void *yPtr, const void *uPtr, const void *vPtr, const size_t &widthInSamples, const size_t &heightInSamples, const size_t &yStrideInBytes, const size_t &uStrideInBytes, const size_t &vStrideInBytes, const size_t &uvPixelStrideInBytes = 0, const int &exifOrientation = 1)

Constructs YUV (yuv420, nv12, nv21, y…) image information.

Parameters

  • chroma: image chroma.

  • yPtr: pointer to the start of the Y (luma) samples.

  • uPtr: pointer to the start of the U (chroma) samples.

  • vPtr: pointer to the start of the V (chroma) samples.

  • widthInSamples: image width in samples.

  • heightInSamples: image height in samples.

  • yStrideInBytes: image stride in bytes for the Y (luma) samples.

  • uStrideInBytes: image stride in bytes for the U (chroma) samples.

  • vStrideInBytes: image stride in bytes for the V (chroma) samples.

  • uvPixelStrideInBytes: image pixel stride in bytes for the UV (chroma) samples. Should be 1 for planar and 2 for semi-planar formats. Set to 0 for auto-detect.

  • exifOrientation: jpeg exif orientation (within [1,7]). The engine will rotate the image based on the orientation info. More info at https://jdhao.github.io/2019/07/31/image_rotation_exif_info.

const void *yPtr() const

pointer to the start of the Y (luma) samples

const void *uPtr() const

pointer to the start of the U (chroma) samples

const void *vPtr() const

pointer to the start of the V (chroma) samples

size_t yStrideInBytes() const

image stride in bytes for the Y (luma) samples

size_t uStrideInBytes() const

image stride in bytes for the U (chroma) samples

size_t vStrideInBytes() const

image stride in bytes for the V (chroma) samples

size_t uvPixelStrideInBytes() const

image pixel stride in bytes for the UV (chroma) samples

virtual bool isValid() const

checks whether the image info is valid or not.

struct UltFaceSdkParallelDeliveryCallback

Abstract class to be used to get asynchronous notifications.

Public Functions

virtual void onNewResult(const UltFaceSdkResult *newResult) const = 0

Notification function to override in order to receive the results.

class UltFaceSdkResult

Result returned by the engine at initialization, deInitialization and processing stages.

Public Functions

int code() const

The result code. >=0 if success, <0 otherwise.

const char *phrase() const

Short description for the code.

const char *json() const

The result as JSON string. May be null or empty if no face found.

const size_t numFaces() const

Number of faces in json string. This is a helper function to quickly check whether the result contains faces without parsing the json string.

bool isOK() const

Whether the result is success. true if success, false otherwise.

class UltFaceSdkEngine

The Face SDK engine.

Public Static Functions

static UltFaceSdkResult init(const char *jsonConfig, const UltFaceSdkParallelDeliveryCallback *parallelDeliveryCallback = nullptr)

Initializes the engine. This function must be the first one to call.

Return

a result

Parameters

static UltFaceSdkResult deInit()

DeInitializes the engine. This function must be the last one to be call. Deallocates all the resources allocated using init function.

Return

a result

static UltFaceSdkResult process_liveness(const UltFaceImageInfo *image)

Performs face liveness detection. The full liveness pipeline includes avant-garde, deepfake detection and identity concealment detection.

Sample code:

Return

a result

Parameters

static UltFaceSdkResult process_inject(const UltFaceImageInfo *stereo_main, const UltFaceImageInfo *stereo_aux, const char *aggressive_mode = "auto")

Performs stream inject check (a.k.a virtual camera detection). Requires a stereo image (main and auxiliary) with some requirements.

Sample code:

Return

a result

Parameters

static UltFaceSdkResult process_recognition(const UltFaceImageInfo *image)

Performs face recognition.

Sample code:

Return

a result

Parameters

static int exifOrientation(const void *jpegMetaDataPtr, const size_t jpegMetaDataSize)

Retrieve EXIF orientation value from JPEG meta-data.

Return

Image’s EXIF/JPEG orientation. Must be within [1, 7]. More information at https://jdhao.github.io/2019/07/31/image_rotation_exif_info.

Parameters

  • jpegMetaDataPtr: Pointer to the meta-data.

  • jpegMetaDataSize: Size of the meta-data.

static UltFaceSdkResult requestRuntimeLicenseKey(const bool &rawInsteadOfJSON = false)

Build a unique runtime license key associated to this device. You must initialize the engine before calling this function. This function doesn’t require internet connection. The runtime key must be activated to obtain a token. The activation procedure is explained at https://www.doubango.org/SDKs/LicenseManager/docs/Activation_use_cases.html.

Return

a result

Parameters

  • rawInsteadOfJSON: Whether to output the runtime key as raw string intead of JSON entry. Requesting raw string instead of JSON could be helpful for applications without JSON parser to extract the key.

Fork me on GitHub