Skip to main content

Performance Metrics, Bias & Diversity

Discover the methods employed to gauge user success and explore how Facia attains top-tier performance, even in highly diverse populations.

Diversity in Facia Training Dataset​

Ensuring the effectiveness of Liveness Detection and Face Matching is paramount, transcending the confines of phone costs or specific skin tones. Our commitment is to deliver a seamless experience for all users, regardless of the device they use or their diverse range of skin tones. The aim is to foster inclusivity and accessibility, making these essential features accessible to everyone without bias based on economic factors or ethnicity. This commitment not only enhances the user experience but also reflects our dedication to providing equitable and reliable services to a broad user base.

Our Expansive Training Sets​

Our expansive training sets are derived from millions of 3D FaceMaps contributed by users globally. This diverse dataset includes information from every compatible iOS device ever produced, spanning over thousands unique Android device models, and encompassing a variety of webcam models, some with very low resolutions, going down to 0.3 megapixels. This comprehensive collection ensures the robustness and adaptability of our technology across a broad spectrum of devices and specifications.

Each month, multiple users securely establish accounts using Facia, achieving an impressive 98%-99% success rate for first-time users. Our ongoing commitment involves dedicated efforts to enhance both security and usability, striving for even higher levels of performance in these critical aspects.

Success Rate for First-time Mobile Users​

The users who are using the mobile devices for testing out Facai’s solution must have to make sure the proper lighting conditions and have to be on the latest version of the SDKs to achieve the below levels:

  • ~90-95% - users get approved in the first liveness attempt.
  • ~3% - of the users perform an unsuccessful Session on their first try, and are shown retry guidance, and then succeed on their second session.
  • ~1-2% - Perform another unsuccessful User Session due to an oversight of the guidance provided, the system displays the guidelines again and then succeeds in the session.

A small percentage, specifically 1-2%, encompasses instances such as spoof attacks, users donning sunglasses who decline to remove them, or individuals opting out of the process. Despite these challenges, our extensive real-world sessions have shown consistent performance across various scenarios. Notably, there haven't been discernible differences in the effectiveness of our system when used on low-end mobile devices. This underscores the robustness of our technology in delivering reliable outcomes across diverse user interactions and device specifications.

Success Rate for First-time Web Browser Users​

For an optimal experience, users are advised to have a compatible web browser, and employ a webcam with video feed capabilities. The adherence to these requirements, coupled with occasional challenges related to incompatible face angles from USB webcams, may lead to a slightly lower initial success rate. However, real users consistently demonstrate the ability to succeed upon retries, resulting in an impressive overall first-time success rate of 90-95%, in line with the performance observed on mobile devices. This underscores the resilience and adaptability of our system across varied user environments and device configurations.

  • ~90-95% - Prove Liveness on their very first Session.
  • ~2-3% - Perform an unsuccessful Session on their first try, are shown retry guidance and then succeed on their second session.
  • ~1-2% - Perform another unsuccessful Session, see more specific guidance and then succeed in few sessions.

Within the remaining 1-2% of sessions, we encounter various scenarios, including spoof attacks, users wearing sunglasses who decline to remove them, and individuals opting to discontinue the process prematurely. It's noteworthy that across multiple real-world sessions, our system has demonstrated consistent performance, revealing no significant differences when using low-end webcams. This emphasizes the robust adaptability of our technology to various user scenarios and hardware specifications.

User Experience for Individuals with Light Skin Tones​

Cameras perform optimally in proper lighting, yet excessive light may result in an overexposed or "blown-out" appearance for individuals with lighter skin tones, particularly in bright sunlight. However, a swift 90-degree adjustment often resolves this problem by modifying the lighting. Facia consistently excels across diverse skin tones, ensuring reliable performance when the camera sensor is not overly saturated.

white-male

Example of Light Skin Tones

Skin Tones in Low-Light Environments​

Camera sensors depend on light for proper functioning, and inadequate light can impede data gathering. Consequently, individuals with darker skin tones might require higher levels of ambient light compared to those with lighter skin tones. Nonetheless, Facia, with its 3D Face capture technology, demands less light in comparison to 2D systems employing the same camera. Additionally, the device's screen, during user interaction, can supply the required illumination. In thorough testing, Facia has demonstrated consistent performance across the entire spectrum of human skin tones, irrespective of fluctuating light conditions.

black-female

Example of Dark Skin Tones

Maintain a Realistic Perspective in Your Due Diligence​

The initial user experience with Facia is typically seamless, providing a fast and intuitive verification process that allows users to quickly resume their daily activities. However, developers often engage in a "test mode," experimenting with various conditions such as adjusting lighting, facing direct sunlight, trying different angles, or partially blocking their faces. While informative, these explorations do not precisely replicate real-world scenarios. Developers conducting usability testing must challenge the limits of Facia capture UI.

It is essential, though, not to let extreme testing scenarios overly influence the overall perspective. Real users expect Facia SDKs to operate seamlessly in their everyday environments, facilitating effortless verification processes. Recognized for their user-friendly design, Facia SDKs offer significantly less friction compared to traditional passwords. It's crucial to avoid letting extreme usability testing distort the broader picture. Hundreds of millions of initial Facia users have successfully and securely accessed their accounts daily, choosing the convenience of a password-free experience over traditional methods. They have embraced this approach and show no inclination to revert to traditional banking practices or password entry.

Challenges in Conducting Liveness Checks on Testing Devices​

While the vast majority of users, around 98-99%, successfully pass the Liveness Check within their first five attempts, a few users may need two or three tries. These situations commonly occur in non-standard camera environments, leading users to adjust their surroundings until they are facing the camera under normal lighting conditions.

Facing Issues with Facia in a Challenging Environment; it's Not Working​

Facia solutions are meticulously crafted for real-world effectiveness, providing users with clear on-screen instructions that guide them through sessions for a successful 3D Liveness demonstration. The impressive 97-99% success rate for first-time users in real-world conditions can be attributed to the user-friendly design, allowing users to promptly address issues by following Retry Screen instructions and adjusting their environment.

Authentic end-users actively seek the effectiveness of Facia’s solution, following on-screen instructions. Unlike test cases, genuine users do not intentionally expose themselves to challenges or engage in repetitive attempts without making necessary adjustments. While Facia excels in diverse conditions, addressing issues such as glare or data loss is crucial, replicating the actions of a real user striving for optimal performance.

Ensuring Accessibility for Atypical Facial Conditions​

Facia is dedicated to ensuring optimal performance globally. Our algorithms, unbiased across typical skin tones, leverage 3D modeling and extensive training on a worldwide dataset to handle diverse challenges. However, atypical facial conditions, such as obstructions, birth conditions, paralysis, scarring, structural effects, and skin conditions, pose challenges for the Liveness AI. Factors like heavy makeup add complexity to distinguishing real users from mask users. While our training covers some conditions, the myriad combinations of atypical traits can make confident Liveness decisions challenging. Users with atypical features may encounter a higher False Reject Rate, potentially affecting successful Liveness verification with Facia.