← Back

Enterprise-Scale Two-Factor Authentication System

Building a secure, scalable 2FA infrastructure for 30M+ users across web, mobile, and enterprise platforms

My Role
Senior Full-Stack Developer
Team
1 Product Designer · 1 Front-end Dev
1 Product Manager · 2 QA Engineers
Timeline
February - June 2023
Tech Stack
PHP · MySQL · Redis · React · REST APIs

As the lead backend developer on this project, I architected and implemented Jotform's platform-wide Two-Factor Authentication system — addressing 2,000+ user requests and meeting enterprise-grade security requirements (HIPAA, SOC 2) while maintaining high availability for 300K–400K daily active users.

The Technical Challenge

Build a 2FA system that could handle millions of authentication requests while integrating seamlessly with existing authentication flows across multiple platforms — without introducing latency or single points of failure.

01

Implement TOTP-based authentication with RFC 6238 compliance

02

Design secure backup code generation and storage system

03

Build admin enforcement APIs for enterprise customers

04

Integrate with existing OAuth and SSO authentication flows

My Technical Contributions

  • Backend Architecture: Designed and implemented the core 2FA service layer in PHP with distributed caching via Redis
  • TOTP Implementation: Built RFC 6238-compliant time-based OTP generation and validation with configurable time windows
  • Secure Storage: Implemented encrypted secret key storage with HSM integration for enterprise deployments
  • Recovery System: Designed backup code generation using cryptographically secure random number generation
  • API Development: Created RESTful APIs for 2FA management consumed by web, mobile, and enterprise admin consoles
  • Database Design: Optimized MySQL schema for high-throughput authentication queries
System DesignBackend DevelopmentAPI DesignSecurity ImplementationPerformance OptimizationIntegration

System Architecture

I designed a modular architecture that separated concerns and ensured the 2FA system could be independently scaled:

Core Components

  • TOTP Service: Handles code generation and validation
  • Secret Manager: Encrypted storage and retrieval of user secrets
  • Recovery Service: Backup code generation and one-time use validation
  • Enforcement Engine: Enterprise policy management and compliance

Infrastructure Decisions

  • Redis: Rate limiting and temporary code caching
  • MySQL: Persistent storage with read replicas
  • Encryption: AES-256 for secrets at rest
  • Audit Logging: Complete authentication event trail

Implementation Highlights

TOTP Algorithm Implementation

Built RFC 6238-compliant TOTP validation with a configurable time step (30 seconds default) and drift tolerance to handle clock skew between client devices and servers.

HMAC-SHA1 GenerationDynamic TruncationTime Window ValidationReplay Attack Prevention

Backup Code System

Implemented one-time use recovery codes with secure generation using cryptographically secure PRNGs. Each user receives 10 codes at setup, stored as salted hashes.

Secure Random GenerationBcrypt HashingSingle-Use EnforcementRegeneration Flow

Enterprise Enforcement API

Built comprehensive admin APIs allowing enterprise customers to enforce 2FA policies at organization, team, or individual user levels with grace periods and compliance reporting.

Policy CRUD EndpointsBulk User OperationsCompliance ReportsAudit Trail APIs

Security Measures

  • Rate Limiting: Implemented progressive delays after failed attempts to prevent brute force attacks
  • Replay Prevention: Each TOTP code can only be used once within its time window
  • Secret Encryption: User secrets encrypted at rest using AES-256-GCM
  • Session Binding: 2FA verification tied to session to prevent token reuse
  • Audit Logging: All 2FA events logged for security analysis and compliance

Performance Optimization

  • Redis Caching: Reduced database load by caching user 2FA status with 5-minute TTL
  • Connection Pooling: Optimized database connections for high-concurrency scenarios
  • Async Processing: Email notifications and audit logs processed asynchronously
  • Query Optimization: Indexed queries achieving sub-millisecond response times

Monitoring & Observability

Implemented comprehensive monitoring to track system health and user behavior:

  • 29 custom events for tracking 2FA interactions and failures
  • Real-time dashboards for authentication success rates and latency
  • Automated alerts for unusual failure patterns or potential attacks
  • Integration with FullStory for user experience analysis

Platform Integration

Integrated the 2FA system across all authentication touchpoints:

Web AppReact-based settings UI with QR code generation
Mobile AppsiOS and Android native integration via REST APIs
Enterprise ServerWhite-labeled deployment with custom branding
Admin ConsoleEnterprise management dashboard for IT admins
OAuth Flows2FA step injection into existing SSO pipelines
Public API2FA enforcement for API key authentication

Post-Launch Iteration

After launch, I addressed issues identified through monitoring:

  • Clock Drift Handling: Expanded time window tolerance after detecting validation failures from devices with inaccurate clocks
  • Recovery Flow Optimization: Built secure identity verification for users who lost both their device and backup codes
  • Rate Limit Tuning: Adjusted thresholds based on real-world usage patterns

Impact & Results

300-400K
Daily logins through 2FA
2,500+
Enterprise servers
30M+
Platform users secured
99.9%
API uptime
  • Zero security incidents related to 2FA implementation post-launch
  • Sub-50ms average response time for 2FA validation endpoints
  • Successfully passed SOC 2 and HIPAA compliance audits
  • Deployed across 2,500+ enterprise server installations