Customer Support Using Multimodal AI Infrastructure

Factored helped a digital bank cut human support needs by 60% using multimodal AI, scaling global service while maintaining security and speed.

Key Takeaways:

Scaled with AI: 70% automation on text chats and 30% on image chats drastically reduced human workload.
Fast, secure, multilingual: Deployed real-time, compliant AI systems across channels in multiple languages.
Built for iteration: Agentic LLM architecture + A/B testing enabled ongoing model improvements and high reliability.

A rapidly growing U.S.-based digital bank with a global customer base but no physical branches. All customer interactions are conducted through digital channels, including mobile apps, websites, email, and phone. With over $1 billion in assets under management, delivering seamless, responsive support at scale was critical to maintaining customer satisfaction and trust.

1:1 Human Interaction Is Expensive

As the bank expanded globally, it faced a major challenge: its customer service operations could not scale quickly enough using human agents alone. With rising volumes of inquiries—averaging thousands of chats daily—it became clear that:

  • Hiring human agents was cost-prohibitive: U.S. salaries for customer support roles can range from $40,000–$50,000 annually or $28–$38/hour in call centers.
  • Customers required 24/7 multilingual support (English, Spanish, Mandarin, etc.), which was difficult to staff.
  • There was a growing “no-reply” rate to service requests.

The bank needed to scale its support team across digital platforms—without relying solely on human agents.

AI-Powered Support Can Manage Most Queries

Factored implemented a three-pillar AI-powered system that combined advanced infrastructure, software engineering, and applied AI/ML to automate customer interactions efficiently and securely.

Pillar 1: Infrastructure

  • Kubernetes-based architecture with two environments (staging and production) enabled fast iteration and high system reliability.
  • Enabled full automation pipelines with A/B testing for new models and deployment strategies.

Pillar 2: Frontend & Backend Engineering

  • Real-time communication enabled via real-time APIs.
  • Built using Python and high performance APIs, ensuring scalability and rapid deployment of updates.
  • Integrated with existing digital banking APIs to access customer profiles, loan status, payment history, etc., in real-time.

Pillar 3: AI & Automation

  • Deployed a multimodal LLM architecture using APIs from 3 industry leading models.
  • One of the models were for image-to-text parsing to optimize accuracy based on context:
    • Document validation.
    • Payment collections.
    • Escalation routing to agents.

AI Chatbot Capabilities

  • Intent detection: Identifies the primary intent (e.g., loan disbursement, payment delays, etc.)
  • Function calling: Handles critical branching decisions—e.g., automating minor issues or escalating users with critical needs to 1:1 human support.
  • Language understanding: Processes multilingual input.
  • Structured validation: Uses a second LLM call to verify model outputs and prevent hallucinations.
  • Agentic architecture: Each "sub-agent" handles a specific intent and performs actions such as escalating to a human, closing the chat, or redirecting to a specialized agent.

Evaluation & Testing:

  • We created mock conversations and replayed past transcripts in staging to evaluate new models.
  • Compared responses using:
    • Baseline (long prompt with manual logic).
    • Agentic approach (LLM chooses from small prompts/sub-agents).
  • A/B testing conducted to optimize response accuracy and automation rate.

+60% Total Reduction 1:1 Human Interactions

  • Achieved 70% automation on text-based chats, and 30% automation on chats with image—critical in reducing overhead and wait times.
  • Maintained compliance and security standards by avoiding sensitive data exposure to LLMs (all personal information pulled through secure APIs).
  • Decreased customer service costs significantly by reducing the need for new hires and minimizing errors in multilingual settings.

Through a robust AI-driven approach, the U.S.-based digital bank successfully scaled its global customer support operations—achieving high automation rates, reducing operational costs, and delivering faster, round-the-clock service to users worldwide. Factored's three-pillar solution ensured rapid iteration, accuracy, and adaptability in a highly regulated financial environment.

Skills
No items found.
Roles
No items found.

Continue Reading

Databricks Based Gen AI Personalized Recommendation
Learn More >
Databricks Based LLMs for Market Research
Learn More >
Reducing Manufacturing Downtime with Predictive Analytics
Learn More >
Want to discuss a solution for you?
Get in touch
Exceptional talent at lower costs
Start interviewing within one week
Have talent placed in under a month