Fujigo

Vivi Voice AI — AI Voice Assistant for Tourism & Care

Solution Overview

International tourists often face language barriers when asking for directions, checking ticket prices, or finding opening hours at tourist spots. Information desks don't operate 24/7, and online information may be outdated or unverified. Elderly people living alone need daily medication reminders and conversation, but not every family has someone nearby.

Vivi Voice AI solves both problems. Users simply ask in Japanese, English, Vietnamese, Chinese, or Korean and receive natural voice responses with ticket prices and opening hours verified from official sources within 90 days. When sources are unavailable, the assistant clearly states «please verify on-site» — it never makes up numbers. One platform runs on three device types: mobile app (iOS/Android), lobby kiosk, and compact ESP32 device. All content is centrally managed through a CMS with four-eyes approval, ensuring accuracy before reaching users.

Detailed Capabilities

Natural Voice Conversation with Interruption Support

Response time under 2 seconds (p95) after the user finishes speaking. If the user interrupts during playback, the assistant stops within 0.3 seconds — as natural as talking to a real person. Speech recognition and voice output use Google Cloud Chirp 3 / Chirp 3 HD; the language model is Claude (Anthropic); knowledge search uses Google Vertex AI embedding on PostgreSQL + pgvector.

Sourced Information, No Fabricated Numbers

Ticket prices and opening hours are displayed only when verified from official sources within 90 days. Answers containing numbers always include sources; when sources are missing, the assistant says «please verify on-site» instead of guessing. Current data: nearly 700 locations in Kyoto, translated and loaded into the AI knowledge base.

Multi-Device, Single Content

App, kiosk, and ESP32 share the same Voice AI service: edit once on CMS, available on all devices. iOS/Android app built with Flutter, tourists use it anonymously without registration. Kiosk runs on Orange Pi 5B with 1280×800 display and ReSpeaker microphone with echo cancellation. ESP32-S3 with 16MB flash serves as either a tourist robot or companion speaker for elderly care.

Four-Eyes Approval, Tamper-Proof Audit Log

Content creators cannot approve their own work — this rule is enforced at the database level, not just by hiding buttons in the UI. Machine translations don't go live automatically: a speaker of that language must approve. Audit logs are append-only, hash-chained for tamper resistance, retained for 5 years. CMS interface available in 3 languages: Japanese, English, Vietnamese.

Centralized Device Management

CMS manages key issuance, rotation, device disabling, and installation location tracking. ESP32 firmware is flashed via USB directly from the browser, preserving WiFi configuration and device keys. Devices authenticate with HMAC signatures and open secure WebSocket channels; audio is compressed with Opus. Over-the-air (OTA) firmware updates are under development.

Public-Space Safety

Kiosks don't open web browsers: official links and directions are shown as QR codes for users to open on their own phones. No payment processing. Emergency instructions are fixed phrases, not AI-generated — eliminating the risk of AI misunderstanding emergency situations.

Companion — Speaker for Elderly Care

ESP32 device provides medication reminders, daily conversation, and activity logging. When anomalies are detected, it sends alerts to family members' apps. Dedicated Companion CMS: elderly profiles and emergency contacts, reminder schedules, alerts, daily reports, family accounts, «memory vault», facility-wide dashboard.

Schedule Planning Based on Free Time

Users say «I have 4 hours free, where should I go?» — Vivi creates an itinerary considering travel time between locations. The app also translates signs and menus via camera, searches maps and nearby places.

Security and Privacy

No audio recording stored, no user location history saved. Role-based access control (admin, operator, editor, reviewer). API keys and webhooks managed centrally. Data isolated per deploying customer.

Use Cases

Hotel — «I have 4 hours free, where should I go?»

Guests share their free time and preferences. Vivi creates an optimal itinerary including travel time. Lobby kiosks greet guests in 5 languages, provide sourced ticket prices and opening hours, and show directions via QR codes on guests' phones.

International Tourists — Sign and Menu Translation

Tourists photograph signs or small restaurant menus with the app; Vivi translates into their language. Anonymous use, no account registration — perfect for tourists needing quick information.

Tourism Exhibition Booth — Kiosk Guidance

Kiosks greet visitors in 5 languages, provide sourced ticket prices and opening hours, and show directions via QR codes. No browser access, no payment — safe for public spaces.

Family with Elderly Member — Companion Speaker

ESP32 speaker provides medication reminders, daily conversation, and activity logging. When anomalies are detected, family members receive app notifications. Family members can remotely monitor reminders, alerts, and daily logs.

System Architecture

App (iOS/Android), kiosk, and ESP32 all connect through a shared Gateway — a single entry point for all devices. The Gateway connects to Voice AI services (recognition · AI · voice output), BFF/API (locations, ticket prices, notifications, devices), scheduling service, and Companion service. CMS handles editing, approval, and operations, writing to PostgreSQL + pgvector knowledge base.

All CMS changes — locations, ticket prices, translations, AI scripts — go through an approval step before reaching users. Approved content is shared across all devices, ensuring consistency between app, kiosk, and robot.

Technology Stack

  • Speech Recognition & Voice Output: Google Cloud Chirp 3 / Chirp 3 HD
  • Language Model: Claude (Anthropic)
  • Knowledge Search: Google Vertex AI embedding on PostgreSQL + pgvector
  • App: Flutter (iOS, Android)
  • Kiosk: Orange Pi 5B, 1280×800 display, ReSpeaker microphone
  • ESP32: ESP32-S3, 16MB flash, WiFi, WebSocket + HMAC
  • CMS: PostgreSQL, four-eyes approval, tamper-proof audit log

Pilot Deployment in Kyoto

Vivi Voice AI is currently piloting in Kyoto with nearly 700 tourist locations loaded into the AI knowledge base. The solution is ideal for tourist attractions, hotels, tourism exhibition booths, and elderly care facilities.

Contact Fujigo for consultation and live demo.