What is Vivi Voice AI? Multi-device voice assistant for tourism and elderly care

One platform, three device types, five languages — an AI assistant for tourism and elderly care

---

Introduction

International travelers often struggle to find reliable information at tourist destinations: language barriers, outdated or inaccurate online data, and information desks that aren’t staffed around the clock. Elderly people living alone need medication reminders and daily conversation, but not every family has someone nearby to check in.

Vivi Voice AI is a voice assistant built to address both challenges. Users simply speak their question — in Japanese, English, Vietnamese, Chinese, or Korean — and receive a natural spoken response, complete with ticket prices and opening hours verified from official sources within the past 90 days.

What makes Vivi different: it runs across three form factors — a mobile app, a lobby kiosk at hotels, and a compact ESP32 device — all managed centrally through a single CMS.

---

1. What problems does Vivi Voice AI solve?

Language barriers in tourism

Visitors to Kyoto can ask about Kinkaku-ji admission, museum hours, or directions to the nearest train station — in Chinese, Korean, or English. Vivi recognizes the speech, retrieves curated information, and responds in the same language without a machine-translation step in between.

Outdated or inaccurate information

Ticket prices and opening hours are only displayed when an official source has been verified within the past 90 days. If no source exists, the assistant clearly states “please verify on-site” rather than guessing a number. Currently, Vivi holds data for nearly 700 locations in Kyoto.

Information desks that aren’t open 24/7

Kiosks placed in hotel lobbies or tourism exhibition halls answer questions continuously, greet guests in five languages, provide sourced pricing, and offer directions via QR codes that open maps on the guest’s phone.

Elderly people living alone

The ESP32 device acts as a companion speaker: it reminds users to take medication, engages in daily conversation, and keeps a diary. When it detects unusual patterns, it sends alerts to the companion app used by family members.

---

2. Three-device, one-platform architecture

Voice App (iOS/Android)

Built on Flutter, the traveler app works anonymously — no account registration required. Core features:

• Voice Q&A: speech recognition → AI response → spoken answer; users can interrupt the assistant mid-sentence
• Location lookup, maps, nearby places, sourced ticket prices
• Camera-based translation of signs and menus
• Itinerary planning based on the traveler’s available time
• Push notifications tied to campaigns

The companion app lets family members track medication schedules, receive alerts, and view the elderly person’s diary.

Kiosk

Hardware: Orange Pi 5B embedded computer, 1280×800 display, ReSpeaker microphone with hardware echo cancellation and noise reduction. Deployed at tourism exhibition halls and hotel lobbies.

Safety considerations for public placement:

• No web browser: official links and directions are shown as QR codes for guests to open on their own phones
• No payment processing
• Emergency guidance is a fixed script, never AI-generated

ESP32 Device

ESP32-S3 chip, 16 MB flash, WiFi connectivity. Roles:

• Tourism assistant robot: voice conversation with a display screen
• Companion speaker: for elderly users; reminders, conversation, anomaly detection to alert family

The device authenticates via HMAC signature, then opens a secure WebSocket channel; audio is compressed with Opus. Wake word is “Hi ViVi”; the device auto-sleeps after 2 minutes of silence. Firmware is loaded via USB directly from the browser-based CMS; WiFi settings and device keys are preserved.

---

3. The AI technology behind it

Speech recognition and voice output

Google Cloud Chirp 3 / Chirp 3 HD — multilingual support with high accuracy, including Japanese with local proper nouns.

Language model

Claude (Anthropic) — understands conversational context, composes natural responses, and handles user interruptions gracefully.

Knowledge retrieval

Google Vertex AI embeddings on PostgreSQL + pgvector — retrieves curated information (locations, ticket prices, opening hours, weather, train schedules) from a vector database.

Performance targets

• From end of user speech to start of spoken response: ≤ 2 seconds (p95)
• Assistant stops within ≤ 0.3 seconds when interrupted

(These are design targets, not yet measured system-wide.)

---

4. CMS — Centralized management system

What it manages

• Tourist locations (name, description, photos, coordinates), merge queue for duplicates
• Ticket prices and opening hours, with official sources and verification dates
• Data sources and automated collection jobs
• Translations: translation queue, bilingual editing, field-by-field approval
• Push notification campaigns and audience segmentation
• AI assistant scripts (prompts), knowledge index (RAG), proper-noun dictionary
• Devices: registration, key provisioning approval, deactivation, device type, installation location, ESP32 firmware loading via USB
• Conversation logs and guest feedback, user management and permissions, API keys and webhooks

Four-eye approval

Content creators cannot approve their own work; this rule is enforced at the database layer, not just hidden in the UI. Machine translations never go live automatically — a speaker of that language must approve them.

Audit log

Append-only, hash-chained to prevent tampering, retained for 5 years. Role-based permissions (admin, operations, editor, reviewer). CMS interface available in three languages: Japanese, English, Vietnamese.

Separate Companion CMS

Elderly person profiles and emergency contacts, reminder schedules, alerts, diary and daily reports, family member accounts, “memory vault,” facility-wide dashboard.

---

5. Real-world use cases

Hotel guest: “I have 4 free hours, where should I go?”

The guest states their available time and interests. Vivi builds an itinerary to suitable spots, factoring in travel time between locations. The lobby kiosk greets guests in five languages, answers with sourced pricing, and provides QR-code directions.

International travelers: reading signs and menus

Guests photograph signs or small-restaurant menus with the app; Vivi translates into their language. The app works anonymously — no registration needed.

Exhibition halls and hotel lobbies: guidance kiosks

Kiosks greet guests in five languages, answer questions about ticket prices and opening hours with sources, provide QR-code directions that open maps on the guest’s phone, and answer questions about hotel amenities.

Families with elderly members: companion speaker

The ESP32 speaker reminds users when to take medication, engages in daily conversation, keeps a diary, and sends alerts to the family companion app when unusual patterns are detected.

---

6. Security and privacy

• No audio recordings stored, no location history retained
• Four-eye approval, tamper-proof audit logs, role-based access
• HMAC device authentication, secure WebSocket channels
• Role-based permissions, centrally managed API keys and webhooks
• Data isolated by deployment customer

---

Conclusion

Vivi Voice AI is more than a voice chatbot. It’s a multi-device AI assistant platform that connects apps, kiosks, and IoT devices through a single CMS, with four-eye approval ensuring information accuracy.

With nearly 700 Kyoto locations already in its AI knowledge base, Vivi is being piloted at tourist sites, hotels, and exhibition halls in Japan. The solution also extends to elderly care, where the ESP32 device serves as a companion speaker for reminders and family connection.

Contact Fujigo for a consultation and demo of Vivi Voice AI for your business.

---