💬 Chapter 5: Human-Robot Interaction

5.1 Interaction Modalities

Human-Robot Interaction (HRI) encompasses all forms of communication and collaboration between humans and robots. WIA-ROB-019 defines standard interaction modalities ensuring natural, intuitive, and safe interactions.

5.1.1 Verbal Communication

Speech is the most natural interaction mode for humans:

Speech Recognition

• Continuous Speech: Process natural conversational speech, not just isolated commands. Handle filled pauses ("um", "uh"), partial sentences, corrections.
• Speaker Recognition: Identify who is speaking. Support 5-20 registered users. Accuracy: >95% for registered users. Privacy considerations—clear consent and data handling.
• Multi-Language Support: Minimum English + local language. Automatic language detection. Seamless language switching.
• Acoustic Models: Robust to noise (SNR down to 5 dB), accents, speech patterns. Adaptation to individual speakers.

Natural Language Understanding

Convert speech transcripts to actionable semantics:

• Intent Classification: Determine user goal (command, question, conversation, etc.). 20+ intent categories. Accuracy: >90%.
• Entity Extraction: Identify relevant objects, locations, times, quantities. Named entity recognition (NER) for people, places.
• Semantic Parsing: Build structured representation of command. Handle ambiguity through clarification questions.
• Context Management: Maintain conversation context. Resolve pronouns ("it", "that"), implicit references. Handle multi-turn dialogues.
• Integration: WIA-INTENT standard provides unified natural language interface across WIA ecosystem.

Speech Synthesis

Generate natural-sounding robot speech:

• Text-to-Speech (TTS): High-quality neural TTS (Tacotron 2, FastSpeech 2, VITS). Natural prosody, emotion, emphasis.
• Voice Selection: Multiple voice options. Match robot persona and context. Allow user preference.
• Speech Rate: Adjustable 0.7-1.3× normal speed. Slower for instructions, faster for known information.
• Languages: Same languages as speech recognition. Natural accent and pronunciation.
• Latency: <200ms from text input to audio start. Real-time for responsive conversation.

5.1.2 Non-Verbal Communication

Gaze and Head Orientation

Eye contact and gaze direction are powerful social signals:

• Joint Attention: Robot looks at same object as human, or guides human attention to objects of interest. Enhances collaboration and mutual understanding.
• Turn-Taking: Gaze patterns signal conversational turn-taking. Look at speaker when listening, away when thinking, at listener when speaking.
• Engagement: Maintain appropriate eye contact. Not too much (uncomfortable) or too little (disengaged). Cultural variations—adapt based on context.
• Expression: Head nods for agreement, shakes for disagreement. Tilts to show interest or confusion.

Gestures and Body Language

• Deictic Gestures: Pointing to objects, locations. Coordinate with verbal references ("this", "that"). Accuracy: point to within 5cm at 1m distance.
• Emblematic Gestures: Symbolic gestures with specific meanings. Waving hello/goodbye, thumbs up, OK sign. Culture-specific—adapt to local norms.
• Expressive Gestures: Convey emotion and emphasis. Open arms for welcoming, crossed arms for thinking, shrugging for uncertainty.
• Whole-Body Postures: Leaning forward shows interest, leaning back provides space. Postural mirroring builds rapport.

Facial Expressions

If robot has expressive face (display, animatronic features):

• Display appropriate emotions aligned with speech and context
• Smile for friendly greeting, concern for problem situations
• Avoid uncanny valley—stylized expressions often better than photorealistic
• Synchronize with speech (co-speech expressions) and events

Proxemics (Personal Space)

Maintain culturally appropriate distances:

• Intimate Space (0-0.5m): Reserved for physical assistance, healthcare. Explicit consent required. Approach slowly, announce intentions.
• Personal Space (0.5-1.2m): For close collaboration, object handover. Default interaction distance for known users.
• Social Space (1.2-3.6m): Normal conversation distance. Default for new interactions.
• Public Space (>3.6m): Formal presentations, public navigation.
• Approach Behavior: Never approach from behind. Front or side approach. Slow down when entering personal space. Stop if human shows discomfort.

5.2 Interaction Design Principles

5.2.1 Transparency and Predictability

Users must understand what robot is doing and why:

• State Communication: Clearly indicate robot state (idle, listening, thinking, acting, error). Visual indicators (LED colors), audio cues, posture changes.
• Intent Communication: Announce intentions before acting. "I'm going to pick up the box now." Give humans time to intervene if needed.
• Feedback: Acknowledge commands. "I'll get that for you." Confirm task completion. "Done. The box is on the table."
• Explanation: Provide reasoning when asked. "Why did you put it there?" → "Because you usually keep tools on this shelf."
• Predictable Motion: Move smoothly and legibly. Humans should be able to predict robot trajectory. Avoid sudden movements.

5.2.2 Adaptability to User

Tailor interaction style to individual users:

• Experience Level: Provide more guidance for novice users, concise interaction for experts. Learn user proficiency over time.
• Preferences: Remember user preferences. Interaction modality (prefer speech vs touch interface), verbosity, personal space distance.
• Capabilities: Adapt to user capabilities. Slower speech for elderly or hearing-impaired. Visual cues for users with hearing difficulties. Simplified commands for children.
• Emotional State: Respond appropriately to user emotion. More cheerful for happy users, gentle and supportive for distressed users. Don't assume—verify through explicit cues.

5.2.3 Error Handling and Recovery

Gracefully handle failures and misunderstandings:

• Clarification: Ask questions when command ambiguous. "Did you mean the red box or the blue box?"
• Confirmation: Confirm high-risk actions. "You want me to delete all files. Is that correct?"
• Apology: Acknowledge mistakes. "I'm sorry, I didn't understand. Could you rephrase?"
• Alternative Offers: Suggest alternatives if can't fulfill request. "I can't lift that heavy box, but I can help you move it with a cart."
• Graceful Degradation: If some capability fails (e.g., speech recognition noisy environment), fall back to alternative modality (touch screen, gestures).

5.3 Social Behaviors and Norms

5.3.1 Politeness and Etiquette

Follow social conventions:

• Use "please", "thank you", "excuse me" appropriately
• Apologize for errors, interruptions, inconveniences
• Don't interrupt humans (unless emergency)
• Respect turn-taking in conversations
• Greet people when encountering them
• Say goodbye when leaving or ending interaction
• Adapt formality level to context (casual vs professional)

5.3.2 Cultural Sensitivity

Respect cultural differences:

• Language and Idioms: Avoid culture-specific idioms that don't translate. Use universally understood expressions.
• Personal Space: Varies by culture (closer in Mediterranean, farther in Northern Europe). Learn from context and user feedback.
• Eye Contact: Some cultures value direct eye contact, others consider it disrespectful. Observe and adapt.
• Gestures: Same gesture can mean different things. Thumbs up offensive in some cultures. Pointing rude in others. Localize gesture vocabulary.
• Time Perception: Punctuality expectations vary. Some cultures value precise timing, others more relaxed. Adapt scheduling behaviors.

5.3.3 Privacy and Consent

Protect user privacy and autonomy:

• Data Collection: Explicit consent before collecting personal data (faces, voices, behavior patterns). Clear explanation of what's collected and why.
• Data Storage: Minimize data retention. Delete when no longer needed. Encrypt sensitive data. Allow user to request deletion.
• Recording: Never record audio/video without consent. Clear indication when recording (LED, announcement). Easy way to stop recording.
• Sharing: Don't share user data without permission. No third-party access except as explicitly agreed.
• Autonomy: Users always have final say. Can override robot decisions. Easy emergency stop. Don't manipulate or deceive.

5.4 Collaborative Task Execution

5.4.1 Shared Goals and Planning

Work together effectively:

• Goal Sharing: Ensure human and robot understand shared goal. Confirm understanding before starting.
• Task Allocation: Negotiate who does what. Consider capabilities—robot handles heavy lifting, human makes decisions requiring judgment.
• Coordination: Synchronize actions. "I'll hold this steady while you tighten the bolt." Timing coordination for handovers.
• Adaptation: Adjust plan based on progress. If approach isn't working, propose alternative. "This method isn't working well. Should we try a different approach?"

5.4.2 Object Handover Protocols

Safe and natural object transfer:

1. Announcement: "I'm handing you the tool now."
2. Approach: Move object toward human at comfortable speed (0.2-0.5 m/s).
3. Presentation: Orient object in ergonomic grasping orientation. Present at comfortable height (waist to shoulder).
4. Waiting: Hold steady until human grasps. Detect grasp through force sensing or vision.
5. Release: Gradually reduce grip force. Confirm human has control before full release.
6. Verification: Monitor to ensure human has object safely. Ready to catch if dropped.

Receiving objects from humans:

1. Ready Signal: Indicate readiness. Open hand, move to receiving position. "I'm ready to take it."
2. Acceptance: Grasp gently. Don't pull—let human release when ready.
3. Confirmation: Increase grip force once human releases. "Got it, thank you."
4. Inspection: Verify successful grasp. If object slipping or awkwardly grasped, regrasp before moving.

5.4.3 Situational Awareness and Safety

Maintain safety during collaboration:

• Human Tracking: Always know where humans are. Update human positions continuously (10+ Hz).
• Collision Avoidance: Never contact human unexpectedly. Slow down or stop if human enters safety zone.
• Force Limiting: If contact occurs, limit force to safe levels (<50N for general contact, <10N for head/face). Immediately retract.
• Emergency Stop: Respond to stop commands (verbal "stop", gesture, e-stop button) within 200ms. Full stop within 500ms.
• Hazard Communication: Warn humans of hazards. "Careful, this object is hot." "Please step back, I need to move this heavy load."

5.5 Multi-Modal Interaction Integration

5.5.1 Complementary Modalities

Combine modalities for richer interaction:

• Speech + Gesture: "Put that [point] over there [point]." Gestures disambiguate verbal references.
• Speech + Gaze: Look at object while mentioning it. "Can you hand me the wrench?" [look at wrench]
• Touch + Speech: Touch screen to select object, speak command. "Move this [touch object on screen] to the table."
• Display + Speech: Show information on screen while providing spoken explanation. Redundant modalities increase understanding.

5.5.2 Modality Selection

Choose appropriate modality for context:

Situation	Preferred Modality	Rationale
Noisy Environment	Touch, gestures, display	Speech unreliable with high noise
Hands-Full	Speech, gaze	Can't use hands for input
Public Space	Quiet gestures, touch	Avoid disturbing others
Complex Information	Display + speech	Visual aids comprehension
Urgent Command	Speech, gesture	Fastest modalities
Privacy-Sensitive	Touch, quiet speech	Minimize exposure

5.6 Long-Term Interaction and Learning

5.6.1 User Modeling

Build models of individual users over time:

• Preferences: Learn task preferences, interaction styles, personal preferences. Adapt default behaviors accordingly.
• Skills: Track user expertise with robot. Provide less guidance as users become proficient.
• Habits: Learn routines and patterns. Anticipate needs. "It's 3pm, would you like your usual afternoon coffee?"
• Social Dynamics: Understand relationships between users (family members, colleagues). Respect social hierarchies and relationships.

5.6.2 Personalization

Customize robot behavior to individual users:

• Name Usage: Use person's name naturally (not excessively). Remember names of regular users.
• Communication Style: Match user's formality level, verbosity, humor. Some users want minimal talk, others enjoy conversation.
• Task Execution: Remember how users like tasks done. "I folded the laundry the way you showed me last time."
• Proactive Assistance: Offer help based on learned patterns, but not intrusively. Balance helpfulness with autonomy.

5.6.3 Continuous Improvement

Learn and improve from interactions:

• Feedback Integration: Learn from explicit feedback (corrections, ratings) and implicit feedback (task success, user satisfaction indicators).
• Error Analysis: Analyze failures to improve. Why did command fail? How to prevent in future?
• Knowledge Expansion: Learn new objects, locations, people, procedures through interaction. "This is a Phillips screwdriver."
• Adaptation: Continuously update models as users change, environments change, new tasks introduced.

5.7 Chapter Summary

This chapter explored human-robot interaction—the interface between humanoid robots and the people they serve. We examined verbal and non-verbal communication modalities, interaction design principles emphasizing transparency and adaptability, social behaviors and cultural norms, collaborative task execution, multi-modal interaction, and long-term learning and personalization.

Natural, intuitive HRI is essential for humanoid robot acceptance and effectiveness. WIA-ROB-019 HRI specifications ensure robots interact in ways that are comfortable, safe, and beneficial for humans while respecting autonomy, privacy, and cultural diversity.

In Chapter 6, we'll examine safety standards and compliance requirements that govern humanoid robot design and operation, ensuring robots operate safely in human environments.