Chapter 3. WIA Emotion AI Standard Overview

Hongik Ingan (弘益人間)

"Benefit All Humanity"

The WIA Emotion AI Standard provides a comprehensive framework for ethical, accurate, and interoperable affective computing. This chapter offers a single-screen overview of the four-phase architecture, the classification framework, the multimodal fusion strategies, and the design principles, so that decision-makers can grasp the standard's structure and value before turning to the detailed Phase 1–4 specifications in Chapters 4 through 8. It also defines the simulator-to-volume mapping table (§3.5b) that anchors English-edition vocabulary to the simulator at https://wiastandards.com/emotion-ai/simulator/.

3.1 Mission and Goals

3.1.1 Mission Statement

The WIA Emotion AI Standard sets out to establish a universal, open framework for emotion-recognition systems that enables innovation and interoperability while putting human well-being, privacy, and accuracy first. The mission is not merely the definition of a technical specification; it pursues four axes simultaneously — protection of user rights, fair competition in the industry ecosystem, consensus among academia, industry, and government, and compatibility across the major regulatory regions (Korea, EU, United States, and the wider Asia-Pacific). The four axes are mutually supportive; standards that emphasise only one of them have historically failed either by under-adoption or by loss of social trust.

The standard does not seek to replace the work of ISO/IEC, IEEE, W3C, ITU-T, or national-association standards bodies; it positions itself as an interoperability layer over them. Its operating principle is neutrality: it represents no single firm, country, or school of thought. The "one country, one vote" governance rule is adopted to forestall standard capture by large-share firms. This contrasts with sectoral-association models in which voting weight scales with contribution.

3.1.2 Core Goals

Table 3-1. The five core goals of the WIA standard and the benefits they enable
Goal	Description	Benefit
Interoperability	Common data format and API	Freedom from vendor lock-in
Accuracy	Minimum accuracy thresholds	Trustworthy results
Ethics	Privacy and consent requirements	Responsible AI
Fairness	Mandatory bias testing	Equitable performance
Transparency	Clear documentation	User-comprehensible outputs

These five goals are not mere slogans; they are realised as twenty-four measurable conformance items. Each item has four constituents — a test procedure, a pass criterion, a reporting form, and a corrective-action workflow — so that abstract values become concretely auditable. The twenty-four items are distributed across the five goals (six, five, five, four, four), reflecting the natural decomposition rather than a relative-importance weighting.

3.2 The Four-Phase Architecture

The WIA Emotion AI Standard organises the affective computing stack into four phases, each of which addresses a distinct layer. Each phase can be implemented and certified independently; full value, however, is realised by adopting all four. Phase decomposition deliberately lowers the entry barrier and enables incremental scaling: Phases 1 and 2 may be adopted first; Phase 3 added when real-time processing is needed; Phase 4 introduced when domain-specific regulatory alignment becomes necessary.

Figure 3-1. The four-phase architecture of the WIA Emotion AI Standard

┌─────────────────────────────────────────────────────────────┐
│                    Phase 4: Integration                      │
│   Healthcare │ Education │ Marketing │ Automotive │ XR     │
├─────────────────────────────────────────────────────────────┤
│                    Phase 3: Streaming Protocol               │
│       WebSocket │ REST │ Real-time streaming │ Security    │
├─────────────────────────────────────────────────────────────┤
│                    Phase 2: API Interface                    │
│  Face │ Voice │ Text │ Biosignal │ Multimodal fusion       │
├─────────────────────────────────────────────────────────────┤
│                    Phase 1: Data Format                      │
│  JSON Schema │ Emotion │ AU codes │ V-A │ Metadata         │
└─────────────────────────────────────────────────────────────┘

3.2.1 Phase 1 — Emotion Data Format

Purpose. Standardises the representation of emotion data.
Scope. JSON Schema, emotion labels, AU codes, dimensional axes.
Detail. Chapter 4.

Phase 1 is the common vocabulary on which every later phase depends and is the most basic certification step. A system that implements only Phase 1 can already qualify as "WIA Compliant"; this lower bar is intentional, lowering the entry barrier to the standard.

3.2.2 Phase 2 — API Interface

Purpose. Defines common API endpoints and methods.
Scope. Per-modality REST APIs, request and response formats.
Detail. Chapter 5.

Phase 2 is the standard transport for Phase 1 data. It follows REST principles and provides per-modality endpoints, so a system that only needs facial-expression analysis is not required to implement voice or text endpoints in order to be conformant. Authentication is supported via OAuth 2.0 and API Key on equal footing.

3.2.3 Phase 3 — Streaming Protocol

Purpose. Enables real-time emotion streaming.
Scope. WebSocket protocol, frame rate, security.
Detail. Chapter 6.

Phase 3 is used by applications that demand real-time processing — driver monitoring, contact-centre call analysis, interactive games. Non-real-time uses (for example, retrospective text-diary analysis) can rely on Phase 2 alone. WebSocket is the recommended primary protocol; a gRPC-streaming adapter is defined in Annex H for microservice environments.

3.2.4 Phase 4 — Integration

Purpose. Domain-specific integration guidance.
Scope. Healthcare, education, marketing, automotive, XR.
Detail. Chapter 7.

Phase 4 reconciles domain-specific regulatory requirements and ethical guidance. Region-specific annexes (Annex KR, Annex EU, Annex US) document how the four-phase architecture meets each regulatory environment.

3.3 Emotion Classification Framework

3.3.1 The Discrete Model — Ekman

The WIA standard adopts seven labels — Ekman's six basic emotions plus Neutral — as the core interoperability layer. The discrete model is intuitive for end-users and lends itself to statistical analysis; it is particularly well-suited to applications such as content recommendation and marketing-effectiveness measurement that compare label distributions across users.

Table 3-2. The seven core emotion labels of the WIA standard with typical Valence-Arousal ranges
Emotion	English label	Emoji	Typical V-A range
Happiness	happiness	😊	V: 0.5 to 1.0; A: 0.2 to 0.8
Sadness	sadness	😢	V: −0.8 to −0.3; A: −0.5 to 0.1
Anger	anger	😠	V: −0.7 to −0.2; A: 0.3 to 0.9
Fear	fear	😨	V: −0.7 to −0.2; A: 0.4 to 0.9
Disgust	disgust	🤢	V: −0.8 to −0.3; A: −0.1 to 0.5
Surprise	surprise	😮	V: −0.2 to 0.5; A: 0.5 to 1.0
Neutral	neutral	😐	V: −0.2 to 0.2; A: −0.2 to 0.2

Display labels in the user's language are layered over a fixed lower-case English key (happiness, sadness, …) so that locale-specific display labels in Korean, Japanese, Chinese, or Arabic resolve to the same internal label key. This separation is compatible with the W3C EmotionML 1.0 vocabulary mechanism (W3C, 2014).^[1]

3.3.2 The Dimensional Model — Valence-Arousal

The WIA standard supports the dimensional model in parallel with the discrete model. The dimensional model composes naturally with regression-style machine-learning and is strong on representing subtle change and mixed emotion. Music recommendation, digital-meditation applications, and adaptive-difficulty games — all of which need continuous tracking of user state — derive particular value from the dimensional output.

Figure 3-2. The Valence-Arousal space

                    +1.0 (high arousal)
                          │
                   Anger  │  Excited
                          │
    -1.0 ─────────────────┼───────────────── +1.0
    (negative)            │           (positive)
                          │
                   Sadness│  Calm
                          │
                    -1.0 (low arousal)

Range:
  Valence: -1.0 (most negative) to +1.0 (most positive)
  Arousal: -1.0 (low energy) to +1.0 (high energy)

WIA Phase 1 recommends populating both the discrete and dimensional outputs simultaneously: discrete labels feed user-facing UIs, while V-A coordinates support statistical analysis and time-series tracking. Conversion between the two representations is straightforward in either direction using the typical ranges of Table 3-2. When the dimensional output's nearest discrete-label V-A centre exceeds a configurable distance threshold, the UI is recommended to fall back to a "mixed" or "uncertain" label rather than coerce a single category — a concrete instance of the standard's transparency principle.

3.3.3 Extended Emotion Labels

Table 3-3. WIA extended emotion labels — five categories
Category	Extended labels
Positive, high arousal	excited, elated, enthusiastic, amused
Positive, low arousal	content, relaxed, calm, serene
Negative, high arousal	stressed, anxious, frustrated, irritated
Negative, low arousal	bored, tired, depressed, melancholic
Cognitive states	confused, focused, interested, engaged

Culturally specific affect labels — for example culturally specific forms of grief, longing, embarrassment, or sustained-relational attachment — may be added through the user-defined-label mechanism. A user-defined label must declare an English key, a display name, a typical V-A range, and a citation; once sufficiently adopted, the WIA Steering Committee can promote it to an official label in the next major version.

3.4 FACS Integration

3.4.1 Supported Action Units

The WIA standard supports all forty-four Action Units defined by FACS (Ekman & Friesen, 1978; Ekman, Friesen & Hager, 2002).^[2]

Table 3-4. Action Units supported by the WIA standard, organised by region
AU range	Region	Count
AU1–AU7	Upper face — brows, forehead	7
AU9–AU17	Nose, upper lip	8
AU18–AU28	Lower face — lips, chin	11
AU41–AU46	Eyelids	6
AU51–AU58	Head pose	8
AU61–AU64	Eye position	4

The sixteen mandatory core AUs are listed in Table 1-5; the remaining twenty-eight are optional. Medical-and research-grade applications are advised to support all forty-four; consumer applications are typically adequate with the sixteen core. The industry reference for AU-recognition accuracy is agreement with OpenFace 2.0 (Baltrušaitis et al., 2018), which the WIA conformance suite uses as a partial benchmark.^[3]

3.4.2 AU Intensity Coding

Table 3-5. WIA AU intensity coding — mapping to the FACS five-grade scale
Intensity value	FACS grade	Meaning
0.0	—	Not active
0.01–0.20	A — Trace	Trace
0.21–0.40	B — Slight	Slight
0.41–0.60	C — Marked	Marked
0.61–0.80	D — Pronounced	Pronounced
0.81–1.00	E — Maximum	Maximum

Conversion between the floating-point and five-grade representations is lossless; conformant systems must emit at least one of the two. AU output is the primary means by which a face-analysis system exposes its internal evidence to auditors, supporting the EU AI Act's transparency and human-oversight requirements.

3.5 Supported Modalities

3.5.1 Facial Expression Analysis

Table 3-6. WIA facial-expression modality — input, resolution, frame-rate, output, latency
Item	Specification
Input type	Image (JPEG, PNG) or video (H.264, VP9)
Resolution	≥ 480p; recommended ≥ 720p
Frame rate	≥ 15 fps; recommended 30 fps
Output	Emotion label, AU intensities, V-A coordinates
Latency target	< 100 ms per frame

3.5.2 Voice and Speech Analysis

Table 3-7. WIA voice modality — input format, sample rate, channels, features, output
Item	Specification
Input type	Audio (WAV, MP3, WebM)
Sample rate	≥ 16 kHz; recommended 44.1 kHz
Channels	Mono or stereo
Features	Pitch, intensity, speech rate, voice quality, prosody
Output	Emotion label, V-A coordinates, confidence

3.5.3 Text Sentiment Analysis

Table 3-8. WIA text modality — input, languages, length, output, features
Item	Specification
Input type	UTF-8 text
Languages supported	≥ 100 languages
Maximum length	10,000 characters per request
Output	Sentiment polarity, emotion label, per-entity emotion
Features	Sarcasm detection, aspect-based sentiment, intensity

3.5.4 Biosignal Analysis

Table 3-9. WIA biosignal modality — signals, sample rates, format, output
Item	Specification
Supported signals	ECG / HR, EDA / GSR, EEG, respiration
Sample rate	HR ≥ 1 Hz; EDA ≥ 4 Hz; EEG ≥ 128 Hz
Format	JSON array or CSV
Output	Arousal, stress index, engagement

Each modality has distinct strengths and weaknesses, and combinations are recommended. Facial expression is strong for discrete-emotion classification; voice is strong for arousal estimation; text contributes intent and context; biosignal is hardest to fake. These complementary properties motivate the multimodal-fusion strategies of §3.6. In medical-grade applications all four modalities are recommended; clinical decisions based on a single modality are excluded from conformance.

3.5b Simulator Five-Panel ENUM and Provisional-Threshold Mapping

The interactive simulator that ships with this volume — accessible at https://wiastandards.com/emotion-ai/simulator/ — is organised as five working panels. Each panel maps one-to-one onto a phase of the four-phase architecture (Phase 1 to Phase 4). This section consolidates the panel headers, core ENUMs, and provisional thresholds used in the simulator so that the volume's vocabulary remains aligned with the simulator across editions. Where vocabulary diverges, the ENUM as displayed in the simulator takes precedence.

Table 3-5b. WIA Simulator 5-Panel ENUM / Provisional-Threshold Mapping
Panel	Simulator header	Core ENUM	Provisional threshold	Volume mapping
0	📊 Emotion Data	INPUT, OPTION, TEXTAREA, Subject ID, Primary Emotion, Confidence (0–1), Valence (−1 to +1), Arousal (−1 to +1), Action Units (FACS), Modality	Per-modality classification accuracy: provisional 76.8% to 92.3%	Phase 1 — Data format (Chapter 4)
1	🔢 Analysis	Valence-Arousal four-quadrant, High Arousal · Low Arousal · Negative · Positive	Analysis window: provisional 0.2 s to 0.3 s	Chapter 1 §1.3.2 — Dimensional model
2	📡 Protocol	STREAMING, STOPPED, message format (JSON, MessagePack, Protocol Buffers)	Streaming latency: provisional ≤ 300 ms (see §6.3 for measured values)	Phase 3 — Streaming protocol (Chapter 6)
3	🔗 Integration	External-system adapters, domain-specific standards (FHIR, SCORM, etc.)	(No simulator-defined threshold — see §7.3, §7.4 for domain thresholds)	Phase 4 — Integration (Chapter 7)
4	🧪 Emotion Test	Predicted Emotion output, six basic emotions (Ekman) with dimensional model auxiliary	Classification response time: provisional 0.2 s to 0.3 s	Chapter 8 — Implementation and certification (test procedure)

The mapping is the alignment reference between the volume's ENUMs and thresholds and those of the simulator; both are updated in lockstep. Where the volume cites a threshold not explicitly stated in the simulator, the citation is rendered in provisional form ("provisional", "approximately", "expected") so that the simulator's authority is preserved.

3.6 Multimodal Fusion

3.6.1 Fusion Strategies

Table 3-10. Four multimodal-fusion strategies and their use cases
Strategy	Description	Use case
Early fusion	Combine raw features before classification	Modalities synchronised in time
Late fusion	Combine classification outputs	Modalities are independent
Decision fusion	Voting or weighted average over decisions	Simple, robust approach
Attention fusion	Context-conditioned learned weights	Modality reliability varies over time

Late fusion and decision fusion are the most common in industrial systems because they preserve modular development; early fusion and attention fusion can deliver higher accuracy but at the cost of more complex training. Attention fusion has gained traction with the rise of transformer-based multimodal models and is particularly effective in environments where modality reliability fluctuates over time (call-noise variation, in-cabin lighting variation, and the like). The choice of strategy also depends on the legal and consent basis: voice analysis on telephony channels typically requires two-party consent under telecommunications-secrecy law, whereas text analysis on user-supplied input may not, so the legal "processing units" of two modalities can differ.

3.6.2 Modality Weighting

Figure 3-3. Default multimodal weights — face 0.40 · voice 0.25 · text 0.20 · biosignal 0.15

Default weights (adjustable):
  Face:      0.40 (highest reliability for discrete-emotion classification)
  Voice:     0.25 (strong for arousal estimation)
  Text:      0.20 (context-dependent)
  Biosignal: 0.15 (hard to fake but noisy)

Adjustment criteria:
  - Signal quality
  - Context (e.g. voice-only call)
  - Cultural factors
  - Per-user calibration

3.7 Design Principles

3.7.1 Core Principles

Privacy by Design. Minimal data collection; consent obligation.
Transparency. Clear disclosure that emotion AI is in use.
Accuracy. Minimum thresholds together with demographic fairness.
Interoperability. Standard formats enable data portability.
Extensibility. User-defined emotions and modalities are supported.
Cultural sensitivity. Cultural difference is reflected in design.
Human oversight. Human review of automated decisions is possible.

3.7.2 Technical Principles

JSON-based. Human-readable; widely supported.
Semantic versioning. Clear upgrade paths (SemVer 2.0).
REST and WebSocket. Standard web protocols.
Confidence scores. Always include an uncertainty estimate.
Timestamps. Time-series analysis is enabled.
UTF-8. Equal treatment of every language, including Korean.
Minimal dependencies. Standard implementation must not be tied to a specific cloud vendor.

3.8 Certification Levels

Table 3-11. The three certification levels of the WIA standard
Level	Name	Requirements	Use case
1	Compliant	Data-format conformance; 75% accuracy	Research, prototypes
2	Certified	Full API conformance; 80% accuracy; bias testing	Commercial products
3	Certified Plus	All requirements; 85% accuracy; external audit	Healthcare; sensitive applications

Level transitions require only the additional tests for the next level rather than re-certification of prior tests. Certificates are valid for twenty-four months; renewal triggers either partial or full re-testing depending on whether the model, code, or training data have changed. A renewal-due notification is issued thirty days before expiry; failure to begin renewal causes automatic expiry, after which the full certification process must be re-run.

3.9 Note on Korean Edition Content

The Korean edition of this volume includes additional sections specific to Korea: detailed alignment with national-association standards (industrial standards, ICT-association standards, and a national cyber-security agency self-assessment toolkit), a five-scenario industry-application section grounded in named domestic enterprises, a five-stage adoption roadmap calibrated to typical Korean enterprise timelines (six to twelve months), and a governance section describing how Korean institutions can contribute to standards revisions through formal channels.

This English edition deliberately abstracts those passages. References to specific named Korean industrial standards, association standards, public-agency tools, and enterprise scenarios become "leading domestic standards bodies", "the relevant national cyber-security agency self-assessment toolkit", "leading domestic conglomerates", "leading domestic universities, government agencies, telecom operators, and platform companies", and "leading commercial SDK vendors". The conformance requirements themselves are identical between the two editions.

3.10 Chapter Summary

Nine key takeaways.

Four-phase architecture. Data format → API → protocol → integration.
Dual model. Both discrete (Ekman) and dimensional (V-A) representations are supported.
FACS. All forty-four AUs are codable.
Four modalities. Face, voice, text, biosignal.
Multimodal fusion. Four strategies are supported.
Three certification levels. Compliant, Certified, Certified Plus.
Simulator alignment. Table 3-5b maps the volume to the simulator's five panels.
Incremental adoption. A phased adoption path lowers the barrier for newcomers.
Governance transparency. All revision proposals and votes are recorded openly in the public GitHub repository.

3.11 Review Questions

Draw the four-phase architecture (data format, API, protocol, integration) and describe each phase's role.
Compare the discrete and dimensional emotion models and explain why simultaneous output is recommended.
Tabulate the seven core emotion labels of the WIA standard with their typical Valence-Arousal ranges.
Describe the user-defined-label mechanism and the four metadata items it requires.
Explain the four multimodal-fusion strategies (early, late, decision, attention) with at least one industrial example each.
Describe the three certification levels with their use cases and accuracy thresholds.
Describe the role of Table 3-5b in maintaining alignment between the volume and the simulator.
Explain how the standard treats culturally specific affect labels through the user-defined-label mechanism.

3.12 Looking Ahead

Chapter 4 turns to Phase 1 — Emotion Data Format — and treats the JSON Schema, field specifications, and worked examples in depth. Phase 1 is the common vocabulary on which every later phase depends and is the longest chapter in the volume. After Chapter 4 the reader will have enough understanding to write an adapter that converts an arbitrary emotion-recognition system's output into the WIA Phase 1 format. The standard's evolution roadmap is recorded in the public GitHub repository.^[99]

Chapter 3 Endnotes

W3C. (2014). Emotion Markup Language (EmotionML) 1.0, W3C Recommendation, 22 May 2014. https://www.w3.org/TR/emotionml/. ↑
Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, ISBN 0-931835-01-1. Updated edition: Ekman, Friesen & Hager (2002). ↑
Baltrušaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018). OpenFace 2.0: Facial behavior analysis toolkit. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition, 59–66. DOI 10.1109/FG.2018.00019. ↑
ISO/IEC 22989:2022. Information technology — Artificial intelligence — Artificial intelligence concepts and terminology. https://www.iso.org/standard/74296.html.
ISO/IEC 23053:2022. Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML). https://www.iso.org/standard/74438.html.
ISO/IEC 23894:2023. Information technology — Artificial intelligence — Guidance on risk management.
ITU-T. (2018). Recommendation F.748.11: Emotion-aware multimedia services.
IEEE 7000-2021. IEEE Standard Model Process for Addressing Ethical Concerns During System Design. DOI 10.1109/IEEESTD.2021.9536679.
IEEE 7003-2024. IEEE Standard for Algorithmic Bias Considerations.
NIST. (2023). AI Risk Management Framework (AI RMF) 1.0. NIST AI 100-1. DOI 10.6028/NIST.AI.100-1.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178. DOI 10.1037/h0077714.
Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In Theories of Emotion. Academic Press, 3–33. DOI 10.1016/B978-0-12-558701-3.50007-7.
Baltrušaitis, T., Ahuja, C., & Morency, L.-P. (2019). Multimodal Machine Learning: A Survey and Taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(2), 423–443. DOI 10.1109/TPAMI.2018.2798607.
IETF RFC 6455. (2011). The WebSocket Protocol. DOI 10.17487/RFC6455.
SemVer 2.0.0. Semantic Versioning Specification. https://semver.org/.
WIA Standards public repository (emotion-ai folder), MIT-licensed source for the simulator, specification, API reference, and ebook assets cited throughout this volume: WIA-Official/wia-standards-public/tree/main/emotion-ai. The standard's evolution roadmap, revision history, and SDK source code are maintained openly in this repository, where the WIA standards committee records its formal verification of all primary sources cited in this chapter. ↑