Real-time interpretation is the stage at which a sign-language recognition system delivers value in the field. This chapter covers the four communication protocols — WebSocket, REST, gRPC, and MQTT — and the interpretation API standard.
Real-time sign-language interpretation is a bidirectional, low-latency system that displays captions and audio on the hearing-side screen at the same time the signer produces a sign. The WIA standard defines five requirements.
The simulator Protocol panel validates the message formats of the four protocols.
WebSocket1), defined in IETF RFC 6455, is a bidirectional communication protocol that runs over TCP. Once connected, client and server can freely exchange messages. The WIA standard adopts WebSocket as the primary recommended protocol for real-time interpretation.
// Client → Server (sign input)
{
"type": "frame",
"session_id": "s_abc123",
"frame_id": 1234,
"timestamp_us": 1700000000000000,
"skeleton": { ... MediaPipe Holistic keypoints ... }
}
// Server → Client (recognition result)
{
"type": "recognition",
"session_id": "s_abc123",
"frame_id_range": [1100, 1234],
"gloss_sequence": ["HELLO", "MEET", "GLAD"],
"translation_ko": "안녕하세요. 만나서 반가워요.",
"translation_en": "Hello. Nice to meet you.",
"confidence": 0.94,
"latency_ms": 87
}Each interpretation session is managed by a unique session_id. At session start the client communicates signer information, sign-language identifier (KSL, ASL, etc.), and target translation language to the server.
REST (Representational State Transfer) is a stateless HTTP-based communication style. The standard defines the following endpoints.
| Endpoint | Method | Description |
|---|---|---|
| /api/v1/recognize | POST | Return recognition results for an uploaded video. |
| /api/v1/translate | POST | Translate a gloss sequence into natural language. |
| /api/v1/sessions | POST | Create a new interpretation session. |
| /api/v1/sessions/{id} | GET | Retrieve session status. |
| /api/v1/sessions/{id} | DELETE | Terminate a session. |
| /api/v1/models | GET | List available models. |
| /api/v1/languages | GET | List supported sign languages. |
gRPC is a high-performance RPC framework developed by Google. It provides Protocol Buffers (protobuf) binary serialization, HTTP/2 bidirectional streaming, and built-in authentication. The standard recommends gRPC for server-to-server communication (e.g., recognition server → translation server).
// sign_language.proto
syntax = "proto3";
service SignLanguageRecognizer {
rpc RecognizeStream(stream FrameRequest) returns (stream RecognitionResponse);
rpc Translate(GlossSequence) returns (TranslationResponse);
}
message FrameRequest {
string session_id = 1;
int64 frame_id = 2;
int64 timestamp_us = 3;
Skeleton skeleton = 4;
}MQTT (Message Queuing Telemetry Transport) is a lightweight pub/sub messaging protocol standardized as ISO/IEC 20922. The WIA standard recommends MQTT for IoT environments (public kiosks, vehicle terminals, wearable cameras).
wia/sl/sessions/{id}/frames — Published by the signer client.wia/sl/sessions/{id}/recognition — Published by the recognition server.wia/sl/sessions/{id}/translation — Published by the translation server.The Ministry of Health and Welfare operates "107 Son-Mal-Ieum Center," a Deaf-only telecommunication relay service. A Deaf user connects via video call to a sign-language interpreter, who relays the message to a hearing party by voice. The WIA real-time interpretation API is compatible with the 107 Son-Mal-Ieum Center system.
"Su-eo Tok" runs on Android and iOS and uses WebSocket to send keypoints to the ETRI server and receive Korean captions for on-screen display. The app has accumulated approximately 120,000 users.
Since 2024 KBS has piloted automatic captioning over the sign-language interpreter video on parts of its main broadcasting schedule. Captions are generated by the KBS in-house recognition engine (MediaPipe Holistic + Transformer).
National Assembly Television launched a pilot in 2025 to provide automatic supplementary captioning over the sign-language interpretation video of plenary session live broadcasts.
Because real-time interpretation APIs transmit signer video, keypoints, and natural-language translation, strong security is mandatory. The standard requires the following.
The technical specifications described in this chapter all serve the broader goal of guaranteeing the everyday communication rights of the Deaf community. Korean Sign Language is, under Article 2 of the Korean Sign Language Act, a public language with status equal to Korean, and the WIA Sign Language Recognition Standard provides the technical underpinning for this legal status.
The standard operates on the Korean national infrastructure: the Korean Association of the Deaf (KAD), the National Institute of the Korean Language (NIKL), the National Information Society Agency (NIA), the Electronics and Telecommunications Research Institute (ETRI), the Korea Advanced Institute of Science and Technology (KAIST), the Korea Institute of Science and Technology Information (KISTI), the Telecommunications Technology Association (TTA), the Korean Standards Association (KSA), the Korean Agency for Technology and Standards (KATS), the Korea Internet and Security Agency (KISA), the Korea Laboratory Accreditation Scheme (KOLAS), and the Korea Communications Agency (KCA), as well as government ministries including the Ministry of Culture, Sports and Tourism (MCST), the Ministry of Health and Welfare (MOHW), the Ministry of Education (MOE), the National Institute of Special Education (NISE), the Korea Communications Commission (KCC), the Ministry of Science and ICT (MSIT), the National Human Rights Commission of Korea (NHRCK), the Ministry of Employment and Labor (MOEL), the Ministry of the Interior and Safety (MOIS), and the Ministry of Justice.
Broadcasters KBS, MBC, SBS, EBS, National Assembly Television, and Arirang International Broadcasting plan to adopt the standard's automatic captioning system; 5G operators SK Telecom, KT, and LG U+ apply the standard to Deaf telecommunication relay services; and AI providers Naver Clova, Kakao i, LG AI Research, Kakao Brain, and SK Telecom X publish KSL recognition APIs compatible with the standard.
The eighteen schools for the Deaf nationwide (Seoul School for the Deaf, Daejeon School for the Deaf, Busan Sungsim School, Incheon Sunhwa School, Gwangju Sunmyeong School, Daegu Yeonghwa School, Gangwon Provincial Dowon School, Chungbuk Cheongju Sungsim School, Chungnam Cheonan Inae School, Jeonbuk Iksan Jeil School, Jeonnam Gwangju Yeonghwa School, Gyeongbuk Andong Yeongmyeong School, Gyeongnam Jinju Hyegwang School, Jeju Yeongji School, Incheon Cheonghak School, Gyeonggi Ansan Jahae School, Ulsan Meari School, and Sejong Sarang School) serve as the standard's KSL education hubs.
This standard is published as an open standard under the MIT license; all simulator code, specifications, and example code are openly available at GitHub WIA-Official/wia-standards-public/tree/main/sign-language. Our hope is that this standard enables Deaf people who use Korean Sign Language to communicate more freely, and that this standard becomes the foundation for sign-language recognition standards across the Asia-Pacific region.
WIA-Official/wia-standards-public/tree/main/sign-language — WebSocket, REST, gRPC, and MQTT reference implementations of the interpretation APIs in this chapter are openly published in this repository.