Chapter 1: Sign Language Recognition Standards Overview — KSL/ISL/ASL

1.1 Sign Language Is a Natural Language

Sign language is a visual-kinetic natural language in which meaning is conveyed through the simultaneous combination of hand shape (handshape), the body location where the sign is produced (location), the motion of the hand and arm (movement), the direction the palm faces (palm orientation), and non-manual markers (NMMs) such as facial expression, eye gaze, mouth morphemes, head tilts, and shoulder rotations. Sign language is not a manual transcription of a spoken language; rather, it is a complete natural language with its own phonology, morphology, syntax, semantics, and pragmatics. William Stokoe's 1960 monograph Sign Language Structure first described American Sign Language (ASL) as a fully systematic linguistic entity, and the global field of sign language linguistics has since accumulated more than six decades of research.¹⁾

In the Republic of Korea, the national sign language is Korean Sign Language (KSL). The Korean Sign Language Act (Act No. 18334, enacted 3 February 2016 and amended 9 June 2021)²⁾ declares in Article 2 that "Korean Sign Language is the public language of Korean Deaf people, with status equal to the Korean language, and the unique language of the Korean Deaf community." That is, KSL is not "Signed Korean" or a manual code for Korean; it is an independent natural language that the Korean Deaf community has developed autonomously over more than a century. The WIA Sign Language Recognition Standard takes this legal and linguistic definition as its starting point.

Worldwide, the World Federation of the Deaf (WFD) estimates that more than 300 distinct sign languages are in use, and ISO 639-3³⁾ registers more than 150 sign language codes. Representative codes include kvk (Korean Sign Language, KSL), ase (American Sign Language, ASL), bfi (British Sign Language, BSL), jsl (Japanese Sign Language, JSL), csl (Chinese Sign Language, CSL), gsg (German Sign Language, DGS), and fsl (French Sign Language, LSF). International Sign (IS) is an auxiliary contact language used at international events. The WIA Sign Language Recognition Standard treats all natural sign languages as equal and does not elevate any one of them to a "standard" sign language. This is both a foundational ethical position for standards development and a sign of respect for the linguistic diversity that Deaf communities have nurtured over decades.

Comparing sign and spoken languages confirms their equivalent status as natural languages. First, sign languages have all five linguistic strata: phonology, morphology, syntax, semantics, and pragmatics. Second, sign languages possess full lexicons, with phenomena including neologism, borrowing, and semantic extension. Third, sign languages have morphemes that express grammatical categories such as tense, aspect, person, and number. Fourth, sign languages exhibit regional dialects, generational variation, and register differences. Fifth, sign languages are acquired natively by children as a mother tongue. KSL satisfies all five criteria.

1.1.1 Simultaneity of Manual and Non-Manual Components

The most distinctive linguistic feature of sign language is "simultaneity." Whereas spoken languages produce phonemes sequentially along a time axis, sign languages produce handshape, location, movement, orientation, and non-manual markers simultaneously within a single sign unit. From an AI recognition perspective, this reduces to a multi-modal synchronous recognition problem. While speech recognition processes one-dimensional time-series audio signals, sign language recognition must process multi-dimensional video time-series simultaneously, which makes it fundamentally a more complex task.

• Handshape: KSL has approximately 50 distinctive handshapes, including representative shapes named after Korean consonant letters such as the "giyeok" (ㄱ) handshape, the "nieun" (ㄴ) handshape, as well as the fist handshape, flat handshape, hook handshape, thumb handshape, and index-finger handshape. The KSL handshape inventory was standardized in the National Institute of the Korean Language's 2018 monograph on KSL phonology.
• Location: KSL defines approximately 12 standard locations, including head, forehead, cheek, chin, neck, chest, abdomen, shoulder, and neutral space. Location is a core distinctive feature, and many minimal pairs differ only in location.
• Movement: Approximately 40 movement patterns serve distinctive functions, including straight, curved, circular, repeated, crossing, approaching, departing, vibrating, and rotating motions. Direction, speed, and repetition count all carry distinctive value.
• Palm Orientation: Six cardinal directions (up, down, forward, backward, left, right) combine with finger orientation to yield more than 12 distinct orientation values.
• Non-Manual Markers (NMM): Raised eyebrows mark interrogatives, head shakes mark negation, mouth morphemes carry adverbial content, eye gaze indexes referents, and shoulder rotations mark role-shift in spatial grammar. NMMs convey grammatical rather than lexical information and must be produced simultaneously with manual signs.

1.1.2 The KSL Phonological Unit — Suhwaso

In KSL phonology these five distinctive features are called suhwaso (수화소), which corresponds to the phoneme in spoken language phonology. Two sign units that differ in only one suhwaso form a minimal pair, demonstrating that KSL has a fully developed phonological system. For example, the signs for MOTHER and FATHER differ only in location; LOVE and RESPECT differ only in palm orientation; and GO and COME differ only in movement direction. Accurately disentangling and recognizing these distinctive features is the primary task of the WIA Sign Language Recognition Standard.

The KSL lexicon is reported at approximately 16,000 lexemes in the 2024 edition of the National Institute of the Korean Language KSL Dictionary. While smaller than the roughly 500,000 entries of the Standard Korean Language Dictionary, KSL achieves rich expressive power through iconicity and spatial grammar. For example, a single VERB-GIVE sign can express "I give to you," "He gives to us," and many other readings by varying the spatial positions of subject and object alone. This spatial grammar is a fundamentally different grammatical system from the case markers, conjunctions, and pronoun systems of spoken languages.

1.2 Scope and Objectives of the WIA Sign Language Recognition Standard

WIA-AY-002 standardizes the following five domains.

1. Data Format Standard: Video, image, skeleton, depth, and inertial measurement unit (IMU) multi-modal inputs are serialized into a single JSON Schema, SignLanguageFrame. Real-time validation is available in the simulator Data Format panel.
2. Recognition Algorithm Interface Standard: The input and output schemas of CNN, LSTM, Transformer, and CTC-based models are unified so that any model can be invoked through the same API regardless of architecture.
3. Communication Protocol Standard: Four protocols — WebSocket, REST, gRPC, and MQTT — are unified into a common message format, guaranteeing bidirectional client-server real-time communication.
4. Integration Standard: Interoperability is guaranteed across broadcast captioning systems (EBS, KBS, NHK), mobile interpretation apps, and interpretation terminals used in medical, legal, and educational settings.
5. Test Standard: Models are evaluated against five metrics: Top-1 accuracy, Word Error Rate (WER), BLEU, SignBLEU, and Latency.

1.2.1 Multi-Modal Input Alignment

The WIA standard aligns four input modalities at the frame level.

The standard mandates that all four modalities share a synchronized timestamp using PTP (Precision Time Protocol, IEEE 1588) so that they can be processed jointly at the same frame. Even when both hands are occluded in multi-camera environments, depth and skeleton data complement each other to preserve recognition accuracy. In mobile environments that use only a single RGB camera, MediaPipe Hands estimates 21 hand landmarks without depth information, so the WIA standard classifies depth modality as "recommended" and treats the RGB + Skeleton combination as the "required" minimum configuration.

1.2.2 Simulator ENUM Code System

WIA Sign Language Recognition Standard adopts the following ENUM code system, used identically across simulator, API, and SDK.

• KSL, ASL, BSL, JSL, CSL, DGS, LSF, IS — Sign language codes (ISO 639-3 aligned)
• VIDEO, IMAGE, SKELETON, DEPTH — Input types
• HANDSHAPE, LOCATION, MOVEMENT, ORIENTATION, NMM — Five suhwaso distinctive features
• WEBSOCKET, REST, GRPC, MQTT — Communication protocols
• REALTIME, BATCH, OFFLINE — Processing modes
• CNN, LSTM, TRANSFORMER, CTC, ATTENTION — Model classes
• ISOLATED, CONTINUOUS, TRANSLATION — Recognition granularity

1.2.3 Confidence and Accountability — Constraints in High-Stakes Domains

The WIA standard explicitly bounds the limits of automatic sign language recognition. In high-stakes domains such as medical diagnosis, legal testimony, and public safety, automatic recognition output must not be the sole basis for decisions. Instead, the participation of a state-certified sign language interpreter is required. The standard always reports a confidence value alongside any recognition result, and any region with confidence below 0.85 is flagged "human interpreter review recommended."

1.3 Korea-Specific Infrastructure

Since the enactment of the Korean Sign Language Act (Act No. 18334, 3 February 2016), the Republic of Korea has built a cooperative infrastructure spanning government ministries, national institutions, research institutes, educational institutions, broadcasters, and civil society. The WIA standard actively leverages this infrastructure.

1.3.1 Government Ministries and National Institutions

• Ministry of Culture, Sports and Tourism (MCST): The lead ministry for the Korean Sign Language Act, MCST publishes a five-year "Korean Sign Language Development Master Plan." The Second Master Plan was issued in 2022.
• National Institute of the Korean Language (NIKL): NIKL is responsible for the KSL Dictionary, KSL corpus construction, and KSL lexical standardization. The KSL Dictionary (officially released in 2017, approximately 16,000 entries as of 2024) is the primary lexical source for this WIA standard.
• National Hangeul Museum (NHM): Maintains a digital archive of KSL visual materials and operates the "Words and Signs" permanent exhibition.
• National Gugak Center: Accumulates case studies of combining traditional performing arts with sign language interpretation.
• Ministry of Health and Welfare (MOHW): Oversees Deaf person registration under the Welfare of Persons with Disabilities Act and hearing-disability medical support. Registered Deaf persons numbered approximately 420,000 in 2024.
• Ministry of Education (MOE) and National Institute of Special Education (NISE): Operates the KSL curriculum across 18 schools for the Deaf and hearing-impaired nationwide.
• Korea Communications Commission and Ministry of Science and ICT (MSIT): Manages mandatory sign language broadcast programming on terrestrial, general programming, and news-focused channels under Article 69 of the Broadcasting Act and the related notification on disabled-persons broadcast access.
• National Human Rights Commission of Korea (NHRCK): Handles discrimination complaints arising under the Anti-Discrimination Against and Remedies for Persons with Disabilities Act (2007),⁴⁾ including refusal of sign language interpretation services.
• Ministry of Employment and Labor (MOEL): Operates a sign language interpreter support program for Deaf workers.
• Ministry of the Interior and Safety (MOIS): Establishes standard procedures for sign language interpretation in disaster broadcasts and public-institution announcements.

1.3.2 Deaf Community Organizations

• Korean Association of the Deaf (KAD): Founded in 1946, KAD is the Deaf-led representative organization for the Korean Deaf community. With 17 regional chapters and 219 local branches, it administers KSL teacher certification, sign language interpreter examinations, and Deaf rights advocacy. The WIA standard adopts the "Deaf-First" principle recommended by KAD as its ethical foundation.
• Korean Sign Language Education and Training Institute: Trains and develops KSL teachers, producing approximately 200 sign language teachers each year.
• Korean Sign Language Research Institute: Carries out academic research on KSL and compiles KSL grammars.
• Korea Association for Welfare of the Hearing Impaired: Operates social integration programs for the hearing-impaired population.
• KSL Research Center under KAD: Operates a neologism discovery and lexical registration committee.

1.3.3 Schools for the Deaf — Hubs of KSL Education

Eighteen special schools serve Deaf students nationwide. Seoul School for the Deaf (founded 1909, the oldest in Korea), Daejeon School for the Deaf, Busan Sungsim School, Incheon Sunhwa School, Gwangju Sunmyeong School, Daegu Yeonghwa School, Gyeonggi Ansan Jahae School, and Chungbuk Cheongju Sungsim School are core centers for KSL education. The WIA standard incorporates a subset of pedagogical video materials from these schools as training data, with personal information and image rights protected under the Personal Information Protection Act and the Welfare of Persons with Disabilities Act. The student population at schools for the Deaf is approximately 1,800 (2024), reaching about 5,000 when mainstreamed Deaf students in inclusive education are included.

1.3.4 Research and Technology Institutes

• Electronics and Telecommunications Research Institute (ETRI): Developed the "Su-eo Tok" Deaf interpretation terminal and the "KETI-Hand" real-time interpretation engine (since 2019), achieving approximately 93% recognition accuracy on KSL as of 2024.
• National Information Society Agency (NIA) AI Hub: Operates the publicly available "AI Training Korean Sign Language Video Dataset" (approximately 110,000 lexical entries, 1,000 hours of video).⁵⁾ It is Korea's primary data infrastructure for sign language AI research.
• Korea Advanced Institute of Science and Technology (KAIST): The Sign Language Recognition Lab at the AI Graduate School has published Transformer-based continuous SLR models.
• Seoul National University and Yonsei University: Sign language linguistics laboratories conduct research on KSL grammar and computational processing.
• Korea Institute of Science and Technology Information (KISTI): Provides distributed storage and training infrastructure for sign language data via the KISTI-5 supercomputer.
• Telecommunications Technology Association (TTA): Develops the "Korean Sign Language Recognition and Synthesis Standards" (TTAK series), notably TTAK.KO-10.1156 (sign language annotation standard).
• Korean Standards Association (KSA) and Korean Agency for Technology and Standards (KATS): Manage Korean editions of KS X ISO/IEC 40500 (KS X 6383, web accessibility) and KS X ISO/IEC 24751 (IT learning accessibility).
• Korea Internet and Security Agency (KISA): Has issued a privacy de-identification guide for KSL video data.
• Korea Laboratory Accreditation Scheme (KOLAS): Accredits conformity assessment testing bodies for sign language recognition models.
• KCMVP: Conducts cryptographic module validation for Deaf-oriented communication terminals.
• Korea Institute of Science and Technology (KIST): Conducts research on humanoid robots and sign language generation (SLG).

1.3.5 Broadcasters — Mandatory Sign Language Programming

Korean broadcasters comply with mandatory sign language programming ratios under the Broadcasting Act.⁶⁾ As of 2024, KBS programs approximately 7% of its terrestrial main broadcasting with sign language interpretation, while MBC and SBS each program at least 5%. EBS broadcasts Deaf-student educational programming with sign language captioning throughout the schedule. National Assembly Television provides 100% sign language interpretation for live plenary sessions. Arirang International Broadcasting has piloted International Sign (IS) interpretation in select time slots. Compared with NHK's Japanese sign language news and the BBC's See Hear, Korea's sign language broadcasting policy is among the strongest in Asia.

Since 2025, "Emergency Disaster Broadcasts" have been required to allocate at least one quarter of the broadcast screen to the sign language interpreter, a joint administrative regulation issued by the Ministry of the Interior and Safety and the Korea Communications Commission to ensure that Deaf viewers can clearly identify the interpretation frame during emergencies.

1.4 Five Recognition Stages of the WIA Standard

The standard organizes sign language recognition into the following five stages, each mapped 1:1 to a simulator panel.

1. Input Stage: Raw streams are received from cameras, depth sensors, and inertial sensors. For KSL recognition, the signer's full upper body must be in frame; the recommended minimum resolution is 1280×720 and the recommended minimum frame rate is 30 fps.
2. Feature Extraction Stage: Keypoint vectors are extracted via MediaPipe Hands (21 keypoints × 2), MediaPipe Pose (33 keypoints), MediaPipe FaceMesh (468 landmarks), or OpenPose (BODY_25 + Hand).
3. Temporal Modeling Stage: The keypoint time series is fed into sequence models such as LSTM, GRU, or Transformer. The average duration of a KSL sign unit is approximately 0.7–1.2 seconds (20–36 frames at 30 fps).
4. Recognition Stage: CTC (Connectionist Temporal Classification)⁷⁾ or encoder-decoder attention architectures convert the sign-unit sequence to a lexical sequence.
5. Translation / Caption Stage: The lexical sequence is translated into natural-language sentences in Korean, English, and so on. For KSL → Korean, multilingual translation models such as mBART and NLLB-200 are commonly used.

1.4.1 Recognition vs. Translation — A Clear Distinction

Sign Language Recognition (SLR) converts sign signals into a sequence of sign-vocabulary IDs. Sign Language Translation (SLT), in contrast, converts that lexical sequence into a natural-language sentence. The WIA standard exposes the two tasks at separate API endpoints and reports SLR confidence and SLT BLEU / SignBLEU scores separately. This separation simplifies debugging, evaluation, and accountability.

1.4.2 Isolated vs. Continuous SLR

By input granularity, SLR is divided into two regimes.

• Isolated SLR: Recognizes a single sign at a time. Datasets include WLASL (ASL, 2,000 lexemes), KETI Sign Language Dataset (KSL, 419 lexemes), and AUTSL (Turkish Sign Language, 226 lexemes). Reported accuracy exceeds 95%.
• Continuous SLR: Recognizes naturally connected sentences. RWTH-PHOENIX-Weather 2014T (German Sign Language weather forecasts, 1,066 vocabulary, 8,257 sentences)⁸⁾ is the canonical dataset. State-of-the-art WER hovers around 21–26%, making continuous SLR substantially harder than isolated SLR. Real-world conversation depends on continuous SLR.

1.4.3 Comparison of Korean Sign Language Datasets

1.5 Contributions and Distinctive Features

1. Equal support for multiple sign languages: Although KSL is the primary reference, ASL, BSL, JSL, CSL, DGS, LSF, IS, and other major sign languages are all supported on equal footing.
2. Synchronous multi-modal alignment: Four modalities (RGB, depth, skeleton, hand landmarks) are required to share PTP-synchronized timestamps.
3. Alignment with Korean infrastructure: The standard directly integrates with the data, standards, and educational assets of NIKL's KSL Dictionary, NIA AI Hub, KAD, ETRI, KAIST, and the Korean Sign Language Education and Training Institute.
4. Alignment with law and accessibility: Aligns with domestic and international legal and accessibility frameworks including the Korean Sign Language Act, the Anti-Discrimination Against and Remedies for Persons with Disabilities Act, the Broadcasting Act, WCAG 2.2, and KS X ISO/IEC 40500.
5. Open standard under MIT license: All simulators, specifications, APIs, and example code are openly published in WIA-Official/wia-standards-public/tree/main/sign-language under the MIT license.

1.5.1 Global Comparison

The Korean Sign Language Act is recognized as one of the world's first statutes to declare a sign language as a public language in primary legislation. This WIA standard extends Korea's legal and institutional assets into the technical-standards domain. By combining Korea's network of schools for the Deaf, the KAD as Deaf-led representative organization, NIKL's standardization mandate, NIA AI Hub datasets, and ETRI's interpretation-terminal technology, Korea is well positioned to be the first leader of sign language recognition standards in Asia.

1.5.2 Code of Ethics — Deaf-First

The standard adopts the following seven ethical principles.

1. Linguistic respect: Treat every sign language as a complete natural language.
2. Participation of stakeholders: Reflect the perspectives of Deaf signers in standards development and revision.
3. Privacy protection: Treat sign language video as the highest tier of personal data because faces and bodies are identifiable.
4. Cultural sensitivity: Respect Deaf culture and Deaf identity.
5. Explicit accountability: Identify the party responsible for any error in automatic recognition output.
6. Cost equity: Do not pass costs onto the Deaf community.
7. Accessibility-first: Apply accessibility standards as the highest priority for every interface.

1.6 Structure of This Volume

This volume comprises the following eight chapters.

1. Chapter 1 (this chapter): Sign Language Recognition Standards Overview — KSL/ISL/ASL
2. Chapter 2: Vision-Based Sign Language Recognition — RGB/Depth/MediaPipe/OpenPose
3. Chapter 3: Multi-modal Hand Gesture, Facial Expression, and Gaze Alignment
4. Chapter 4: Sign Language Corpus and Annotation Standards (ELAN/SignBank/HamNoSys)
5. Chapter 5: Neural Network Models (CNN/LSTM/Transformer/CTC)
6. Chapter 6: Real-time Interpretation API and Communication Protocols
7. Chapter 7: Deaf Accessibility, Law and Policy
8. Chapter 8: Evaluation Metrics, Certification and the Future

Each chapter is required to include five elements: (i) the standard definition, (ii) primary-source citations, (iii) a Korean infrastructure mapping section, (iv) a deep-link to the simulator, and (v) end-notes. The interactive simulator provides five panels: Data Format, Algorithms, Protocol, Integration, and Test.

1.6.1 Note on Korean Edition

A Korean-language edition of this volume is published in parallel. The Korean edition follows the same chapter structure, the same primary-source citation list, and the same simulator and GitHub references. The Korean edition's "Korean infrastructure mapping" sections discuss KSL-specific contexts in greater depth, while this English edition retains the Korean infrastructure mapping at a level sufficient for international readers to understand the institutional landscape behind KSL.

1.6.2 Reader Guide

This volume targets four primary reader groups.

• Sign language recognition system developers should focus on Chapters 2 through 6 for technical specifications.
• Sign language linguistics researchers should focus on Chapter 1's linguistic foundations and Chapter 4's corpus standards.
• Broadcasting and telecommunications operators should focus on the real-time interpretation API in Chapter 6 and the legal and accessibility alignment in Chapter 7.
• Policy makers and standards drafters should focus on the global comparison in Chapter 1 and the certification and future outlook in Chapters 7 and 8.

We also welcome the participation of Deaf readers and members of the Deaf community in the planning, review, and revision of this volume. All content complies with WCAG 2.2 AA web accessibility, and KSL interpretation video and Korean and English captions are provided in parallel.

1.99 Closing Remarks and Korean Infrastructure Cross-Reference

The technical specifications described in this chapter all serve the broader goal of guaranteeing the everyday communication rights of the Deaf community. Korean Sign Language is, under Article 2 of the Korean Sign Language Act, a public language with status equal to Korean, and the WIA Sign Language Recognition Standard provides the technical underpinning for this legal status.

The standard operates on the Korean national infrastructure: the Korean Association of the Deaf (KAD), the National Institute of the Korean Language (NIKL), the National Information Society Agency (NIA), the Electronics and Telecommunications Research Institute (ETRI), the Korea Advanced Institute of Science and Technology (KAIST), the Korea Institute of Science and Technology Information (KISTI), the Telecommunications Technology Association (TTA), the Korean Standards Association (KSA), the Korean Agency for Technology and Standards (KATS), the Korea Internet and Security Agency (KISA), the Korea Laboratory Accreditation Scheme (KOLAS), and the Korea Communications Agency (KCA), as well as government ministries including the Ministry of Culture, Sports and Tourism (MCST), the Ministry of Health and Welfare (MOHW), the Ministry of Education (MOE), the National Institute of Special Education (NISE), the Korea Communications Commission (KCC), the Ministry of Science and ICT (MSIT), the National Human Rights Commission of Korea (NHRCK), the Ministry of Employment and Labor (MOEL), the Ministry of the Interior and Safety (MOIS), and the Ministry of Justice.

Broadcasters KBS, MBC, SBS, EBS, National Assembly Television, and Arirang International Broadcasting plan to adopt the standard's automatic captioning system; 5G operators SK Telecom, KT, and LG U+ apply the standard to Deaf telecommunication relay services; and AI providers Naver Clova, Kakao i, LG AI Research, Kakao Brain, and SK Telecom X publish KSL recognition APIs compatible with the standard.

The eighteen schools for the Deaf nationwide (Seoul School for the Deaf, Daejeon School for the Deaf, Busan Sungsim School, Incheon Sunhwa School, Gwangju Sunmyeong School, Daegu Yeonghwa School, Gangwon Provincial Dowon School, Chungbuk Cheongju Sungsim School, Chungnam Cheonan Inae School, Jeonbuk Iksan Jeil School, Jeonnam Gwangju Yeonghwa School, Gyeongbuk Andong Yeongmyeong School, Gyeongnam Jinju Hyegwang School, Jeju Yeongji School, Incheon Cheonghak School, Gyeonggi Ansan Jahae School, Ulsan Meari School, and Sejong Sarang School) serve as the standard's KSL education hubs.

This standard is published as an open standard under the MIT license; all simulator code, specifications, and example code are openly available at GitHub WIA-Official/wia-standards-public/tree/main/sign-language. Our hope is that this standard enables Deaf people who use Korean Sign Language to communicate more freely, and that this standard becomes the foundation for sign-language recognition standards across the Asia-Pacific region.