An example of sign language expression as multi-channel language excerpted from the dataset [22]. Annotators translated the Korean sentence "기록적 폭우로 산사태 위험지역 접근금지 및 산림 인근 주민은 지정된 장소로 즉시 대피하여 주시기 바랍니다. (Due to the record heavy rain, access to the landslide risk area is prohibited, and residents near the forest are requested to evacuate immediately to the designated area.)' and then transcribed.

An example of sign language expression using a graph structure

Grammatical roles for the eight types of transcribed NMSs

A summary of the large-scale sign language datasets.

Distribution rates by the number of word occurrences of the German-DGS parallel corpus

Statistics of the scripts of three drama series

Distribution rates per word frequency on the scripts of three drama series and combinations of drama series

Examples of Korean and KSL sentences

A transcription parsing process for a KSL sequence

Examples of Korean and KSL sentences. *Corpus construction for the drama "Are You Human?'s is in progress.

A distribution of transcription errors by error type

An overview of our translation system. First, the named entity transformation module transforms input sentences (see (A)). Next, a PLM-based Encoder (e.g., BERT encodes the transformed

An example of fingerspelling word transformation

Representative expressions per number type with examples

Using a PLM as an encoder

Statistics of training, validation, and test subsets for the ETRI Ko-KSL dataset. OOV is the abbreviation of "out-of-vocabulary'. *The number of words is calculated by splitting sentences in morpheme-level.

Statistics of training, validation, and test subsets for the NIASL2021 Dataset. OOV is the abbreviation of"out-of-vocabulary', *The number of words is calculated by tokenizing sentences using the same PLM tokenizer for our SLT models.

Translation scores for all models on the validation/test subsets of the ETRI Ko-KSL dataset. "NE' is the abbreviation of Named Entity. The figure in bold indicates the top score by the respective metric.

Translation scores for all models on the validation /test subsets of the NIASL2021 Dataset. "NE' is the abbreviation of Named Entity. The figure in bold indicates the top score by the respective metric.

Translation scores by increasing the size of parallel data

Comparison oftranslation scores ofPLMs. L denotes the number oflayers ofa PLM and H denotes the number ofmulte-head attentions ofa PLM.

Translation scores depending on hyperparameter settings.

Qualitative results for all models

Pearson correlation coefficients between sentence-level BLEU-4 score and human evaluation scores (Accuracy & Comprehension)

An example ofsign language production using a commercial avatar player [26]

Naturalness and accuracy scores per category

Pearson correlation coefficients between sentence-level BLEU-4 score and human evaluation scores (Naturalness & Accuracy).

Average identification rates by participants for named entities

Four representative types of errors

An example of BFS-based graph linearization

Special tokens added the vocabulary of our translation model

An example of extracting n-grams from a graph

An example of converting a sequence to a directed graph (edges are only allowed from left to right nodes)

Statistics of training, validation, and test subsets for the NIASL2021 Dataset. OOV is the abbreviation of"out-of-vocabulary'' *The number of words is calculated by tokenizing sentences using the same PLM tokenizer for our SLT models.

Translation scores for all models on the validation/test subsets of the ETRI Ko-KSL dataset. "NE' is the abbreviation of Named Entity. Thefigure in bold indicates the top score by the respective metric.

Case 1. The seq2graph model successfully generates a gloss sequence with non-manual signals.

Case 2. The seq2graph model successfully generates a gloss sequence with spatial and non-manual signals

Error case 1: node misprediction

Error case 2: syntactic difference