Using Eye-Tracking Study to

Investigate Lexicon and Passage Reading

of Hong Kong Deaf Individuals

XU Yue irenexuy@link.cuhk.edu.hk
Supervisor：Prof. Philip Thierfelder
01 My research project

02 How did we come up with my research topic?

CONTENTS

03 Timeline

04 Advice
Research background

Problem: Deaf individuals might experience a significant delay � literacy development

or encounter difficulties in learning to read (Kyle & Cain, 2015; Marschark et al., 2009).
Significance: Reading skills play a crucial role in future academic performance and
personal development � deaf people are able to achieve age-appropriate reading skills
(Harris & Moreno, 2004; Talwar et al., 2023; Walker et al., 1994).

the study of the unique characteristics of deaf natural reading

Research background

Chinese Writing system: Characters - Words - Sentences

 Each Chinese character represents a morpheme and a syllable (M. Wang et al.,
2006)
 Exception:
• Single morpheme words composed with multiple Chinese characters
(loanwords):
• Cantonese: “檸檬”(ning4 mung1), “士多啤梨” (si6 do1 be1 lei2)
• Mandarin: “菩薩” (pu2 sa4), “巧克力” (qiao3 ke4 li4)
• Part of Speech (POS)
Research background • Word frequency
• Number of strokes
• Homophone density
• Age of acquisition
Chinese Writing system: Characters - Words - Sentences • Imageability and concreteness,
etc.
 Chinese words are not always predicted solely based on the constituent
morphemes.
• 電腦 � computer
• 電 � electricity, 腦 � brain
 The meanings of Chinese characters are context-dependent and multi-character
items are more frequently required to mean something in modern Chinese (Shei,
2014). -> Concepts are best expressed through words
My research project: Research questions • Word frequency
• Number of strokes
• Homophone density
• Age of acquisition
• etc.

1. How do lexical properties and categories affect deaf passage reading?

2. Would visual perceptual skills and reading fluency predict certain reading
patterns or characteristics of the DHH?

3. Are there any similarities and differences in how these lexical variables
affect reading between the Deaf or Hard of Hearing (DHH) and the Normal
Hearing (NH)?
My research project: Methodology

Participants

N
DHH readers 23
NH readers 21

 Both groups are matched in age and educational background.

 Personal information sheet
 Several background measures: Reading fluency & Visual Perceptual Skills
My research project: Methodology

Background measures
Reading Fluency Test: To test subjects’ ability to read quickly and accurately (Rasinski, 2012)
 43 sentences adapted from Thierfelder et al. (2020a)
 What to do: To read as many sentences as possible within a two-minute time limit +
To determine whether each sentence is correct or not.
 Grading criteria:
Scores = The number of characters in the correctly labeled sentence - the number of characters
in the incorrectly labeled sentence
My research project: Methodology

Cloze Test: To validate readability while measuring reading comprehension (Shanahan et al.,
1982; Van Dijk & Kintsch, 1983).
 Open-ended cloze test with 3 short texts adapted from the Chinese Language Proficiency
Test (HSK) (Peng et al., 2021).
 What to do: Fifteen fill-in-the-blank items within 10 minutes � make the text more
coherent and cohesive.
 Grading criteria:
Score = The number of correct answers
My research project: Methodology

The Woodcock-Johnson IV Visualization - Spatial Relations and Block Rotation:

 To evaluate spatial relationships and the ability to mentally manipulate visual
representations � visual perceptual discrimination ability (Schrank, 2014; Schrank et al.,
2016).

WJIV-Spatial Relations
WJIV-Block Rotation
My research project: Methodology

Descriptive information of the participants in the deaf and the hearing group
Deaf Hearing
Sample size 23 21
Female (%) 56.5 71.4
Mean age (Age range) 29.04 (20-44) 28.67 (20-44)
Educational background (%)
High school diploma 8 (34.7) 4 (19)
Associate, higher diploma, higher certificate 4 (17.3) 5 (23.8)
Bachelor 9 (39.1) 9 (42.8)
Above Bachelor 2 (8.6) 3 (14.2)
Mean (SD) of Reading Fluency Test*** 18.09 (8.65) 30.38 (7.37)
Mean (SD) of Cloze Test** 11.52 (3.67) 13.81 (1.36)
Mean (SD) of WJIV-Spatial Relations 17.57 (2.09) 17.67 (3.18)
Mean (SD) of WJIV-Block Rotation 15.22 (2.80) 16.33 (3.01)

Note. * p<.05, ** p<.01, *** p<.001. T-tests were conducted for comparison of Reading Fluency Test, Cloze Test, WJIV-Spatial Relations,
and WJIV-Block Rotation between the deaf and the hearing. For education background, total number of participants with percentage of
participants in parentheses.
My research project: Methodology

Stimuli
 Paradigm: Natural reading paradigm
 20 passages are extracted from The Little Prince (Traditional Chinese ver.)
• Each paragraph ≤ 10 lines
• Each line ≤ 33 characters
• 1.5 spacing
• Summary:
• 240±61 characters
per passage
• A total length of 5422
characters (including
punctuation markers).
My research project: Methodology

Stimuli
 10 passages: To make a judgment concerning the content of the passage within a 10-second
time limit.
Title of the passage Comprehension judgement Answer

獨一無二的幻象 小王子認為他的那朵花是整個宇宙中獨一無二的。 L. 錯誤
S. 正確 L. 錯誤
小王子與點燈人的對話 小王子對點燈人星球上的日落次數感到驚嘆。 S. 正確
S. 正確 L. 錯誤

 Word segmentation:
• Standard: the Academia Sinica Standard (Huang et al., 1997)
• Example: *我*喜歡*吃*蘋果*。*
Area of Interest (AOI) 1 AOI 2 AOI 3 AOI 4
My research project: Methodology

Lexical variables:
zipf_frequency Raw descriptive statistics of lexical variables
function in Variables Mean (SD) Min Max
wordfreq
Word frequency 5.67 (1.27) 0.75 7.79
python library
(Speer, 2022) Number of strokes 12.88 (7.68) 1.00 49.00
Word length (1-4) 1.49 (0.61) 1.00 4.00
Homophone density 2.96 (3.09) 1.00 28.00

Note. The mean of each lexical variable was calculated over 4334 word tokens.

 To convert each word into its corresponding Jyutping (粵拼):

characters_to_jyutping function in the PyCantonese package (J. Lee et al., 2022)
 To calculate the total number of words with the same pronunciation and the
same tone using CC-Canto dictionary (License: CC-BY-SA-3.0)
My research project: Apparatus

 Eyetracker: Eyelink 1000 Plus Desktop Mount with a sampling rate of 1000Hz
 Monitor: 24-inch with a resolution rate of 1920 × 1080 and refresh rate of
144Hz
 Each sentence was in black with SimSun 16-point font on a white background.
 The participants read the texts binocularly while we collected the eye-movement
data of the right eye from them.
My research project: Data analysis

 Data exportation - Eye-tracking measures

 Early measures: First Fixation Duration (FFD), Gaze Duration (GD)
GD
 Late measure: Total Reading Time (TRT) FFD

 Data filteration
我 喜歡 吃 蘋果
 Data analysis
TRT
• Environment: R (version: 4.2.3)
• Models: Linear Mixed Models (LMM) � lmerTest package (Kuznetsova et
al., 2017)
My research project: Data analysis

 Exploration of each group:

• Fixed factors: Word frequency, Homophone density, Word length, Number of strokes,
Reading fluency
• Random factors: Participants, Words
 Comparison between DHH and NH groups:
• Fixed factors: Hearing status (DHH, NH), Word frequency, Homophone density, Word
length, Number of strokes, Reading fluency
• Random factors: Participants, Words
My research project: Findings

RQ1: How do lexical properties and categories affect deaf passage reading?
RQ2: Would visual perceptual skills and reading fluency predict certain reading patterns or
characteristics of the DHH?
• Word frequency: Higher-frequency words were processed more efficiently than lower-
frequency words in FFD, GD, and TRT.
• A notable facilitating effect of word length in the FFD of the deaf group.
• Deaf with higher reading fluency:
a) process words more efficiently, and their processing efficiency improved as word
frequency increased.
b) have a sustained advantage of processing words with increasing horizontal
complexity throughout eye-tracking time courses from FFD, GD, to TRT.
My research project: Findings

RQ3: Are there any similarities and differences in how these lexical variables affect reading
between the Deaf or Hard of Hearing (DHH) and the Normal Hearing (NH)?
• Fixation times and accuracy:
a) No significant differences between the two groups in terms of FFD, GD, and TRT.
b) NH group had significantly higher accuracy of passage reading comprehension
judgements than their DHH counterparts.
• Homophone density:
a) The deaf demonstrated no influence by homophone density.
b) The hearing presented a greater reliance on phonological information.
My research project: Findings

RQ2: Are there any similarities and differences in how these lexical variables affect reading
between the Deaf or Hard of Hearing (DHH) and the Normal Hearing (NH)?
• Word frequency:
a) The deaf group had a greater decrease in TRT for words with higher frequency than the
hearing group.
• The number of strokes and word length:
a) The deaf exhibited a relative visual processing advantage � The hearing group
experienced a notably greater increase in GD and TRT for words that were longer than
the deaf group.
How did we come up with our research topic?

Brainstorming
My interest:
 Content: Literacy development, reading and writing, lexical-semantic processing
 Subject: Deaf individuals
 Empirical skills/Research training: Experimental methods
 e.g. eye-tracking, electroencephalography (EEG)

Keywords: Reading, Deaf or Hard of Hearing (DHH), Experiment

How did I come up with my research topic?

Brainstorming Narrow down

Explore: Literature review
 What research has been done in the past that is relevant to the topic.
 To find interesting gaps or debates.

The necessity of paying attention to deaf passage reading

• No consensus on how word properties and categories affect deaf reading.
• Chinese passage reading remains understudied.
a) Research gap: Few studies have focused on on the performance of deaf
people in terms of lexical decision in traditional Chinese reading
b) Method/Paradigm: A large number of research use visual word recognition
tasks or single sentences to study the reading process
Timeline

Deadline Action
Pre-experiment 2024.2 Design research materials (stimulus)

2024.3 Finish stimulus design + pilot test + recruit participants + Write Introduction section

During-experiment 2024.4-2024.5 Doing experiments (1.5 month) + Write Literature Review and Methodology

After-experiment 2024.5-2024.6 Data analysis (1-1.5 month) + Write Data analysis, Results, Discussion, and
Conclusion + Revision
Timeline
My expected timeline…

Deadline Action should be taken

Pre-experiment 2023.12.31 Design research materials (stimulus)

2024.1 Finalize topic selection with supervisor and begin dissertation research and writing;
Recruit participants Finish stimulus design + pilot test

During-experiment 2024.2 Finish all the experiment; Write introduction, literature review, method sections

After-experiment 2024.3 Data analysis; Write the rest of the sections if possible (If not, at least most parts of
results and discussion); Finish Draft 1 (if possible)

2024.3 Revision (draft 2, 3, final ver.)

Timeline

Time management:
 Start your research project as early as possible.
 Give each section more time than you think it
will take.
Other Advice

 Attitude: Be curious, persistent, flexible

 Communication:
 With peers:
 Share your research topics and ideas
 Discuss possible solutions, etc.
 With supervisor:
 Regularly update your research progress
 Seek support, etc.
References

Harris, M., & Moreno, C. (2004). Deaf children’s use of phonological coding: Evidence from reading, spelling, and
working memory. Journal of Deaf Studies and Deaf Education, 9(3), 253–268.
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects
models. Journal of Statistical Software, 82(13). https://orbit.dtu.dk/en/publications/lmertest-package-tests-in-
linear-mixed-effects-models
Kyle, F. E., & Cain, K. (2015). A comparison of deaf and hearing children’s reading comprehension profiles. Topics in
Language Disorders, 35(2), 144–156.
Lee, J., Chen, L., Lam, C., Lau, C. M., & Tsui, T.-H. (2022). PyCantonese: Cantonese linguistics and NLP in python.
Proceedings of the Thirteenth Language Resources and Evaluation Conference, 6607–6611.
https://aclanthology.org/2022.lrec-1.711/
Marschark, M., Sapere, P., Convertino, C. M., Mayer, C., Wauters, L., & Sarchet, T. (2009). Are deaf students’ reading
challenges really about reading? American Annals of the Deaf, 154(4), 357–370.
Peng, Y., Yan, W., & Cheng, L. (2021). Hanyu Shuiping Kaoshi (HSK): A multi-level, multi-purpose proficiency test.
Language Testing, 38(2), 326–337. https://doi.org/10.1177/0265532220957298
Schrank, F. A., Decker, S. L., & Garruto, J. M. (2016). Essentials of WJ IV cognitive abilities assessment. John Wiley
& S o n s .
https://books.google.com/books?hl=en&lr=&id=VASmCgAAQBAJ&oi=fnd&pg=PR15&dq=WJ+IV+Visualizatio
n+adult&ots=7KQUa3mnjZ&sig=0EX8sMLM08mLQQxYJYlQTHIsvbc
References

Schrank, F. A., McGrew, K. S., Mather, N., Wendling, B. J., & LaForte, E. M. (2014). Woodcock-Johnson IV tests of
cognitive abilities. Riverside.
Shanahan, T., Kamil, M. L., & Tobin, A. W. (1982). Cloze as a measure of intersentential comprehension. Reading
Research Quarterly, 229–255.
Shei, C. (2014). Understanding the Chinese language: A comprehensive linguistic introduction. Routledge.
https://books.google.com/books?hl=en&lr=&id=yGOvBAAAQBAJ&oi=fnd&pg=PP1&dq=Chinese+units+of+lan
guage&ots=ZSeAClOzDb&sig=dvRiNMqr77eNYj0lhtr4sYs9Il0
Speer, R. (2022). rspeer/wordfreq: V3. 0. Version v3. 0.2. Sept.
Talwar, A., Magliano, J. P., Higgs, K., Santuzzi, A., Tonks, S., O’Reilly, T., & Sabatini, J. (2023). Early Academic
Success in College: Examining the Contributions of Reading Literacy Skills, Metacognitive Reading Strategies,
and Reading Motivation. Journal of College Reading and Learning, 53(1), 58–87.
https://doi.org/10.1080/10790195.2022.2137069
Thierfelder, P., Wigglesworth, G., & Tang, G. (2020a). Orthographic and phonological activation in Hong Kong deaf
readers: An eye-tracking study. Quarterly Journal of Experimental Psychology, 73(12), 2217–2235.
Van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension.
https://www.academia.edu/download/77815914/Teun_20A_20van_20Dijk_20__20Walter_20Kintsch_20-
_20Strategies_20of_20Discourse_20Comprehension.pdf
References

Walker, D., Greenwood, C., Hart, B., & Carta, J. (1994). Prediction of School Outcomes Based on Early Language
Production and Socioeconomic Factors. Child Development, 65(2), 606. https://doi.org/10.2307/1131404
Wang, M., Cheng, C., & Chen, S.-W. (2006). Contribution of morphological awareness to Chinese-English biliteracy
acquisition. Journal of Educational Psychology, 98(3), 542.
黄居仁, 陈克健, 陈凤仪, 魏文真, & 张丽丽. (1997). 《 资讯处理用中文分词规范》 设计理念及规范内容. 语言文
字应用, 1, 94–102.