Student Reactions to Concordance Data:A Task Using I-D Word Identification

Student Reactions to Concordance Data: Reflections on A CALL Based Task Using I-D Word Identification and Approaches for Future Research.


This is a report about one experience with using a part of Tom Cobb’s(1997) Compleat Lexical Tutor called I-D Word Identification with a group of my own students. I wrote an article about this and other tools on this website in the Wired section of 2009 November’s issue of The Language Teacher published by the Japan Association for Language Teaching.

I-D Identification is a tool which takes a list of  high frequency vocabulary words, like the first or second one thousand words of the British National Corpus or words from the Academic Word List  and hides one word in a jumble. With this jumble comes a set of concordance lists with the hidden word removed, simulating a cloze-like task. The student has to find the word with the help of the concordances.

While the data collection for this report is inconclusive, this paper also offers ideas for future research: two tools will be reviewed, plus other research approaches to provide a fuller picture of how students use and respond to concordance data.

What is a concordance?

Concordances are one kind of output derived from a corpus search in which the node word is listed with all the connected words to the left and right of the node word are presented. In this way, the user can know how the word is used in context and what kinds of words are frequently connected to it.

A screen shot of what a student would encounter in the I-D task.

The method and participants

This task was performed by third year Japanese high school students on September 16, 2009 during 5th and 6th period  in the computer lab. The class was an elective class for oral communication which is team-taught by two teachers. I was asked to give a Power Point presentation about my home town L.A. for the first part of the two period lesson.  Afterward I had the students do some Internet research about L.A. and find five new words that they had learned from the search.

Nine students participated in the lesson, and they later formed pairs or groups of three to conduct the Internet research. The final task  was to use those words in the I-D task. The students worked on the I-D task as a pair initially and after they worked individually once they grew accustomed to the process.

What I wanted to do was use the list of new words that each student had discovered in their Internet research and apply that to the I-D Word task on the Internet. I made sure to exploit the save option included on the website and gave the link to the students so they can access it on their own computers. One snag that I discovered is that the words the students selected were so low frequency that the program couldn’t find any concordances to offer hints.My backup plan was to have the students do the regular tasks on the website.

Once the students started, I informally observed how they responded to the task.


With the students, it seemed that the focus of their efforts were to find the hidden word,by concentrating on the jumble presented but some seemed to ignore the concordance lines while some did not.  One way to tell is if the student uses the cursor to scan the concordance lines.   The teacher with whom I team teach noticed that the students seemed to avoid using the concordances as well. One such reason was that the number of lines offered seemed overwhelming to the students, and so they ignored them. Another reason that my colleague noted was the complexity of the examples: the students could not readily understand the examples due to their vocabulary level and ability.

But the task also engaged  the students. Everyone focused on the task at hand, most likely because it was via a computer, rather than on a printed page. Also the interactive quality, where the student had to highlight the word with their cursor to find the right word, and the gratification of answering correctly also contributed to this engagement. Also the fact that anyone can save their work and post it on the Internet was also motivating. One such student found websites of her favorite soccer team in English, found words that she enjoyed for her own task and saved it on-line. I then projected the link on a large screen for other students to try.

Conclusion and Reflections for future research.

It seems based on the observations listed above that there was a mismatch with the skills of the students with the material presented in the concordances. One approach that would be to have a ready made I-D task prepared in advance rather than have them search for the words in the Internet research activity.  It would have prevented the students from selecting low frequency words, which actually forced me to change the task radically. In so doing the I-D task would have been tailor made for their level. These words would have been culled from the presentation itself, using higher frequency words to expand their vocabulary. Such a list could be drawn from the presentation using Cobb’s vocabulary profiling tool found on the same website as the I-D Task page.

One question does arise: how many examples of a concordance are enough to help or hinder a student to grasp the meaning of a word or phrase? With the students in this class, the number of lines seemed to overwhelm, but if they were given only five, or ten, would it make a difference?

In the future  to ensure the validity of an observation as this is to use a checklist and observe each students actions with the mouse.This is one way to acquire data on a qualitative level, but if we need more concrete, quantitative data, we need something to measure the the choices our students make, especially when they are presented with concordances on the computer. How do we know they are even using them in the first place?

One tool which has potential to leave a record of mouse movements called IO graph created by Anatoly Zenkov and Andrey Shipilov (2010). The movements of the mouse leave a line where the pointer has passed. What is also interesting is that a circle is left in the spot where the pointer sits for a protracted period of time, getting larger and larger  the longer it stays in one spot. Even though this tool is envisioned for artistic purposes, it also has the potential to reveal how  a student behaves in relation to on-line tasks, such as reading concordances. If the student uses the mouse to read, we can know how long they dwell at a certain section.

Another behavior that I noted while observing are some students’ tendency to talk to themselves or to their partner while working the task. What they are saying may hint at their thinking process.  I have noticed this with one or two students, in which they unconsciously verbalized their choices. Another way to gather more conclusive, qualitative data is through this method, called a “think aloud”, where the participant is encouraged to talk aloud about their thoughts in response to the task.   Also having the students to record their scores would be another more quantitative way to approach this investigation. Are the scores improving over time or worsening?

I believe that data regarding students working with concordances are far and few between. At the same time this record is just a step toward looking at how students respond to concordance data, and how this can inform CALL education.


Cobb, T. (1997) The Compleat Lexical Tutor [website]
University of Montreal, Quebec. Retrieved on
September 10, 2009 from <>.

Parise,P. (2009)Take corpus linguistics in your own hands with the compleat lexical tutor.The Language Teacher.31(11) 24-26.

Zenkov,A. & Shipilov, A. (2010) IO-graph [computer program] retrieved from

9 thoughts on “Student Reactions to Concordance Data:A Task Using I-D Word Identification”

  1. Hey Peter, Tom here/Lextutor

    I have finally got around to linking your study from the ID-Word entry page. Maybe you will be interested to know that ID is now fully functional on Mac Safari and Firefox.

    Meantime, have you looked further into the IO idea for investigating computer use?

    Best, Tom

    1. Hi Tom,

      Thank you for taking the time to comment on my article here.
      At the moment I haven’t had a chance to repeat this experiment, but I invite anyone to try to replicate this study using IO Graph or any other similar device. I found a paper on a similar program called Cheese (Mueller & Lockerd, n.d) where the authors describe how the traces of mouse movement relates to the readers movements on a web page. I think such data would be helpful for research on how students respond to concordances.

      It great to hear that ID can work on Safari and Firefox, and maybe sometime down the road Chrome too? I haven’t used Explorer for years now but am an avid Firefox user.

      Speaking of years….

      Happy New Year!


  2. Thanks for posting this report on your use of the I-D tool. The blanking out of the search word and offering a number of lines is a very powerful context search. It reminds me of Tim Johns’ game ‘Contexts’ which was available to play at the university of Birmingham before Windows (it was MS DOS). However the problem with I-D is that the hiding of the word in random letters becomes the game in itself. The student has no ‘need’ to look at the concordance lines really – they just need to find feasible strings of letters. I think the game would work much better if all the option words were given (however many they are) and the student needs to choose the right one. Then the only way to do it would be to look at the concordance lines. I would be delighted if the game could be tweaked to allow that instead of burying the words in jumbled letters. (That is a nice game to play but without concordance lines)

    1. Thank you Tilly for taking the time to read and comment,

      When I did this I found it was hard to tell if they were just trying to find a pattern or if they were really using the concordances, hence my suggestion to screen-cast their desktops and try the think-aloud technique. If two or three words are presented like you suggest then that would be a more thorough challenge for the learner. That would be a nice variation of this tool. You could throw in inflections for example “think” vs “thinking” vs “thought” and the concordance could hint at the right selection and at the same time show its proper usage. I think if you contact Tom Cobb at he might tweak it, if he has the time. I would love to build one of these applications myself but need to get up to speed on Java to make it happen…. Unless someone out there knows of a good Java online course I could try…

      Thank you,


  3. HI Peter,
    I like the idea of offering all the variations of the word’s lemma – that also would ‘concentrate the mind’! It could be fairly automateable I assume (for the academic word list at least). I will try to find time to contact Tom Cobb.

  4. Hi Peter,

    Looking forward to trying some thing like this in the future. Just finished Susan Hunston’s Corpora in Applied Linguistics which suggests using prepared corpora to avoid the context problem. Also, Rules, Patterns and Words by Willis is full of ideas.


    1. Thank you for reading Peter,

      I think when I did this I was really interested in how learners would respond to concordances. The game on lextutor is one way to expose them. I am also beginning to find that they need a specific guide to make sense of concordances.
      I will have to have a look at Hunston again, as well as Willis. Thank you for suggesting these.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: