Data quality can affect key dependent variables in eyetracking.

-
Data quality – boring, I know - but really important!
We’ve just published a paper on eyetracking data that in some ways was similar to a debate that’s been happening about brain connectivity in autism. Several recent papers (see here) suggested that people with Autism may show reduced connectivity between different areas of the brain. So there was great shock and horror when a paper raised the possibility that, in fact, this may be not because there is a genuine difference between the brains of typical people and those with Autism – but rather because people with Autism may just tend to move slightly more while they are in the scanner. The article raised the possibility that this may create the impression of reduced connectivity between brain areas, without this actually being the case.
Eyetracking, in which you present still images or movie clips to participants and track where on-screen they are looking, is a powerful and versatile method that is widely used within infant psychology. Most researchers use the built-in software provided by eyetracker manufacturers – which sets out, probably deliberately, to make researchers feel very confident in their results. When you are viewing a live replay of the gaze data, for example, the use large amounts of smoothing and interpolation to give the impression that the tracking looks very sensible, and real.
In fact, though, if you visualise the raw data that eyetrackers are recording, you see that in fact we shouldn’t necessarily have that much confidence at all. In particular, one problem is that the same eyetracker, working in identical conditions, can record very good quality from one child, and very bad quality from another child. In the paper we identify two problems in particular – one, low precision, and the other, low robustness. We also show that, on average, worse quality data tends to be recorded from younger (relative to older) participants, from fidgetier participants (as opposed to those who sit very still, and later (relative to earlier) in a testing session.


This would be OK if it were a random source of error – unpredictable measurement noise that did not systematically influence results. In fact, though, as we also show, data quality shows strong relationships with several different aspects of data quality. See the example below, in which we take a sample of real data obtained during the recording of a face (top left) and analyse it for the proportion of time the participant has spent looking at the eyes vs other areas of the face (top right). We then manipulate the data to simulate the effect of low precision (bottom left) and repeat the same analyses (bottom right). It appears that the bottom sample shows less looking to the eyes area – but in fact the only difference between the two samples is data quality. We also conduct a number of different analyses looking at other aspects of data quality – such as fixation duration and various reaction time measures.
If you’re interested you can read more about this here, here and here.

x

Comments

Post a Comment

Popular posts from this blog

Parental neural responsivity to infants’ visual attention: how mature brains influence immature brains during social interaction.

-
The neuroimaging techniques that we currently have work best for studying brain activity when one person is on their own, looking at a computer screen. But, as I’ve argued here , this is useful for understanding some, but not all, of the ways in which we actually pay attention in the real world. In particular, current neuroimaging techniques are less well suited for helping us understand the dynamic influences of social context. This is particularly important, of course, for early development – given that infants spend the majority of their early lives interacting with caregivers. We know that, when one social partner pays attention to an object, this increases the attention that another social partner pays to that object . But how are these types of short-term, transient influences on attention substantiated in the brain? In our new paper, which is out now in PLoS Biology , we looked at this... We asked parents and the infants to engage in free, tabletop play with toys in two set...
Read more

How the differences between what babies are incredibly bad at, and what they’re incredibly good at, might all lie on one dimension.

-
I’m sitting watching an eye gaze trace of an infant, doing a short-term memory experiment in the next door room. It’s a simple task. There are two windows on the screen and no distractions. An elephant appears in one window, then disappears, then a flower appears in the middle between the two windows, and then disappears again. If they look back to the window where the object disappeared, they get a reward. The baby who’s doing the experiment is 12 months old. Previously, outside in the waiting room, he was showing off his cruising skills, walking around the room, flashing coy smiles at the research assistant. His mother pulled out her phone to take a photo and he recognised what was coming, and struck a pose that he knew would make us all laugh. On the eye gaze trace that I’m watching I can see that the baby has followed the object, seen it disappear, and looked to the flower for about 500ms (half a second), until it disappears. I can see, then, from his eye movements (and from h...
Read more

Should we only use the prior literature as the basis for deriving new hypotheses?

-
Image
"Reviewer 1 raises more troubling issue: the sense that this paper emerged from ad hoc observations [...] and the consequent lack of a coherent theoretical framework to drive the hypotheses." I've just had the second set of angry rejections for a paper (the one that I blogged about here ). Just as they did at the first journal I submitted it to - and despite the fact that, the second time, I'd watered it down as much as I possibly could - the reviewers hated the fact that the initial motivation for the paper came not from the previous literature, but from an idea that had occurred to me one day while I was watching a testing session. I did start, the second time around, with a literature review that was a comprehensive as I could make it. I described how the majority of research into arousal (operationalised as levels of Autonomic Nervous System activity) has considered it in one of three different contexts. First, as a reflexive response - studying how exter...
Read more