Mouse tracking technology: study internal information processing

Hello, here is the next line up , I meow Jun sister ~

In the field of brain science, what we study most is the internal information processing process of individuals. In the research process, we used many methods, including behavioral data analysis, such as response time, accuracy, and EEG data, such as ERP.

In this issue, we will introduce a new technology for studying individual information processing- Mouse Tracker (MouseTracker) .

1 Principles and technical introduction

1.1 Application Principle of Mouse Tracking Technology

The physiological principle of mouse tracking technology is: the movement of the hand can reflect the information processing process in the brain in real time.

Because the relationship between perceptual input and motor response is dynamic and continuous, the information obtained by face processing will be transmitted to the motor cortex in real time and continuously in the form of pulse waves.

Therefore, it can track and record the running trajectory of the mouse when the subject moves the computer mouse to react. Through the analysis of the coordinates of the mouse movement trajectory, the dynamic process of psychological processing can be measured in real time. This technique is especially suitable for measuring mouse movement trajectories when performing multiple selection judgment tasks ^[1] .

For example , a study explored the learning of geometrical examples for pupils by finger tracking.

In Experiment 1, students were required to track the index finger during the example learning process, without index finger tracking and labeling the three learning conditions of the sample, comparing the students' far and near migration test scores and their scores on the difficulty of the migration test.

The experimental results showed that the far and near migration scores of the tracking group were significantly higher than those of the marked group and the non-tracking group, and the difficulty of the far migration problem was lower in the tracking group.

In Experiment 2, the experimental conditions were changed, that is, students were required to track the samples with their index fingers, use the mouse to track the samples, and watch the animations of others tracking the samples with the index fingers.

The experimental results show that the far migration scores of the subjects in the index finger tracking group and the mouse tracking group are significantly higher than the far migration scores of the subjects in the viewing tracking group, and the subjects of the index finger tracking group and the mouse tracking group score low on the difficulty of the long migration test. The ratings of the participants in the watch tracking group. There were no significant differences between the index finger tracking group and the mouse tracking group in the far and near migration scores and the difficulty of the migration test.

The above experiments show that finger tracking can improve the learning performance of elementary students in geometric samples, and this advantage can be extended to mouse tracking. It is explained on the side that mouse tracking is an effective and reliable measurement method ^[1] .

1.2 Analysis program and performance of the software package MouseTracker

1.2.1 Analysis procedures

MouseTracker is the latest operation software package of mouse tracking technology, which contains 3 operation procedures: experiment design procedure , data collection procedure and data analysis procedure .

It can be used for visual experiment design, such as adding label buttons and adjusting its position and size. In addition, it can also present pictures, character strings and sound stimuli.

1.2.2 Steps and parameters of trajectory analysis

First, data preprocessing is required , which includes three steps of re-measurement of distance , time normalization, and averaging .

Re-measure distance is to re-measure all mouse tracks in the standard MouseTracker coordinate system;

Time normalization refers to the use of linear interpolation to normalize the recorded mouse trajectory so that each trajectory contains the same number of coordinate points;

The average is to calculate the average number of all x coordinate values ​​and all y coordinate values ​​at each coordinate point under a certain condition, and obtain the average trajectory of each subject under a certain condition.

Secondly, the spatial attraction needs to be calculated .

Refers to a degree of suction space under certain experimental conditions the mouse trajectory alternative correct degree of deflection incorrect alternative tag label toward the front, suction indicators reflect the spatial degrees of maximum deviation (MD) and the area under the curve (AUC) .

MD refers to the maximum vertical deviation between the actual trajectory and the ideal trajectory (the line between the start and end of the trajectory);

AUC refers to the geometric area between the actual trajectory curve and the ideal trajectory curve. The larger the value, the more the trajectory is toward the unselected label.

In addition, the complexity, distribution, speed and acceleration of the trajectory can be analyzed as needed.

1.2.3 Analysis of sources of differences between tracks

One of the more questions about mouse tracking technology is whether the difference in mouse movement trajectory between different experimental conditions is due to independent variable operation or experimental error.

As shown in the figure below, when a study studies stereotypes, the results show that the average mouse trajectory in Condition 1 is more biased to the opposite of the correct label than the average mouse trajectory in Condition 2 (dotted line in the figure). One label (circled in Figure 1), the difference between MD and AUC under the two conditions is significant.

But the difference between the above condition 1 and condition 2 may also be caused by some discrete experimental error. For example, in condition 1, the mouse track of half of the trials points directly to the correct label (dotted line in the figure below), and in the other half of trials In the middle, before pointing to the correct label, the mouse points to another label opposite to its position (the triangle connection in the figure below) .The average result of the two is a curve that biases the incorrect alternative label (circle connection in the figure below) .

In order to distinguish whether the experimental results are derived from experimental operations or experimental errors, the current researcher mainly solves this problem by examining the bimodality of the distribution of AUC values ​​(referring to the Z score of AUC values) of all trials trajectories.

If the distribution of the AUC values ​​of all trials trajectories in condition 1 is bimodal, that is, some trajectories are not biased towards incorrect labels, while other trajectories show a very strong trend towards incorrect labels (as shown in the histogram in the figure above) Shown), you can make an inference: the experimental error caused the difference in the above experimental results.

2 Research examples

2.1 Real-time, continuous and dynamic integration of social categories across channels

Some studies have used mouse tracking technology to analyze cross-channel social categories.

In the experiment, visual information and auditory information are presented simultaneously. The visual information is photos of gender atypical faces, and the auditory information is typical male voices, typical female voices, atypical male voices, and atypical female voices. Participants' task is to classify gender according to face, and refer to voice information if necessary.

Mouse trajectory analysis shows that when face stimuli and gender atypical sound stimuli (compared to gender-specific sounds) are presented at the same time, before the participant clicks the correct gender label button, the mouse trajectory is clearly biased to the opposite gender label, the MD of the trajectory Both the value and AUC value are greater than the former MD value and AUC value, and the difference is very significant.

The experiment shows that auditory stimuli have a certain effect on visual information, thus proving that auditory information and visual information are gradually and dynamically integrated together.

2.2 Used in the field of polygraph

Before finally denying the facts, people may affirm the facts unconsciously, and then assess whether this content should be exposed as a reaction. Based on this understanding of deception, the mouse tracking technology of the dynamic monitoring process can reflect the real-time dynamics of the cognitive process, better reveal the psychological process of the subject's lying response, and has good reliability and validity.

In the experiment, the subjects conducted honest and deceptive responses based on the color patches, and after the experiment, the subjects were asked to fill out a 100-question confirmation sheet to check the experimental results.

The results show that in terms of reaction time, honest response time is significantly less than deception response time, and the maximum vertical deviation (MD) of deception response between the actual trajectory of the mouse movement and the ideal trajectory is significantly greater than that of honest response The vertical deviation value is later, and the area under the curve (AMC) is also larger. The mouse trajectory of spoofing response is also more uneven than that of honest response ^[3] .

3 Summary

This issue introduces a systematic introduction to the mouse tracking technology, including its principles and technical methods, and uses examples to help you understand and analyze. I hope it can help everyone ~

Article first appeared in the public micro-channel number under the line up , the original documents and tutorials Welcome to respond to keywords in the public No. # # mouse tracking download yo ~

Typography: Huahua

Picture from: pixabay

references

[1] Zhang Xiaobin, Zuo Bin, Xue Mengjie, Jiang Wen, Hou Feixiang. Real-time measurement of individual construction process based on mouse tracking technology [J]. Advances in Psychological Science, 2012, 20 (05): 770-781.
[2] Du Xuejiao. Finger tracing promotes the learning of geometric examples for elementary school students [C]. Chinese Psychological Association. Summary of the 21st National Psychological Academic Conference. Chinese Psychological Association: Chinese Psychological Association, 2018: 22-23.
[3] Zhang Lei, Wang Long, Liu Hongguang.Exploring the Feasibility of Mouse Tracking Technology Applied in the Field of Lie Detection [J] .Psychological Technology and Application, 2014 (03): 34-36.