Implementation of real-time online mouse tracking on overseas quiz ...

Mouse Tracking Data and Sample Analysis

Fig. 14. Sample visualization of mouse tracking data from Fig. 13 where all the coordinates from 44 students are plotted. The left image represents mouse click visualization with triangles for left clicks, squares for middle clicks, and pentagons for right clicks. The middle image represents mouse movement visualizations where increasing overlap of the coordinates is represented by a darker color. The right image represents a heatmap visualization where yellow indicates 5 seconds and red indicates over 10 seconds.

Fig. 13. Screenshot of mouse tracking data of students from National University of Mongolia who attempted a quiz session on a Moodle server at Kumamoto University.

The authors were able to obtain mouse tracking data from 44 students in the School of Engineering and Applied Science, National University of Mongolia, for an online quiz session on the server in the Human Interface Cyber Communication Laboratory, Kumamoto University. Fig. 13 shows a screenshot of the mouse tracking data in form of a table. The table size is approximately 145 MB, containing 393585 rows and 22 columns. Are the rumors that mouse tracking produces a notoriously large amount of data true? The answer to this question is “yes.” A half year Moodle log data with a similar number of students was only approximately 300 kB, while the mouse tracking data represented in Fig. 13 was 145 MB after 3 hours and 30 minutes.

The mouse tracking data contains so much information that a separate report is required to discuss the characteristics of the data and the types of possible analysis. There are many different types of analysis and discussions on mouse tracking data and several examples were reviewed earlier. In this report, sample visualization based on heatmaps and the trails of the mouse tracking data is presented, as shown in Fig. 14. As expected, left clicks occurred more often for selection of the questions. However, there were also left clicks associated with some of the questions and the visualization shows that the left clicks were dragged. This can be interpreted as highlighting the questions by the students. Middle clicks occurred most frequently for question four; however, the reason for this occurrence is not clear. Right clicks were most common on the top of the page, where some students probably decided to explore the available features. As expected, there were numerous trails such that it seemed that there were too much to visualize all at once. The heatmap indicates that most of the students placed the mouse cursor on the questions and choices. There were also few students who placed the mouse cursor outside the questions. Probably, these were individuals who preferred to keep the mouse cursor away from the text while reading. Further analyses are outside the scope of this work.

Resource Costs of the Mouse Tracking Process

Fig. 15. CPU percentage time series during mouse tracking implementation between National University of Mongolia and Kumamoto University. The horizontal axis represent the time of day and vertical axis is the CPU usage percentage.

Fig. 16. RAM usage time series during mouse tracking implementation between National University of Mongolia and Kumamoto University. The horizontal axis represents the time of day and vertical axis is the RAM usage in megabytes.

Fig. 17. Data rate during mouse tracking implementation between National University of Mongolia and Kumamoto University. The horizontal axis is the time of day and the vertical axis represents the data rate in bytes per second.

Similar to the local mouse tracking experiment with five users, the resource costs were also measured, allowing us to determine whether large-scale implementation is possible or not. Although 44 students attempted the quiz, the session was divided into two sessions and each session contained only 22 students. The students were informed that the first session would start at 12:00, followed by a break at approximately 14:00. The second session started a few minutes later and finishes at 15:30. As such, the entire process took 3 hours and 30 minutes (12600 seconds). The three Figures Fig. 15, Fig. 16, and Fig. 17 seems relevant to the informed schedule where a decrease in the graph was observed at 14:00 for a few minutes. The number of events generated during this time (12600 s) was 393585. The average number of events per second was 31 (393585 divided by 12600) or 31 Hz. When 31 Hz is plotted in Fig. 9, the result of 115 kB is obtained, which is close to the measured average data rate of 105 kB in Table 2.

Is large-scale mouse tracking implementation possible? This is possible if resource usage is balanced and distributed. Implementing real-time mouse tracking is a better choice than implementing non-real-time tracking. For example, if mouse tracking is first accumulated and subsequently submitted all at once, this will cause a bottleneck at the server. Fig. 15, Fig. 16, and Fig. 17 would not show constant usage but would show idle activities at the beginning, which would become constantly high in the middle. This is arguably an inefficient use of resource. Real-time implementation helps to evenly distribute the transmission of the mouse tracking data.

Compared to the local mouse tracking experiment with the five users, the resource costs are expected to increase because more users (22) were involved in this implementation, but there were more unexpected findings. The unexpected aspect is that the standard deviation is very high. As such, not only are there many positive spikes, but there are also many negative spikes, which further indicates that the number of events per second generated by the users is very dynamic. It should be noted that there was no limitation on the number of events per second that the students were allowed to generate. Based on Table 1 and Fig. 9, the data rate should increase in excess of 5 MBps for the worst-case scenario where 22 students simultaneously generate 70 events per second. However, this scenario never occurred as shown in Fig. 17, indicating high dynamics, and the very low probability of the worst-case scenario.

In Table 2, not only does the standard deviation increase indicating high dynamics, the distance between the median and maximum also increases, as represented by the taller spikes. The highest spike occurred at 14:28:40 when 228 events were submitted to the server and surprisingly, this was attributed to only two users. This occurrence either contradicts the assumption of the authors that a user can generate up to 70 events per second or there was a delay in transmission, and the submitted events were incorrectly aggregated. When 228 events per second are plotted in Fig. 9, the result 849 kBps is obtained, which is close to the actual measurement in Table 2, where the maximum data rate during this implementation was 837 kBps.

Conclusion and Future Work

The first conclusion is that the online mouse tracking application was successfully implemented. The overseas quiz that was session monitored with real-time mouse tracking at the National University of Mongolia to Kumamoto University was successfully conducted and at present, mouse tracking is still running on the server. The mouse tracking data containing mouse clicks, mouse movements, and mouse scrolls was obtained, but the analysis of this data will be challenging because of the large size. Additionally, this demonstrates the possibility of tracking on a mobile device using scroll, touch, and zoom events.

Are the rumors concerning high resource cost in mouse tracking true? Can a single swipe generate hundreds of mouse coordinates? Does mouse tracking over a minute generate in excess of a megabyte of data? Based on the result of this investigation, the answer to these questions is “yes.” In that case, is mouse tracking implementable on a large scale? The answer is also “yes.” One server with its specification highlighted in the Experiment and Implementation section was able to handle a classroom of users, and resource monitoring showed that there was much reserve capacity. Other institutions or corporations should have no difficulty in implementing mouse tracking because they typically have big data centers (large number of networks and servers, and distributed resources). For example, a corporation such as Google should not encounter difficulties, although this might be different for technologically challenged entities.

The second conclusion is that mouse tracking is implementable if resource usage is distributed. In this work, the mouse tracking data were transferred in real-time to evenly distributed resource usage, instead of aggregating the data and transmitting them together, which may cause bottlenecks. Unfortunately, the nature of mouse tracking is such that it is difficult to predict. As such, it is challenging to determine resource allocation. The data acquired as part of this work showed the high dynamic characteristic of mouse activities, as reflected in the high standard deviations observed during monitoring of resource usage. When 22 students attempted the quizzes, the resource usage peaks were very high, but only temporarily. This was identified as spikes. Both upper spikes and lower spikes were observed, where upper spikes indicate momentary high-level activity and lower spikes indicate the opposite.

If the amount of available resource is limited, then the resource cost of mouse tracking can be reduced. The mouse tracking application developed in this work can limit the number of events per second or frequency. Additionally, it can exclude unnecessary data. Moreover, even prior to this mouse tracking resource usage investigation, research on the compression of mouse tracking data already existed.

This opens many paths for future work. Although real-time implementation assisted in the distribution of resource usage, the characteristics of the resource usage data showed how mouse tracking can potentially destabilize the system. The use of load balancing techniques can help stabilize the implementation. To achieve the minimum system requirement for mouse tracking, more experiments with different machine specification needs to be conducted. In addition, resource measurement on the client-side needs should be conducted to achieve the minimum system requirement for the client. Even though the developed mouse tracking application was able to limit activity level recording, the settings are still manually inputted. Adaptive settings are required for optimal usage. Although it was useful to conduct overseas implementation, more users and longer implementations are required to further evaluate the viability of real-time online mouse tracking.

Acknowledgements

The authors are very grateful to Muhammad Bagus Andra, Hamidullah Sokout, Irwansyah, and members of the School of Engineering and Applied Science, National University of Mongolia, for participating in the experiment. The authors would also like to thank Masayoshi Aritsugi, Hendarmawan, Hamidullah Sokout, Alhafiz Akbar Maulana, and Sari Dewi for inspiring this research topic. A special thanks to Muhammad Bagus Andra and Ni Nyoman Sri Indrawati for suggesting some interesting ideas. The authors would also like to thank Fahd Ouassarni for providing suggestions with respect to compressing the mouse tracking application codes. Finally, the authors would like to thank Alvin Fungai for initiating this research and for his assistance in proofreading.

Funding

Part of this work was supported by JSPS KAKENHI Grant-in-Aid for Scientific Research 19H1225100 and 15H02795.

Conflict of interests

The authors declare that they have no conflict of interest.

Availability of Data and Materials

The datasets generated and/or analyzed during the current study are available in the Mendeley repository titled ’Data for: Implementation of Real-Time Online Mouse Tracking Case Study in a Small Online Quiz’.

Reference

(2016) Logs. URL https://docs.moodle.org/36/en/Logs
(2019) jquery write less, do more. URL https://jquery.com/
(2019) The top 10 best web analytic tools. URL https://www.sparringmind.com/best-web-analytics/
Alhasan K, Chen L, Chen F (2018) An experimental study of learning behaviour in an elearning environment. In: 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), IEEE, pp 1398–1403, DOI 10.1109/HPCC/SmartCity/DSS.2018.00231
Arapakis I, Leiva LA (2016) Predicting user engagement with direct displays using mouse cursor information. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, ACM, pp 599–608, DOI 10.1145/2911451.2911505
Arroyo E, Selker T, Wei W (2006) Usability tool for analysis of web designs using mouse tracks. In: CHI’06 extended abstracts on Human factors in computing systems, ACM, pp 484–489, DOI 10.1145/1125451.1125557
Authors (2019a) 0fajarpurnama0/real-time-online-mouse-tracking-animation Authors (2019b) Data for: Implementation of real -time online mouse tracking case study in a small online quiz. DOI 10.17632/vznyfcx9xk.1, mendeley Data, v1
Barrios VMG, Gütl C, Preis AM, Andrews K, Pivec M, Mödritscher F, Trummer C (2004) Adele: A framework for adaptive e-learning through eye tracking. Proceedings of IKNOW pp 609–616
Bluehost (2016) Web analytics for beginners - presented by bluehost. URL
Buscher G, Cutrell E, Morris MR (2009) What do you see when you’re surfing?: using eye tracking to predict salient regions of web pages. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, pp 21–30, DOI 10.1145/1518701.1518705
Busjahn T, Schulte C, Sharif B, Begel A, Hansen M, Bednarik R, Orlov P, Ihantola P, Shchekotova G, Antropova M, et al. (2014) Eye tracking in computing education. In: Proceedings of the tenth annual conference on International computing education research, ACM, pp 3–10, DOI 10.1145/2632320.2632344
Calvi C, Porta M, Sacchi D (2008) e5learning, an e-learning environment based on eye tracking. In: Advanced Learning Technologies, 2008. ICALT’08. Eighth IEEE International Conference on, IEEE, pp 376–380, DOI 10.1109/ICALT.2008.35
Canali D, Bilge L, Balzarotti D (2014) On the effectiveness of risk prediction based on users browsing behavior. In: Proceedings of the 9th ACM symposium on Information, computer and communications security, ACM, pp 171–182, DOI 10.1145/2590296.2590347
Cantoni V, Perez CJ, Porta M, Ricotti S (2012) Exploiting eye tracking in advanced e-learning systems. In: Proceedings of the 13th international conference on computer systems and technologies, ACM, pp 376–383, DOI 10.1145/2383276.2383331
Chen MC, Anderson JR, Sohn MH (2001) What can a mouse cursor tell us more?: correlation of eye/mouse movements on web browsing. In: CHI’01 extended abstracts on Human factors in computing systems, ACM, pp 281–282, DOI 10.1145/634067.634234
Chivu R, Turlacu L, Stoica I, Radu A (2018) Identifying the effectiveness of e-learning platforms among students using eye-tracking technology. In: 4th International Conference on Higher Education Advances (HEAD’18), Editorial Universitat Politècnica de València, pp 621–628, DOI 10.4995/HEAD18.2018.8046
Chourishi D, Buttan CK, Chaurasia A, Soni A (2011) Effective e-learning through moodle. International Journal of Advance Technology & Engineering Research (IJATER) 1(1):34–38
Cooke L (2006) Is the mouse a” poor man’s eye tracker”? In: Annual Conference-Society for Technical Communication, vol 53, p 252
Demšar U, Çöltekin A (2017) Quantifying gaze and mouse interactions on spatial visual interfaces with a new movement analytics methodology. PloS one 12(8):e0181818, DOI 10.1371/journal.pone.0181818
Dentzel Z (2013) How the internet has changed everyday life. BBVA OpenMind:” Ch@ nge
Dragunova M, Moro R, Bielikova M (2017) Measuring visual search ability on the web. In: Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, ACM, pp 97–100, DOI 10.1145/3030024.3038272
Drake JR, O’Hara M, Seeman E (2015) Five principles for mooc design: With a case study. Journal of Information Technology Education: Innovations in Practice 14(14):125–143
Duggan GB, Payne SJ (2011) Skim reading by satisficing: evidence from eye tracking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp 1141–1150, DOI 10.1145/1978942.1979114
Eger L (2018) How people acquire knowledge from a web page: An eye tracking study. Knowledge Management & E-Learning 10(3):350
Ehmke C, Wilson S (2007) Identifying web usability problems from eye-tracking data. In: Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1, British Computer Society, pp 119–128
Fungai A, Usagawa T (2016) Isolating hidden recurring patterns on unlabeled access log data in learning management systems to identify drop out risk students. In: 11th International Student Conference on Advanced Science and Technology, Kumamoto University, pp 9–11
Guo Q, Agichtein E (2008) Exploring mouse movements for inferring query intent. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, ACM, pp 707–708, DOI 10.1145/1390334.1390462
Guo Q, Agichtein E (2010) Towards predicting web searcher gaze position from mouse movements. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, ACM, pp 3601–3606, DOI 10.1145/1753846.1754025
Harrati N, Bouchrika I, Tari A, Ladjailia A (2016) Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior 61:463–471, DOI 10.1016/j.chb.2016.03.051
Henrie CR, Bodily R, Manwaring KC, Graham CR (2015) Exploring intensive longitudinal measures of student engagement in blended learning. The International Review of Research in Open and Distributed Learning 16(3), DOI 10.19173/irrodl.v16i3.2015
Holmqvist K, Wartenberg C (2005) The role of local design factors for newspaper reading behaviour-an eye-tracking perspective. Lund University Cognitive Studies 127:1–21
Holsanova J, Rahm H, Holmqvist K (2006) Entry points and reading paths on newspaper spreads: comparing a semiotic analysis with eye-tracking measurements. Visual communication 5(1):65–93, DOI 10.1177/1470357206061005
Hsu TC, Chang SC, Liu NC (2018) Peer assessment of webpage design: Behavioral sequential analysis based on eye tracking evidence. Journal of Educational Technology & Society 21(2):305–321
Hu J, Zeng HJ, Li H, Niu C, Chen Z (2007) Demographic prediction based on user’s browsing behavior. In: Proceedings of the 16th international conference on World Wide Web, ACM, pp 151–160, DOI 10.1145/1242572.1242594
Huang J, White RW, Dumais S (2011) No clicks, no problem: using cursor movements to understand and improve search. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, pp 1225–1234, DOI 10.1145/1978942.1979125
Huang J, White R, Buscher G (2012) User see, user point: gaze and cursor alignment in web search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp 1341–1350, DOI 10.1145/2207676.2208591
Hyönä J, Lorch Jr RF, Kaakinen JK (2002) Individual differences in reading to summarize expository text: Evidence from eye fixation patterns. Journal of Educational Psychology 94(1):44, DOI 10.1037/00220663.94.1.44
Ivanović M, Klašnja-Milićević A, Ivković J, Porta M (2017) Integration of eye tracking technologies and methods in an e-learning system. In: Proceedings of the 8th Balkan Conference in Informatics, ACM, p 29, DOI 10.1145/3136273.3136278
Jarodzka H, Brand-Gruwel S (2017) Tracking the reading eye: towards a model of real-world reading. Journal of Computer Assisted Learning 33(3):193–201, DOI 10.1111/jcal.12189
Jarodzka H, Holmqvist K, Gruber H (2017) Eye tracking in educational science: Theoretical frameworks and research agendas. Journal of Eye Movement Research 10(1), DOI 10.16910/jemr.10.1.3
Johnson A, Mulder B, Sijbinga A, Hulsebos L (2012) Action as a window to perception: measuring attention with mouse movements. Applied Cognitive Psychology 26(5):802–809, DOI 10.1002/acp.2862
Kakasevski G, Mihajlov M, Arsenovski S, Chungurski S (2008) Evaluating usability in learning management system moodle. In: ITI, vol 2008, p 30th
Khan S, Singh Y, Sharma K (2018) Role of web usage mining technique for website structure redesign. International Journal of Scientific Research in Computer Science, Engineering and Information Technology 3(1)
Kim NW, Bylinskii Z, Borkin MA, Gajos KZ, Oliva A, Durand F, Pfister H (2017) Bubbleview: an interface for crowdsourcing image importance maps and tracking visual attention. ACM Transactions on Computer-Human Interaction (TOCHI) 24(5):36, DOI 10.1145/3131275
Koh KH, Fouh E, Farghally MF, Shahin H, Shaffer CA (2018) Experience: Learner analytics data quality for an etextbook system. Journal of Data and Information Quality (JDIQ) 9(2):10, DOI 10.1145/3148240
Kumar R, Tomkins A (2010) A characterization of online browsing behavior. In: Proceedings of the 19th international conference on World wide web, ACM, pp 561–570, DOI 10.1109/ITI.2008.4588480
Lagun D, Agichtein E (2011) Viewser: Enabling large-scale remote user studies of web search examination and interaction. In: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, ACM, pp 365–374, DOI 10.1145/2009916.2009967
Lagun D, Ageev M, Guo Q, Agichtein E (2014) Discovering common motifs in cursor movement data for improving web search. In: Proceedings of the 7th ACM international conference on Web search and data mining, ACM, pp 183–192, DOI 10.1145/2556195.2556265
Lai ML, Tsai MJ, Yang FY, Hsu CY, Liu TC, Lee SWY, Lee MH, Chiou GL, Liang JC, Tsai CC (2013) A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational research review 10:90–115, DOI 10.1016/j.edurev.2013.10.001
Lee RS, Liu JN, Yeung KS, Sin AH, Shum DT (2009) Agent-based web content engagement time (wcet) analyzer on e-publication system. In: Intelligent Systems Design and Applications, 2009. ISDA’09. Ninth International Conference on, IEEE, pp 67–72, DOI 10.1109/ISDA.2009.189
Leiva LA, Huang J (2015) Building a better mousetrap: Compressing mouse cursor activity for web analytics. Information Processing & Management 51(2):114–129, DOI 10.1016/j.ipm.2014.10.005
Li LY, Tsai CC (2017) Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education 114:286–297, DOI 10.1016/j.compedu.2017.07.007
Liebling DJ, Dumais ST (2014) Gaze and mouse coordination in everyday work. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, ACM, pp 1141–1150, DOI 10.1145/2638728.2641692
Linawati L, Wirastuti ND, Sukadarmika G (2017) Survey on lms moodle for adaptive online learning design. Journal of Electrical, Electronics and Informatics 1(1):11–16, DOI 10.24843/JEEI.2017.v01.i01.p03
Liu Z (2005) Reading behavior in the digital environment: Changes in reading behavior over the past ten years. Journal of documentation 61(6):700–712, DOI 10.1108/00220410510632040
Lupu RG, Ungureanu F (2013) A survey of eye tracking methods and applications. Buletinul Institutului Politehnic din Iasi, Automatic Control and Computer Science Section 3:72–86
Manson SM, Kne L, Dyke KR, Shannon J, Eria S (2012) Using eye-tracking and mouse metrics to test usability of web mapping navigation. Cartography and Geographic Information Science 39(1):48–60, DOI 10.1559/1523040639148
Martín-Albo D, Leiva LA, Huang J, Plamondon R (2016) Strokes of insight: User intent detection and kinematic compression of mouse cursor trails. Information Processing & Management 52(6):989–1003, DOI 10.1016/j.ipm.2016.04.005
Maruya K, Watanabe J, Takahashi H, Hashiba S (2015) A learning system utilizing learners’ active tracing behaviors. In: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, ACM, pp 418–419, DOI https://doi.org/10.1145/2723576.2723655
Mueller F, Lockerd A (2001) Cheese: tracking mouse movement activity on websites, a tool for user modeling. In: CHI’01 extended abstracts on Human factors in computing systems, ACM, pp 279–280, DOI 10.1145/634067.634233
Nakano H, Iriguchi N, Sugitani K, Kita T, Musashi Y, Migita M, Matsuba R, Ohta Y, Gobayashi T, Tsuji K, et al. (2005) The instructional effects of on-line tests on the large-scale it courses. In: Information Technology Based Higher Education and Training, 2005. ITHET 2005. 6th International Conference on, IEEE, pp F4B–7, DOI 10.1109/ITHET.2005.1560304
Nandi D, Hamilton M, Harland J, Warburton G (2011) How active are students in online discussion forums? In: Proceedings of the Thirteenth Australasian Computing Education Conference-Volume 114, Australian Computer Society, Inc., pp 125–134
Navalpakkam V, Churchill E (2012) Mouse tracking: measuring and predicting users’ experience of web-based content. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp 2963–2972, DOI 10.1145/2207676.2208705 NT B (2015) Top 10 heatmap analytics tools for marketers. URL https://bigdata-madesimple.com/top-10-heatmap-analytics-tools-for-marketers/
Parikh S, Kalva H (2018) Eye gaze feature classification for predicting levels of learning. In: Proceedings of the 8th Workshop on Personalization Approaches in Learning Environments (PALE 2018).
Kravcik, M., Santos, OC, Boticario, JG, Bielikova, M., Horvath, T. and Torre I.(Eds.). 19th International Conference on Artificial Intelligence in Education (AIED 2018), CEUR workshop proceedings, this volume, pp 1–6
Paturusi SD, Chisaki Y, Usagawa T (2012) Development and evaluation of the blended learningcourses at sam ratulangi university in indonesia. International Journal of e-Education, e-Business, e-Management and e-Learning 2(3):242
Pernice K (2017) F-shaped pattern of reading on the web: Misunderstood, but still relevant (even on mobile). Nielsen Norman Group Pivec M, Trummer C, Pripfl J (2006) Eye-tracking adaptable e-learning and content authoring support. Informatica 30(1)
Poon LK, Kong SC, Yau TS, Wong M, Ling MH (2017) Learning analytics for monitoring students participation online: Visualizing navigational patterns on learning management system. In: International Conference on Blended Learning, Springer, pp 166–176, DOI 10.1007/9783319593609_15
Purnama F, Fungai A, Do TM, Siagian AHAM, Annas A, Susanto H, Hendarmawan, Usagawa T, Nakano H (2016a) Introductory work on section based page view of web contents: Towards the idea of how a page is viewed. In: 11th International Student Conference on Advanced Science and Technology (ICAST), Kumamoto University, pp 9–11
Purnama F, Fungai A, Usagawa T (2016b) Demonstration on extending the pageview feature to page section based: Towards identifying reading patterns of users. In: 7th International Conference on Science and Engineering, Yangon Technological University, pp 304–307
Rakoczi G, Pohl M (2012) Visualisation and analysis of multiuser gaze data: Eye tracking usability studies in the special context of e-learning. In: Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference on, IEEE, pp 738–739, DOI 10.1109/ICALT.2012.15
Ramakrisnan P, Jaafar A, Razak FHA, Ramba DA (2012) Evaluation of user interface design for leaning management system (lms): Investigating student’s eye tracking pattern and experiences. Procedia-Social and Behavioral Sciences 67:527–537, DOI 10.1016/j.sbspro.2012.11.357
Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychological bulletin 124(3):372
Rayner K (2009) Eye movements and attention in reading, scene perception, and visual search. The quarterly journal of experimental psychology 62(8):1457–1506, DOI 10.1080/17470210902816461
Rheem H, Verma V, Becker DV (2018) Use of mouse-tracking method to measure cognitive load. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications Sage CA: Los Angeles, CA, vol 62, pp 1982–1986, DOI 10.1177/1541931218621449
Rodden K, Fu X (2007) Exploring how mouse movements relate to eye movements on web search results pages. In: SIGIR Workshop on Web Information Seeking and Interaction, pp 29–32
Rodden K, Fu X, Aula A, Spiro I (2008) Eye-mouse coordination patterns on web search results pages. In: CHI’08 extended abstracts on Human factors in computing systems, ACM, pp 2997–3002
Rodrigues M, Gonçalves S, Carneiro D, Novais P, Fdez-Riverola F (2013) Keystrokes and clicks: Measuring stress on e-learning students. In: Management Intelligent Systems, Springer, pp 119–126, DOI 10.1007/978-3-319-00569-0_15
Salmeron-Majadas S, Santos OC, Boticario JG (2014) An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context. Procedia Computer Science 35:691–700, DOI 10.1016/j.procs.2014.08.151
Sin K, Muthu L (2015) Application of big data in education data mining and learning analytics–a literature review. ICTACT Journal on soft computing 5(4)
Sopu HT, Chisaki Y, Usagawa T (2016) Use of facebook by secondary school students at nuku’alofa as an indicator of e-readiness for e-learning in the kingdom of tonga. The International Review of Research in Open and Distributed Learning 17(4), DOI 10.19173/irrodl.v17i4.2333
Sungkur RK, Antoaroo MA, Beeharry A (2016) Eye tracking system for enhanced learning experiences. Education and Information Technologies 21(6):1785–1806, DOI 10.1007/s10639-015-9418-0 Tarasewich P, Pomplun M, Fillion S, Broberg D (2005) The enhanced restricted focus viewer. International Journal of Human-Computer Interaction 19(1):35–54, DOI 10.1207/s15327590ijhc1901_4
Tsai MJ, Hou HT, Lai ML, Liu WY, Yang FY (2012) Visual attention for solving multiple-choice science problem: An eye-tracking analysis. Computers & Education 58(1):375–385
Tzafilkou K, Protogeros N (2017) Diagnosing user perception and acceptance using eye tracking in web-based end-user development. Computers in Human Behavior 72:23–37, DOI 10.1016/j.chb.2017.02.035
Usagawa T, Sugitani K, Kita T, Iriguchi N, Migita M, Matsuba R, Musashi Y, Nakano H (2006) Assuring the basic it literacy levels for every student by the university-wide blended learning. In: Information Technology Based Higher Education and Training, 2006. ITHET’06. 7th International Conference on, IEEE, pp 647–651, DOI 10.1109/ITHET.2006.339680
Voßkühler A, Nordmeier V, Kuchinke L, Jacobs AM (2008) Ogama (open gaze and mouse analyzer): open-source software designed to analyze eye and mouse movements in slideshow study designs. Behavior research methods 40(4):1150–1162, DOI 10.3758/BRM.40.4.1150
Wang H, Chignell M, Ishizuka M (2006) Empathic tutoring software agents using real-time eye tracking. In: Proceedings of the 2006 symposium on Eye tracking research & applications, ACM, pp 73–78, DOI 10.1145/1117309.1117346 Wei H, Moldovan AN, Muntean C (2009) Sensing learner interest through eye tracking. Technology 22:23rd
Wen M, Rosé CP (2014) Identifying latent study habits by mining learner behavior patterns in massive open online courses. In: Proceedings of the 23rd ACM international conference on conference on information and knowledge management, ACM, pp 1983–1986, DOI 10.1145/2661829.2662033
Yang FY, Tsai MJ, Chiou GL, Lee SWY, Chang CC, Chen LL (2018) Instructional suggestions supporting science learning in digital environments based on a review of eye tracking studies. Journal of Educational Technology & Society 21(2):28–45
Zheng C, Usagawa T (2018) A rapid webcam-based eye tracking method for human computer interaction. In: 2018 International Conference on Control, Automation and Information Sciences (ICCAIS), IEEE, pp 133–136, DOI 10.1109/ICCAIS.2018.8570532
Zushi M, Miyazaki Y, Norizuki K (2012) Web application for recording learners’ mouse trajectories and retrieving their study logs for data analysis. Knowledge Management & E-Learning 4(1):37, DOI 10.34105/j.kmel.2012.04.004

Implementation of real-time online mouse tracking on overseas quiz session From server administrator point of view (ending pre-print)

Overseas Implementation between Mongolia and Japan