Project Overview
The SAFeSEA project was an EPSRC-funded collaboration between The Open University and Oxford University, investigating how extractive summarisation techniques could be used to analyse students' essays and generate appropriate formative feedback.
Writing summaries has been a long-standing educational activity but using summaries of your own writing as a source of reflection remains a more open issue. Can existing methods of information extraction and summarization be adapted to select content for feedback? How effectively can these methods deliver feedback? What effect do these techniques have on essay improvement, both within current and future drafts, and on self-regulation and metacognition?
My principal responsibility on the project was to design and evaluate the prototype of a web-based application (called OpenEssayist), as an experimental platform for addressing these questions. The context of application of our research agenda was to support OU students in writing their assignments essays for real postgraduate modules . Given the parameters of the targeted deployment (1500+ words open-ended essays, online & distant education, uneven conditions for writing), the approach I took was to focus on a more open, discovery-based environment, where various external representations of the summarisation elements would be offered to students, for them to explore on their own initiative .
The main design duties consisted on integrating the summarisation tools developed by the computational linguists in Oxford , developing the infrastructure of the web application, designing the various external representations deployed in the system, building the overall navigation of the system and developing the monitoring and data mining tools. The design was supported by several sessions of formative evaluations (focus groups in the early stage, desirability and usability testing, accessibility testing).
The OpenEssayist system has been deployed for two successive evaluations in real-life conditions: a first group of 32 students at the OU, followed by 3 groups running simultaneously at the OU (48 students), Hertfordshire University (12 students) and the British University in Dubai (20 students).
Selected Publications
- Field, D., Pulman, S., Van Labeke, N., Whitelock, D., and Richardson, J. T. E. (2013). Did I really mean that? Applying automatic summarisation techniques to formative feedback. In Proceedings of the 9th International Conference on Recent Advances in Natural Language Processing (RANLP'13 - Hissar, Bulgaria). pp. 277-284. [PDF]
- Van Labeke, N., Whitelock, D., Field, D., Pulman, S., and Richardson, J. T. E. (2013). What is my essay really saying? Using extractive summarization to motivate reflection and redrafting. In Proceedings of the AIED Workshop on Formative Feedback in Interactive Learning Environments (FFILE @ AIED'13 - Memphis, TN). pp. 1-8. [PDF]
- Field, D., Richardson, J. T. E., Pulman, S., Van Labeke, N., and Whitelock, D. (2013). Reflections on characteristics of university students’ essays through experimentation with domain-independent natural language processing techniques. In Proceedings of the 2013 International Computer Assisted Assessment Conference (CAA'13 - Southampton, UK). pp. 1-8. [PDF]
- Alden, B., Van Labeke, N., Field, D., Pulman, S., Richardson, J. T. E., and Whitelock, D. (2013). Using student experience to inform the design of an automated feedback system for essay answers. In Proceedings of the 2013 International Computer Assisted Assessment Conference (CAA'13 - Southampton, UK). pp. 1-10. [PDF]
- Van Labeke, N., Whitelock, D., Field, D., Pulman, S., and Richardson, J. T. E. (2013). OpenEssayist: extractive summarisation and formative assessment of free-text essays. In Proceedings of the 1st International Workshop on Discourse-Centric Learning Analytics (DCLA @ LAK'13 - Leuven, Belgium). pp. 1-5. [PDF]