Educational Technology Lab
Dept. Of Pedagogy
Philosophy, Pedagogy, Psychology (PPP), School of Philosophy
National & Kapodistrian University of Athens

 

Learning how to learn with microworlds: feedback evaluation and help seeking

Nikoleta Yiannoutsou, nyiannoutsou@gmail.com

Educational Technology Lab, University of Athens, Greece

Manolis Mavrikis, m.mavrikis@lkl.ac.uk

London Knowledge Lab, Institute of Education, University of London

Abstract

We report on on-going research about students’ learning how to learn with microworlds. We argue that a study of metacognitive skills should take into account microworld characteristics and especially the way learners interact with microworlds. Our analysis focuses on an important phase during learner-microworld interaction; evaluation upon which subsequent actions and meaning generation is built. Research results revealed six different evaluation types one of which –impasse acknowledgement-was further investigated in the context of help seeking.

Keywords

Feedback, Learning to learn, transformation tools, help seeking, evaluation,

Introduction

Papert used the term “mathetics” to refer to the process of learning how to learn in constructionist environments: I have defined mathetics as being to learning as heuristics is to problem solving. Principles of mathetics are ideas that illuminate and facilitate the process of learning (Papert, 1980, p. 120). Although Papert’s two mathetic principles – relate the new with something familiar and develop ownership over the new by constructing something with it (ibid) – have guided the design of constructionist environments, few studies in the area of constructionism have stressed this element in learning as most studies in the area focus on the learning of the subject matter (i.e. maths, science, programming etc). Metacognitive awareness in constructionist environments has been described to involve problem finding skills, cognitive flexibility, continual evaluation and monitoring the solution process, controlling destructions and anxiety, becoming aware and activating problem solving strategies (Harel & Papert, 1991a).

In this paper we report research on elements of ‘learning how to learn’ process with constructionist environments. We focus especially on the evaluation process which is related to the feedback generated by the microworld. We see feedback interpretation as a complex learning process that involves not only learning of the subject matter but also reflection, monitoring and evaluation of the learning process (Schraw, 2007). Our research revisits metacognitive awareness in constructionist environments from the point of view of evaluation because, as we will show in the next section, it is an important part of the learner interaction with the microworld. The second part of our analysis discusses how students deal with situations which are evaluated by them as impasses. Impasses are crucial from a learning and meta-learning perspective. From the learning perspective, impasses can trigger the construction of new knowledge or they can prevent essential interaction with the microworld. From the meta-learning perspective, impasses entail handling frustration, seeking new resources, reflecting on previous actions, evaluation and integration of the suggested solutions. In the following sections we describe how evaluation is related to microworld characteristics and we discuss help seeking as a context for handling impasses.

Microworlds as transformation tools: transforming user actions into microworld behaviour

In order to describe the characteristics of microworld feedback we will borrow from Verillon & Rabardel, (1995) the concept of transformation because it is related not only to tool use but also to learning. Specifically, with respect to tool use, transformation has a dual role a) individuals use the tools to transform the environment (ibid) and b) individuals make sense of the tools through the interpretation of the causal relationship between the user actions and the transformations of the environment. When it comes to learning, tool-use is associated with transformations of the task/object of the user/learner (i.e. instrumentation Verillon & Rabardel, (1995)) and of the relationship of the learner to the knowledge integrated/represented in the tool (Mariotti, 2002).

Microworlds come with a transformation mechanism which changes user actions into a representation familiar to the learner (1st mathetic principle: associate the new with something familiar Papert 1980) compatible to the concept negotiated and usually completely different from the actual action performed. To further illustrate this we will use the familiar example of drawing a house with a roof in Logo. To draw a house the student has to type Logo commands in the editor. This action is projected on the microworld (i.e. the typed commands appear in the editor) but this is not where the story ends because this action is transformed within the microworld into a behaviour completely different from the initial action. Command typing leads to a sketch of the house appearing on the screen. Thus, in microworlds learner actions are processed in two ways: one is what we called “projection of user action” where learner’s actions on the tool is rather analogous to what it appears on the tool (i.e. pressing the letters “fd” in the Logo editor results in having the letters “fd” appearing in the logo editor). The other is the transformation of action into microworld behaviour (i.e. any change in the state of the microworld that is caused by user actions.) according to “rules and laws” built in the specific microworld. Learning is intertwined with unlocking this transformation mechanism in order to find the causal–effect relationship between user actions and microworld behaviour. This idea draws upon the causality effect which is described as meaning making mechanism for tool use by Verillon & Rabardel, (1995). One way of depicting the process of learner interaction with the microworld is shown in fig. 1.

 

 

 

 

 

 

 

 

 


Figure 1. The loop of learner interaction with the microworld

According to the picture above, learner interaction with the microworld in traditional settings is structured around the following pattern:

§  The learner performs an action in the microworld (point 2 in fig 1) which is based on a specific goal that they have either directly (e.g. provided by a given task) or indirectly (e.g. set during the problem solving process (point 1 in fig 1). Examples of goals and user action could be the following: adding a command in order to construct a house, changing the values of a variable or dragging a slider in order to change the value of a parameter and investigate its effect, etc

§  This action is projected on the tool (point 3 in fig 1 e.g. the typed commands appear on the screen). In contrast to several tools (e.g. presentation tools or word or image processing) the loop of user interaction with the microworld does not end with the projection of the user action in the tool (e.g. the text typed). Evaluation, reflection and new task analysis is a prerequisite for subsequent interaction with the microworld.

§  The transformation mechanism changes the user action into microworld behaviour: e.g. drawing a house with a roof that doesn’t covers the house. The specific graphical output (or an explicit message from the microworld) is a behaviour generated by the microworld (point 4 in fig 1) as a response to the student’s action (typing the commands that create the house) and thus provides feedback either directly or indirectly.

§  In order for the students to know if their goal has been achieved they need to evaluate (point 5 in fig 1) the behaviour generated by the microworld (i.e. the house with the flawed roof) against the expected behaviour according to their goal (a house with a proper roof).

The results of this evaluation will either end the loop or will become the basis for the next user action (e.g. change the first turn of the turtle). Evaluation of microworld behaviour might also result to a new conceptualization of the task and lead to the formulation of new goals.

One important aspect of evaluation in this loop is that it is expected to be performed by the learner on a perceptual basis (Balacheff & Sutherland, 1994). But how can the learner evaluate something that he/she is learning and do not know already? In the case of microworlds the learner is expected to evaluate the impact of his/her actions on the microworld (i.e. the behaviour generated) based on the fact that the feedback is a representation of a concrete object or phenomenon which is familiar to the learner (i.e. the sketch of a house). Thus the learner evaluates the behaviour of the microworld against the expected results of his/her actions according to the task or to their goal. To further illustrate this let’s use the example of the house construction: the learner might not know how much the turtle should turn in order for the roof to fit the house but as soon as the commands are executed the learner can evaluate if the graphical output looks like a house or not.

Evaluation of the behaviour might be followed by an interpretation of why things happened this way (that is an interpretation of the causal relationship between user actions and microworld behaviour) and of a reflection/evaluation on the user action. Reflection on user actions is usually indirect because it is based on the impact these actions are having on the microworld (i.e. which action on the microworld caused this behaviour). Evaluation is necessary for the next action the way a chess player takes into account his/her opponent’s actions. It is in this sense that the evaluation of the microworld behaviour is a prerequisite for the next action on the microworld --- it shapes and directs the user interaction with the microworld. If such an evaluation will not take place then it is most likely that the learner will perform, at best, random actions on the microworld and may not complete the task. For this reason some microworlds scaffold evaluation providing explicit feedback for example through intelligent analysis of a certain task the student is undertaking drawing explicit attention to the lack of a goal achievement (see Mavrikis et al., to appear). Our analysis in the data section shows the focus of student evaluation of microworld behaviour and how students handle impasses during their interaction with the microworld. More often than not, evaluation might trigger the need for social interaction and particularly help-seeking on behalf of the learner or can be used as an opportunity for a justified intervention on behalf of a teacher. After summarising the characteristics of microworld feedback we elaborate on the social element of evaluation.

The characteristics of microworld feedback

We described above how the transformation mechanism in constructionist environments shapes the feedback generated by the microworld (microworld behaviour) as a response to user’s action. This feedback has the following characteristics:

§  It is another representation of the user action (consider for example the command “fd 5” typed in the Logo editor which is transformed in a representation of a line of specific length) which

o   is relevant to the concept under investigation

o   is usually a specific object or a phenomenon (Balacheff & Sutherland, 1994)

o   is familiar to the learner so that it allows perceptual control of the actions on the microworld (ibid). (The learner knows what a sketch of a house looks like – connecting the new with the familiar Papert 1980)

o   evolves along with the learning process (Balacheff & Sutherland, 1994) as this process is manifested through the learner actions

§  It is an integral part of the microworld design and it is generated according to a “domain of phenomenology (ibid)” which in essence determines the translation mechanism, that is how user actions will be transformed into microworld behaviour and what kind of behaviour would this be (“phenomena at the surface of the screen” ibid). Thus, the domain of phenomenology actually determines which representation of user actions is best for supporting the negotiation of meanings with the specific microworld (consider here the idea of “body syntonicity” Papert 1980 dominant in the turtle graphics which leads into the drawing of shapes as the trace of a moving turtle – the case of circle here is indicative)

§  It is not necessarily an evaluation of user action (as opposed to the feedback offered by drill and practice environments which explicitly evaluate response correctness) but could be implicit or explicit in drawing students’ attention to the lack of goal achievement

Help seeking as a social dimension of feedback interpretation

We believe that a particularly interesting consequence of evaluation of microworld behaviour is that it can act as a trigger for social interaction and particularly help-seeking from peers or the teacher which in turn can play a pivotal role in how the feedback is interpreted and shape both the meaning and importance of subsequent user actions. Research in the area highlights the distinction between executive and instrumental help-seeking (Nelson-LeGall, 1985). Executive help-seeking involves seeking answers to problems directly. This may lead to task completion but does not facilitate deeper understanding. Instrumental help-seeking involves requesting help for demonstrating or explaining the method by which the problem can be solved, allowing the student to retain responsibility for the solution and to acquire new knowledge. This way the help seeker not only can remedy their immediate problem, but also ensure long-term autonomy. The type of help students seek and provide is influenced by implicit approaches to learning in general and therefore engaging and reflecting in help-seeking and giving is an important element of self-regulated learning (Nelson-Le Gall, 1987; Karabenick, 1988).

Tools and tasks

This study takes place in a learning environment mediated by the Metafora System – a platform which integrates microworlds with computer-supported collaborative learning (CSCL) tools (i.e. a planning tool and a discussion tool). Integration involves communication between elements of the different tools such as accessing microworlds through the planning tool, exporting instances of microworlds to the discussion tool etc (for a more detailed description see Mavrikis et al 2012 and other publications in http://www.metafora-project.org). In this study students used 2 out of the four microworlds integrated in the Metafora system and the discussion tool which is called LASAD. In the first phase of the study students used the 3d juggler microworld which consists of 3d objects (three balls and four bases) placed in 3d space. Students can control the motion of the balls changing the values of the motion variables (speed, wind direction, altitude, azimuth etc). The task in which students were engaged with the 3d juggler was to manipulate the motion of the balls so as to hit each other’s base in a circular manner (i.e. the red ball should hit the blue ball’s base, the blue ball should hit the green’s base, and the green ball the red’s base).

The second phase of the study involved the use of eXpresser microworld and LASAD discussion tool. eXpresser is a microworld for building animated models using figural patterns of tiles to support students’ learning of mathematical generalisation. It also incorporates intelligent support; and tools that help teachers track students’ progress. The task with which students were engaged involved the construction of a train track model in eXpresser using different coloured tiles to distinguish the different patterns that constitute it in the way they visualized it (see an example in fig 2 where the pattern is embedded in a discussion). Their final goal is to derive a rule based on the structure of their constructed train-track model that gives the total number of tiles for any model. Therefore the choice of patterns and structures is left to the students. Students during this phase were asked to use LASAD to ask for help from other students or their teacher. LASAD is a discussion tool where students can contribute remotely to the same discussion space and can also integrate in the discussion models they created in eXpresser (we use the term referable objects which is described in detail in Mavrikis et al 2012). Discussions in LASAD have the form of concept maps aiming to visualize the contribution types in the discussion (see figs 2 and 3)

Method

In our study we employed design based research (The Design Based Research Collective 2003) because a) it is grounded on theory and research results b) aims at studying interventions as opposed to other qualitative methods (Collins et al 2004) and c) informs theory and the design of the intervention. Our study is divided in two phases, the first phase was grounded on theory about metacognitive awareness and microworld feedback aiming to explore how students evaluate microworld behavior. This study took place in one of the Public Junior High Schools in Athens (2nd Experimental Junior High School). Four 13 year old students participated in the study and worked in groups of two with the Metafora Platform and specifically the 3d –juggler microworld for three and a half hours (in one session). The second phase of our study was grounded on the theory about learning to learn together and especially on help seeking as well as on results of the first phase of the study. The second phase took place in a school in UK, four 11 year old students and their teacher participated in the study which lasted five school hours (in 4 sessions). Students used the Metafora System and especially the eXpresser microworld and LASAD discussion tool.

Data Analysis

In this section we analyze a set of data that aim at casting light in two research questions a) what are the characteristics of learning how to learn with microworlds and b) how students handle situations which are evaluated as impasses in the process of interacting with the microworld.

Evaluation types of microworld behavior.

The first part of our analysis focuses on the evaluation phase of the learner interaction loop with the microworld. We consider evaluation as a crucial element of learner interaction with microworlds for the following reasons a) it directs learner actions with the microworld b) it requires the use of skills that are central to learning how to learn (planning, reflection, monitoring (Schraw, 2007)) and c) student problems in proceeding in meaning generation might be grounded on the evaluation phase where they fail to interpret the microworld feedback)

Our data collected during the first phase of the study with the 3d juggler mwd, were analyzed with respect a) to the user controlled elements of the student microworld interaction i.e. evaluation, user action, and b) to the microworld feedback and the task (see fig 1). Our analysis revealed six different types of evaluation:

i. Personal: This type of evaluation is directed towards the group member who performed or suggested the action It appears in cases where the microworld behaviour is either quite close to the result or quite opposite to it. So, it might be fine tuning of the previous action or might lead in a complete different action as in trial and error. Involves only microworld behaviour.

22: What are you DOING????
48: Don’t be stupid
126: I am God!

ii. Boolean: Boolean evaluation focuses on the behaviour generated by the microworld and has the form of good -bad, right – wrong. Involves only microworld behaviour and occurs in contexts similar to the ones described in personal evaluation.

19 NO
50: Good! You see..
81: Oh! What is this??? Wait! Wait!
105: Nothing!
116 This is not what we want

iii. Descriptive - problem focused: When students use this type of evaluation they focus on describing the problem they encounter. The wording of the problem is formed upon the difference between the actual microworld behaviour and the result the students aim for. In this type of evaluation there is no explanation of why things happen this way

110: It goes up!
19: It hit the green base
50: Ahhh! It didn’t even touch that one!
100: You see? It moves towards this base, that’s the problem

iv. Task-goal oriented evaluation: In this type of evaluation students compare the microworld behavior against the task or the goal they have set (lines 61 and 62). Another variation might initiate the formulation of a new goal (change the turn instead of making the ball to hit the red base) for student actions or the analysis of the task into subtasks (Line 71):

63. S1: That was close!
64: S2: No, not really. This ball should go here (indicates that the red ball should hit the blue base) but this ball should stay here. Right?

71:S1:Wait! One step at a time. It has to turn more. Let’s do that for now.

v. Explanatory - Causal: This type of evaluation offers an explanation that connects student actions with the generated microworld behaviour. Thus, students do not just identify what the problem is as in descriptive - problem focused type but they also attempt to interpret feedback and explain why the problem occurs or why things happen this way.

110 When we move this one [he points at one of the sliders], it goes up. This one controls how high the ball will go.
115: This one is for the direction! Move that!

vi. Acknowledging an impasse: This evaluation involves again microworld behaviour but there is also a dimension related to the limitations of students’ actions (I can’t understand.... no matter what we do) and an evaluation on them: the strategies we tried doesn’t give us any idea of how to proceed. This acknowledgement might interrupt the loop of the interaction with the microworld and at this point students might call for the teacher or stop for a while and try to come up with a completely different idea compared to the ones tried before.

73: I can’t understand! No matter what we do, the ball moves straight
94: Same thing again! Straight line!

The data we presented in this section involved the evaluation of microworld behaviour. Our main observation was that only one of the six types of evaluation –the explanatory causal- involved a reflection on student actions which in turn were connected to the microworld behaviour. This evaluation type which in essence is a conjecture about the mechanism that transforms student actions into microworld behaviour seems to formulate the basis upon which students grounded the next action on the microworld. With Personal and Boolean evaluation types students express the results of the comparison between the actual microworld behaviour and the expected behaviour. These evaluation types are in a subtle way an evaluation of student actions in the sense that based on this evaluation students might repeat an action that seems close to what they expected or they might try out something completely different if the previous action led to an unexpected result (as in trial and error). These evaluations are different from the explanatory type in that they are less fine grained (yes/no type) and as such they do not offer an explanation on what is the problem and how things work or why happen this way. Descriptive, problem-focused evaluation is enriched with a description of what seems to be the problem which is expressed mainly in contrast to what the expected result was. With respect to learning how to learn, being able to identify the problem is the first step to resolve it. Task or goal-oriented evaluation compares microworld behaviour against the set task or goal or becomes the basis upon which a new goal is formulated. So it seems that this type of the evaluation can lead to a) revisiting – and even reconceptualising - the task and b) to breaking down the task into sub-tasks or goals both of which are important elements of learning how to learn. Finally, the last evaluation type is what we called acknowledgement of an impasse where microworld behaviour triggers an evaluation of student actions. From the point of view of learning how to learn this evaluation has to do with acknowledgement of the limitations of implemented strategies and with seeking new resources and ideas. The way students handle this situation varies: they might stop working for a while, they might get frustrated, they might check out what other students do (“floating of ideas” Harel & Papert, (1991b)) or they might call the teacher to help them out. Impasses are very crucial moments during interaction with microworlds because overcoming them might be grounded on construction of new knowledge and advancement of previous strategies. Next we describe an intervention aiming to record how students handle such impasses when they deal with them in the social practice of help seeking. 

Handling impasses in the context of help seeking

In this section we discuss data from the second phase of our research where students worked in separately in the task of constructing train tracks with eXpresser. At the beginning of the task we introduced the idea of help seeking to the students telling them that if they felt that they needed help with their task in expresser they should describe their problem in the shared space of LASAD for other students and the teachers to see and offer their suggestions and comments. LASAD was chosen over face to face communication for two reasons a) exchanges on a problem encountered and discussed by one group become public entities that might be useful and enriched or modified by another group b) in LASAD discussions student constructions are integrated with the form of referable objects (see fig 2 and 3) and become part of the discussion –thus as the discussion unfolds different states of the construction are integrated in it c) LASAD offers the potential to structure discussion defining different types of contributions. For this study we used contributions such as “help request” for the students to describe the problem they were encountering and bring in the discussion the problematic construction state,” microworld actions such as change symbolic expression or find relationships”, “comments” and “my microworld” contributions where students could bring into the discussion a specific construction state.

Our data in this section are derived from student exchanges in LASAD. In the pictures below (fig 2 and 3) we depict two instances of help request with larger LASAD discussions. Figure 2 depicts an interaction between two students (S1 and S2) with an intervention from the teacher. The episode depicted in fig 3 takes place after another student S3 was prompted to check out the discussion between students S1 and S2 . After reading the discussion map in fig 2, and because of the lack of detail, S3 was not helped and instead posed a similar question, which the teacher decided to answer due to the lack of other students who could help at the time. Both S1 and S3 encounter the same problem: when the ‘play’ button is clicked the variables involved in two patterns change randomly and thus the construction looks ‘messed-up’. This problem can be solved by ‘linking’ the variables together using a symbolic expression to represent the relationship between them (Mavrikis et al. 2012). In both episodes students use the same “method to ask for help”: they combine the verbal description of the problem (in both cases a rough description of the microworld behavior: how to make both patterns move together) with constructions that either represent the problem (fig 2) or the expected result (fig. 3).

The main observation when comparing the two discussions is that student discussion is dominated by construction examples (fig 2) rather than the more expert-guided discussion that consists of requests for verbal descriptions of problems and solutions (fig 3). More specifically in the first episode help seeking and problem resolution had the following form: S1, described his problem, S2 opened the model, identified the problem and suggested a solution in the discussion (i.e. change the expression of green to repeat red -1) and provided a corrected model. S1 copied the solution suggested by S2 in his model and asked from S2 to provide further help on the next step (what is the rule that makes the model to work). In this discussion (despite the teacher’s request) there was no further elaboration on the problem and its solution and thus it could not offer any information about this problem to S3 later. In terms of help seeking, S1’s and S2’s behavior seems inherently executive (ie seeking or providing the solution to the problem) rather than instrumental (i.e. explaining the method to resolve the problem) (Nelson-LeGall, 1985).

Fig 2 Help request: student interaction         

A completely different situation is observed in the second episode where the teacher aims at guiding S3 – rather than providing a solution – by suggesting that he first observes and expresses relationships. This helps S2 to resolve the problem. The question that we are posing, therefore, for further research is how we could structure the social interaction between the students so as to facilitate deeper understanding through the process of help providing and help seeking.

 

            Fig 3 Help request: student – teacher interaction

Concluding Remarks and future research

Our analysis on students’ learning how to learn with microworlds revealed that the evaluation process is a crucial element during learner interaction with the microworld and can take various forms linked to social aspects (personal evaluation) different understandings of the concepts under investigation (Boolean evaluation and explanatory – causal evaluation), and learning how to learn skills (i.e. describing the problem, identifying an impasse, reconceptualising or forming subtasks or goals). Thus our research contributed in elaborating on the different types of evaluation that might take place during interaction with a microworld and linked them to other elements of this interaction (task/goal and subsequent actions on the microworld). Our study further focused on situations which are evaluated by students as impasses and we analyzed how students handle these impasses in the social practice of help seeking. Research results showed that dealing with an impasse is a critical point, rich in learning opportunities which can lead to the construction of new knowledge. Students however, seem to have difficulties in articulating their problem in explaining the method for resolving the problem and in personalizing suggested solutions before integrating them in the constructions. Based on these results our future research will focus on designing an intervention which aims at structuring the help seeking process with a set of contributions in LASAD which are based on meta-cognitive skills (reflecting, setting goals and subtasks) and microworld actions (e.g. finding relationships, describing unexpected results).

Acknowledgements

This study is funded by EU-R&D project “Metafora - Learning to Learn Together: A visual language for social orchestration of educational activities” (EC/FP7, Grant agreement: 257872).

References

Balacheff, N. Sutherland, R. (1994) “Epistemological Domain of Validity of Micro Worlds: The Case of LOGO and Cabri-geometre.” In R. Lewis and P. Mendelsohn Lessons from Learning: Proceedings of the IFIP Conference TC3/WG3. 3 , Archamps, France, 6-8 September 1993, pp 137–150,

Collins A., Joseph D., Bielaczyc, K. (2004). Design Research: Theoretical and Methodological Issues. In The Journal of the Learning Sciences, 13 (1), 15-42, Lawrence Erlbaum Associates.

Harel, I., & Papert, S. (1991a). Software design as a learning environment. In I. Harel & S. Papert (Eds.), Constructionism (pp. 41–84). Norwood, New Jersey: Ablex Publishing Corporation.

Harel, I., & Papert, S. (Eds.). (1991b). Constructionism. Norwood, New Jersey: Ablex Publishing Corporation.

Karabenick, S. A. (1988). Strategic Help Seeking Implications for Learning and Teaching. Lawrence Erlbaum Associates, NJ, USA.

Mariotti, M. A. (2002). Influence of technologies advances in students’ math learning. In L. D. English (Ed.), Handbook of International Research in Mathematics Education (pp. 757–786). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Mavrikis, M., Kahn, K., Dragon, T. (2012) Constructionist Discussions With and Around Microworld Referable Objects. In Constructionism 2012, Athens, Greece.

Mavrikis, M., Noss, R., Hoyles, C. Geraniou E. (to appear). Sowing the seeds of algebraic generalisation: designing epistemic affordances for an intelligent microworld. In Noss, R. and DiSessa, A. (eds) Special Issue on Knowledge Transformation, Design and Technology, Journal of Computer Assisted Learning

Nelson-Le Gall, S. (1981). Help seeking : An understudied problem-solving skill in children. Developmental Review, 1:224–246.

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.

Schraw, G. (2007). The use of computer-based environments for understanding and improving self-regulation. Metacognition and Learning, 2(2-3), 169–176. doi:10.1007/s11409-007-9015-8

The Design-Based Research Collective (2003). “Design-Based Research: An Emerging Paradigm for Educational Inquiry”, Educational Researcher, 32(1), pp. 5–8

Verillon, P., & Rabardel, P. (1995). Cognition and Artifacts : A Contribution to the Study of Though in Relation to Instrumented Activity. European Journal of Psychology of Education, X(1), 77–101.