Silvija Seres: Hello and welcome to LØrn. We are at Velveeta center for learning in Bergen. I'm Silvija Seres and my guest is Barbara Wasson, professor and director of the Center for the Science of Learning and Technology at the University of Bergen. Welcome Barbara. 

Barbara Wasson: Thank you.

Silvija: To be perfectly honest I've heard about you for a very long time ago. At the front several friends of mine have told me you have to meet Barbara. So it's sort of fun that we meet for the first time live in podcast. But it's great to meet you. You're Canadian?

Barbara:  Yes.

Silvija: You came to Norway almost 30 years ago.

Barbara: Yes just about 30 years ago

Silvija: right.

Barbara: Yes.

Silvija: And you work in this crossing of technology and really the science of learning. 

Barbara: Yes.

Silvija: And you have international perspectives on that. You chose to stay in Norway and do this work from there. So I'll be really curious about you know what. What advantages can one have doing research in this area from Norway? But before we do that tell us a little bit about who you are and about this wonderful center of yours.

Barbara: Yeah. Okay. I can tell you first about the center in 2015. The government wanted to set up a Learning Sciences Center with a focus on what we call learning analytics. And there was a competition and we won the competition. So we started the center in 2016. 

Silvija: This was Canada. 

Barbara: No. 

Silvija: Here in Norway. 

Barbara: Yes. So in Norway and I sort of came out of the idea that it actually had it had its roots in a commission that looked at the Future of Higher Education in MOOKS and what MOOKS were going to how then explore the impact because now you had these open courses that were available internationally and lots of people were there. And what was that going to do for higher education? And one of the aspects around MOOKS is the fact that you can get a lot of data out of them. So the data logs and everything that you leave behind and what are the implications of that for understanding learning. So that's really what we're looking at. But not just about MOOKS it's about all the new kinds of data that we can get in education. We can get sensor data, we can get the log files, we can get movement data, all these things and how does that help us understand learning and what's going on in learning situations.

Silvija: Barbara Wasson, you're going to be my new best friend.

Barbara: Okay. 

Silvija: Because in Lorn we now have 365 podcasts about Norwegian tech. We believe that this is also a sort of instrumental skill for the future you need to understand the stories.

Barbara: Yes.

Silvija: And what we notice is that you know people talk about lifelong learning for grownups but there's really nowhere to go to get the breadth of the local content fresh enough local enough. And so we'd love to measure. And we see that there are lots of these so-called learning management systems Elvis took, some Norwegian, some international but really making sense of people's learning. I really don't know what to do.

Barbara: No, it's really difficult. 

Silvija: What do you learn? I mean what do you look for patterns that you see, do people consume things in the digital world more or less? Does it stick better? Can you say anything about that?

Barbara: I guess

Silvija: I'll just say one more sentence because I think in order to get the H.R. departments in all these institutions and companies to actually invest in this lifelong learning of all of their employees they need some sort of measurable success. And you know the only measurable success you get today is a degree from some MBA or you know and I really don't think we can afford to send people to take degrees all the time every year. So how do we measure this nano stuff that's going on?

Barbara: Yeah I think yeah. So lots of things there. First of all data tells a story which you know is what we try to look and find stories and data and I like to distinguish between different kinds of things we can know about the data. And one of them which is really quite easy is what we call about learning centric learner centric analytics where we're looking at behaviors of students, what they click on and how long they watch movies. Those kinds of aspects of it are very easy to get. Basically like what Facebook does.

Silvija: What can it tell you?

Barbara:  It can tell you about engagement and patience. It can tell you if you're a designer if you're a teacher and you're designing learning materials that can tell you which ones are of interest to the students and which ones are not. For instance, especially when you invest a lot of information and that like what they find useful that kind of thing but to get out what we call learning centric analytics trying to understand learning processes that's much more difficult.

Silvija: That the effect how much understanding has created is that.

Barbara: Well the interesting thing is that if you look at educational research people especially in the area of technology and learning have always tried to ask what is the effect and that's what everybody wants to know. They want to know what is the effect of this but the aspect is not necessarily what is the effect. We don't, it's often efficient and it's often the better question is this how are we learning differently? So we're trying to look for what we call productive learning practices so people are engaged in deeper conversations or they are getting access to more videos on things or they're trying to build their own understanding by getting resources on the Internet.

So to get to that deeper level of understanding it's much more difficult to measure. So if we look at this field of learning analytics which is really only about five years old it's really been very much on behavior orientation how they do that but trying to understand deeper and deeper understanding is it's much more difficult and we don't have the answers which is why it's a new field but it real sorry.

Silvija: But nobody else has the answers either. I mean, are we advanced in this area internationally?

Barbara: We are advanced in the fact that the government recognized we should have a center in this area. Yes. So we have a lot of demand on our time. We just did our year report and last year we had 172 meetings or something like that and gave 52 presentations. I'm invited to five conferences now in the next two to three months just to talk about this. This area of learning analytics so it's a very big interest. But as a researcher who's been in the field for a long time and I'm 10 years older than you so I have I have a PhD in A.I from 1985. So that was a long time ago.

Silvija: There is a lot of hot air. Maybe not all that much revelation.

Barbara: But the interesting thing is that when I came to Norway I finished my PHD in 1990. I had a PHD in artificial intelligence and nobody was interested in it. So for me it's a little bit comical that almost twenty five years later it's like the hottest thing in the world. And the thing is this theory I worked in during my PHD work and Masters actually is trying to build what we call tutoring systems. So what we were trying to do in those days was to build systems that would teach one on one like a teacher who would do so. We know from research that the most effective way to teach somebody is to sit there and find out what you do not understand and then to give you instruction in exactly that area. And so we tried to build those systems when they were knowledge based systems rule based systems just like you are programs here they told people how to. They told the computer how to behave when we knew this and that about students in particular ways and now we come to today and everything is going around to the big data perspective where we collect a lot of data but we can't explain the what comes out the other end of it because it's more machine learning oriented and one of the big questions we have in our center is can we learn enough from the machine learning algorithms to actually improve learning in a certain way. So I have a good example of that but...

Silvija: ...please do. 

Barbara: Yeah. So if we think about multi smart Ørving I guess most people here use multi smart Ørving or a lot.

Silvija: You online the math tool...

Barbara: ...math system. Yeah. Now it's based on data. It's based on an algorithm that's developed in New York by a company called Newton and it's based on recommender outcomes just like Amazon based on you buying this book.

Silvija: So my kids are doing their homework on multi smart Irving and the homework is usually spent 15 minutes doing it. 

Barbara: Yes 

Silvija: And I understand there are some dynamics in it. So I didn't I don't know if it's the faster you do it to the harder problems you get or the more correct 

Barbara: It has to do with correctness and the problem is that actually I'm not the first answer that they give is not used in the algorithm because sometimes the teacher helps or their parents help and that can then give a false indication of how good the student is. So you can get things or do something difficult. But on the other hand the system also can give you. Yeah. So they play the system a little bit as well. But the difference there is the algorithm that you're given because that systems are adapting to you to give you practice programs exactly where you are. It's taken on the average of all students. It's not taken on you as an individual and what you know when what you don't know. So it's we don't know whether it's adaptive enough that you're actually going to learn something you're going to get good practice but whether it's going to take you to the next level. It's not so clear.

Silvija: if you're really good. It might not be pushing you enough.

Barbara: No I don't think so. I think the best students are not challenged. And it's partly a decision also of Yelland all that they do not allow you to go into the next level so if you're you know if you're in Grade 4 You can't go to the grade 5 level and things like that.

Silvija: That's a problem. So I am probably not objective at all but my kids are really good at math. And I'm always frustrated because they're bored yet. And so then is it a business decision not to let them slip into you know books and knowledge that they haven't. The school hasn't paid for on their behalf or. Why not?

Barbara: Now we'll get into our I think we'll get into our cultural differences okay. Yes because both of you and I come from different cultures when it comes to learning and we allow people to excel in academics I also came to note when I came to Norway I was really surprised that you're the heroes are the sports people not the people that do good at school and people would even try to hide that they were good at school. So that's a little bit of a different culture. But I think the fact that they don't want people to go ahead and to push the limits.

Silvija: they don't want polarization in a way.

Barbara: Yeah it's everybody you know once the leak is stealing it everybody should be very much more like. So I don't know. It is a business decision inside the company but I think it's based on the culture of going too far ahead.

Silvija: I want to go back to something you said you worked in these A.I expert systems that were recommender systems for new knowledge in a way. And you're saying that you're a little disappointed that not more is happening in this area because we could be using machine learning deep learning to be better at challenging kids to go deeper.

Barbara: I think yeah I think definitely but I think part of the problem is that like when you talk about technology it's many things like a lot of the examples that people give are more communities facilitating communication or are they giving access to information. And there's very less focus on systems that actually have a content that can help you learn that particular content and there's lots of issues around how do you there comes a Norwegian problem because how do you put the framework around what knowledge is going to be and only teach them those things but these systems are actually really really good at teaching procedural knowledge. In the U.S. there's a lot of the systems, especially a colleague of mine at Carnegie Mellon has developed what they call cognitive tutors and they're used in many many states in the USA to teach people mathematics but multi smart Irving I would say is not a teaching program. It's a practice. Yeah and there's a big difference there. So the focus has moved more away from these systems that are trying to teach you something to facilitate learning.

Silvija: I have a father, he's a mathematician. And you know he's a bit of a Socrates in a way that you know you ask him a question and he won't let you go for you know half an hour until you've really looked at it from all the important angles that he believes. And you know and it's always down to the basics. And you know I used to be very impatient with him but he's the one who made me love questioning things. And what worries me about some of these digital learning tools is that they're very good practice tools. I think the GMAT is one of these multiple choice things that also is dynamic, probably the same algorithm even when I was applying to MBA and I learned to game it you know you have to go through these really boring problems. You know two people are digging a hole and how many hours and how many shovels. And I don't know you know. Trains are coming from here and there with lots of sweat while they calculate if you manage to do two or three or four of those quickly and correctly then it starts giving you really easy algebra problems and you know once you get over that threshold you're home free. And it's all about practice and it's really difficult to get kids to start questioning why am I doing this this way. Do they teach you to answer in a specific way and then you think maybe even less.

Barbara: Yeah no I can see that for sure. I had a father like that as well. Yeah no I. Yeah. And that's why there are roles for all these differences. When I came to Norway and I couldn't work in A.I in education which was my background then I moved into collaborate collaborative learning and was actually involved in the first network classroom where you had collaborative systems where students sat in two different classrooms in grade six sixth grade was in a school in Toronto and they could put information into an empty database and they could give comments back and forth to each other and this was the first time in the world they had done this kind of system. And before that it had been like you had you were trying you had a system the way I was talking about was trying to teach you something the system just allowed you to put information in.

Silvija: So this is super interesting because you wrote to me earlier that you believe that the most important things that kids need to learn and these goals for grown-ups I guess as well are critical thinking, new ways of collaboration, all this creativity, ethics and so on. And you know we spoke about critical thinking which is not just about following the process but thinking about why and now you're saying giving us a very good practical example of this new way of collaboration that is enforced facilitated by this technology.

Barbara: Yeah. And it was kind of interesting because my background and this is one thing I think is very different than that then in Norway. I'm a computer scientist but I have worked with ICT in learning and taken education classes all along the way. So I actually even have classes in special pets. But the interesting thing is that's allowed there. You're allowed to have these cross disciplinary interests. And in Norway it's much more difficult to allow somebody who likes my department to allow somebody who had a psychology background to come in and do a masters in computer science or information science in my department wouldn't happen.

Silvija: yet you talk about all these tread patterns.

Barbara: Exactly. But the system is so rigid it doesn't allow that and it's not open enough yet to allow these sort of things to be more interdisciplinary.

Silvija: Why is it difficult? I'm just thinking why don't we simply demand that you know out of the 70 seaports student vector  points that you get that you know at least 15 have to be real stuff in psychology and you choose three other subjects from completely different faculties.

Barbara: Yeah. I don't know, I think a lot of it. Well now we're getting into my opinions, the Norwegian educational system. Yeah. No, I think it's very different from where I come from. You had much smaller classes over very short periods of time and you had the room then to take an opportunity you had the opportunity to take much more elective courses and you could pick from different areas and bring it together. Whereas here we have these large classes that are 15 vector all taught 2 courses in 15 vectors each semester and that's the only decent you have so you get a much narrower education. I found here in Norway there's a much narrower education say within computer science even though I had a much broader background.

Silvija: Is it because every course has to be very much pre approved and prearranged.

Barbara: Yeah it's terrible. I mean if you want to make a change you know if I want to make a course it takes almost two years to get it in which is bad.

Silvija: we find the university that will let us make our own brews and then you and I make these mini courses. I think there'll be huge funds, so when people talk about digitalization ICT is cool. I think there is very much focus on, you know, using the tools and understanding the tools learning to program. And now we're talking about it's not just that it's about understanding the new opportunities they open up how much programming you need to know before you can do that. I mean you say you have a background in ICT PhD in A.I. how necessary was that for you to be able to do what you're doing today.

Barbara: Yeah. It's interesting because I come from that generation. My father actually got the second computer in Canada in 1958 before I was born. And so I grew up with this but I didn't know anything about it. I knew there was something called Fortran which was programming languages but he didn't really know what it was and I didn't even start in computer science I started in engineering and then I switched actually right before I was in my first semester into computer science and fell in love with programming. And I think that one thing we know from reading if we're gonna think research perspective now we're into where I don't think we know about introducing algorithmic thinking and programming in the young schools one thing that we do know from research is that it doesn't translate these visual languages don't translate very well to the script based programming. So just because we're going to introduce it in the school doesn't mean we're going to have a lot of really good programmers out the other end. But I don't think the problem with research is it takes so much slower to do the research than it takes to come with the technological innovations and we can't keep up. We can't keep up with it. We're trying to understand how this changes practices and how things are done differently and that might take us a year whereas than the next technology is there.

Silvija: There's several things we. We are sort of running out of time but I'm just going to keep going because I can't stop now. One thing I want to in a way challenge you or ask you about is the way we measure different school efficiency across cultures and nations is these PISA test and I have an opinion that it's measuring things very much the old fashioned way. They don't. It doesn't check whether you actually have this creative thinking growth mindset a critical abilities etc. Could we get some more of that kind of understanding from the sort of data that you are seeing? 

Barbara: The understanding.

Silvija: Can you do a new PISA for the new generation of education? 

Barbara: There actually is one of the PISA is on collaborative learning and they have an intelligent agent that sits there and interacts with people. I was on I was giving some input to that. Well my problem with it was the problem I thought was about it. I teach computer supported collaborative learning at university. And one thing we always talk about collaboration but we don't teach people how to collaborate. Actually we know that when you collaborate you need to negotiate common goals. We know that you need to control the process in some way but we don't teach this but we start trying to measure it suddenly. And that was my problem I said I don't even think that my master's students that have studied this collaboration and as a fog would do well on that test.

Silvija: It doesn't happen just by giving them the tools you need to teach them rules of engagement first and then it helps them practice.

Barbara: Yeah. And so I think that the new kinds of data and that's I mean that's also if we think about what we were talking about how things change everything is driven by the assessment system and where we place value. And the fact that we have to have final exams and now we're doing a lot of studying on how we can process or give marks along the way. And I mean when I came to Norway also it was very much shock that you could study law for four years never engaged with an instructor or student and write an exam and they were surprised that 60, 85 you know I think it was something like 80 percent failed and then they went and they put in a digital system that allowed people to write a little text and give comments on each other and suddenly it flipped 80 percent for passing. But we know that's good pedagogy. We know that you need feedback along the way and we need feedback in the process. So in many other countries you get marks for the work you do during the semester. It's not just the summative at the end. And I think that we're moving towards that now but we're trying to understand how can we measure these things. And now we want to have collaboration and creativity and how can we measure those we don't know. We don't know the good ways to measure that you know what.

Silvija: We are trying to learn that slate and that's a good thing.

Barbara: Yeah we're interested in these kinds of questions though. 

Silvija: So another question I want to ask you is I have a friend who 's been one of the really successful edtech tools that has grown very big in the U.S. as well. And he was saying that you know there is people who look at it almost like magic. You know they believe that by importing this whether it's a learning management system or you know suddenly by digitalizing with that one step you've created magic. And at some point he was you know he was that these school leaders were talking to him and do you know the expectations were completely unrealistic. And he said they had to stop this woman. She had one of the best schools in the U.S. and you know he was saying well you know and he knew that she was a very passionate cook and he was asking her well you know you're a good cook. And if I was to become such a good cook as you, you know what would you recommend me to do buy the best you know the best parts and whatever and then it would all make the food itself which is of course that you know need to know how to use it and how to mix things and how to do so well exactly the same thing with these digital tools we need to know what we want to do with them first.

Barbara: Yeah.

Silvija: How do we help.

Barbara: Yeah I don't know, I'm a strong believer in the empowerment of the teachers. And we have to teach teachers to be creative and they are creative and they have to be given time. I mean one of the biggest problems that we see from a research perspective is the time to integrate tools into your practice. Everybody can learn to use the tool but to integrate it into your practice and to have the time to learn to do that and a lot of times the researchers on the first time people are trying tools and that's we know we get better with using things over time. You have to try and fail and figure out. So I get a little bit annoyed with the research literature that often says that tool doesn't work and I think well that was the wrong tool for the wrong task and we need to give it time to work into the practice. So I know my colleague Austria at the University they studied a formative assessment when it was brought into the schools and they fault most schools try it for a little while and give up on it in the schools that really got it well they use six years to get it into the soul of the school and the way that they do their pedagogy and things like that. So we have to give things time. And this is where we have all of this you know technology changing so fast and coming in. But when we looked down at the underneath of all of it that's sort of a you know we're trying to give students access to the best possible ways to learn.  I mean that's what teachers want to do and that's what we have to believe in them.

Silvija: What are the positive aspects we're coming close to the end of our talk the positive aspects of doing this from Norway something that we are especially good at or that can help us understand is better than the rest of the world 

Barbara: I think there's two things. One is that we have had I.C.T. skills as one of the equal to math and reading and listening and mathematics. We've had that as we were the first country in the world to do that. So we see that it's something that's not on the side of everything else it's integrated across disciplines and I think the other thing is that teachers in Norway have a lot of trust. They may not feel that but they do as compared to many other countries and that teachers are allowed to make choices for themselves about what they want to learn or what they want to use in their in their teaching and they can make a lot of decisions on their own and I think those two things together. When I look at European projects. If we go to we had partners in Germany and Austria you hardly find technology in the schools at all. They're told exactly what tools they have to use and how they do it etc.. So I think that freedom here to trust teachers to find things and to participate in these you know Internet groups and we had this dello group before in Norway the teachers used to share a lot of things through and we allow them that freedom to do that. I think that's positive. 

Silvija: Barbara, if people remember one thing from our conversations we talked about so many things. What do you think is kind of the central point?

Barbara: I think what I'd really like people to learn is that the question not to ask is this better than something else. Because technology fundamentally changes the way we do things and the better question is to ask how we do things differently. Can we see that we engage people in more depth of off topic for instance and these kinds of questions it's not is this better it's how is this different and how is this changing learning and understanding that,

Silvija: I'm going to go and check out slate and see are the publications we can read.

Barbara: Yeah. But I'm not sure that they're so interesting because they're maybe too theoretical but the one thing that we have been involved in is that there is the Open University in the Netherlands. They put out a report called innovating pedagogy every year. And last year we participated with them in developing that. So some things in there were like learning with drawings and different kinds of things like that. So that's a nice report if people haven't seen that so you can just search for innovating pedagogy at Open University and you'll come up with those reports.

Silvija: Innovative pedagogy at Open University. Thank you. Barbara Wasson the director and professor at Center for the Science of Learning and Technology at University of Bergen. Thank you so much for coming here and inspiring us at Lorn.

Barbara: Thank you.

Silvija: And thank you for listening.