Research quality and integrity

Neelam Raina: Hello and welcome to Middlesex University Research podcast.  

My name is Neelam Raina, and I am the Director of Research at Middlesex University. At Middlesex University, we do many different types of research that has widespread impact on our students, on our academics, and the wider community outside. The ambition of our research is to transform lives, and we do that in many different ways. 

These podcasts are recordings done at Middlesex University by colleagues here, who will talk about their research, their ideas, why do we do research, how does it matter, and all kinds of issues in-between. So welcome to Middlesex University research podcasts. Thank you for joining us. 

Hello and welcome to Middlesex University Research Podcast. Today we have a special guest here, Professor Anne Boddington, who was the Pro Vice-Chancellor for Research and Knowledge Exchange at Middlesex University, and she's back here with us at Hendon to speak about all things research. I will let her introduce herself and then we shall get into a very interesting conversation. So, welcome Anne.  

Anne Boddington: Hi Neelam, it’s nice to be back at Middlesex. My name is Anne Boddington. I was, as you said, Pro Vice-Chancellor for Research and Knowledge Exchange at Middlesex for nearly two years as an interim position. Prior to that I was Pro Vice-Chancellor for Research, Business and Innovation at Kingston and Chair of REF 2021 for Art & Design: History, Practice & Theory.  

Neelam Raina: I am most excited about this. Let's start with the elephant in the room that is the REF. REF is, for those of us who don't know about it, REF is Research Excellence Framework that the UK runs across the university sector once every so many years. It changes its mood and its operation and what it's looking for. 

So maybe Anne can tell us a little bit about REF 2029. It looks very different. Is it just me or is it a really different REF this time?

Anne Boddington: It's a development of the REF as we knew it. REF 2021 was an evolution in and of itself which came out of the Stern and the REF 2014, which was the first Research Excellence Framework. 

REF 2021 was a kind of halfway house, whereas REF 2029 will be a full shift in how the Research Excellence Framework looks. Partly because it is emphasising more on institutions. It's about the institution and how the institution supports research as well as the research that was conducted within an institution. 

It's more, I would say, a sampling exercise. It takes a sample of research outputs. It is no longer focused on individuals. It's focused on a sample of the research in that field, in a unit assessment as we know them - as in the example I showed, the Art and Design model, which was Unit of Assessment 32. 

The REF is still fundamentally an audit. One of its purposes is to understand quality research as it is conducted and as it comes out of all forms of funding and all forms of institution. It is to underpin the distribution of what we know as quality research in England and what the Scots know as the Research Excellence Grant (REG) in Scotland. That is what funds, or partly funds, the UK’s dual funding system, which is QR plus grants from research councils.

I think the other thing about REF 2029 is that, for the first time, it has a question about the health of the sector. For the first time, panels will not only be asked questions about the research quality but also about what the overall health of the sector looks like. I think that's particularly pertinent at a time like this when universities – and higher education generally – are going through quite a tough time financially and economically, particularly in arts and humanities.

Neelam Raina: It's been a tough time for all of us in this space, and I don't think it’s over just yet. However, it does lead to the big existential question: why do we do research? Why does it matter? We are public institutions here to serve the public and grow knowledge, but when we are struggling for finances, how does research really matter? Why is it important that universities do it, and how do we contribute to society as a whole through this activity which is quite time-consuming?

Anne Boddington: It is, and one of the things I would say about REF is that the intention has always been that it would be less time-consuming and more systematised in its structures. We have yet to see that happen in that role.

But the question as to why we do research, which is how do we know and what do we know about any field in which we're working and how do we know that it's advancing – and that is through the process of research.

I would say that if we are going to be excellent teachers, we need to be able to say to students what is it we don't know about our subjects that they need to find out? To be honest, one of the things I would say is, why do we teach what we already know? One of the reasons to do research is to try and answer that question: what is it we don’t know about our field, what is evolving in the field, and how do we inspire next generations to take up that mantle? If what we tell them is what we've known for the last 50 years, how will they ever discover what we don't know, and how do they ever take that forward?  

For me, the thing that's important about research is that it is, or should always be, at the cutting edge of the field. It’s not about being new to an individual - it's about being new to the world and how we make it new to the world. Positioning students in that space, rather than taking them back 50 years before moving forward, seems the wrong way around. 

But in terms of teaching, and if you want to bring the relationship between teaching and research together, it’s because we need to understand what we don't know about the world – so we can do something about it and inspire next generations to come with us on that journey. 

Neelam Raina: Thank you. There is a degree of ambition, aspiration, and also optimism in many ways when we do research because we are hoping to find: one, something, two something of value; and three, something of value that can be of value to many others apart from ourselves. Research carries a whole big hope and is inherently future facing. 

Anne Boddington: Yes, and it's about how we might look differently at a subject that we might have looked at in one way for a period of time - and now want to look at it differently. You and I have had conversations about how something known in one language may not be known in another. 

There’s a lot of work around translations, interpretations, about rethinking the position of women and different cultures, and how they come at a question. There are still ways of revisiting old questions in new ways and that’s as valid as, for example, discovering the next new form of artificial intelligence – or even human intelligence. 

All of those things are really vital, and they are as vital for me in learning and teaching as they are in research. Even a teaching intensive institution needs to understand research and scholarship. It may not do the same intensity of research, but the fact that there is a curiosity by definition in the university about a subject, takes it beyond learning the field and learning what is in the field.  

Neelam Raina: That leads very well and quickly into the whole idea of why do we do this? I know there are words around impact that are bandied about every time REF comes around - but let's put REF to one side for now. The whole purpose of undertaking this research, which is often curiosity driven or reflective - looking back at things that are not fit for purpose - could be challenged or understood from different perspectives, depending on where you're coming from. It is about the aspiration for change, transformations, and impact. The word “impact” moves around in different circles, but research that is impactful embraces the many understandings of what impact means.  

What do you think about impact outside of the REF space – this is impact of the university sector that is curiosity driven and teaching driven, asking questions to the different space? What is the relationship between this idea of excellence and impact as a guiding principle for those who joined this space, who just finished a PhD, or are undertaking a PhD - who are still feeling their way around, and what they want to see as impact in the next 20 years of their career? 

Anne Boddington: This may be about the fields I come from, so I’ll caveat that - but for me, the important thing about the idea of impact is that it's about the difference you make and the evidence of the difference you make. If you took it in learning and teaching terms, you would say it's the difference between an output and an outcome.

Research might be the paper you produce, and that's about the new insight or a new knowledge that you might be developing. What you do with that information is about what impact is. So where do you want to take that? Do you want to take that into policy? Do you want to take that into the community? Do you want to take that to your family? Impact can be at many different levels. It doesn't have to be in anything to do with Research Excellence Framework. 

If you take it in a university sense - if you took Middlesex University - what do we do for Hendon? What matters to the people beyond the walls of this institution? How do we use the new knowledge, new insights, skills and competencies that come with that to change the structures of what happens outside the window, in that very real sense of how can we enable and support others to do that?

That's where higher education is really valuable in a place-based sense. Irrespective of what part of the sector we sit in as an institution, what difference do we make to the world just beyond our borders - literally outside the window?

Then there's how we might learn from that and how we might take that or join that up to research either regionally, nationally, or internationally. And vice versa. If you learn something internationally, can you bring it back and does it relate locally? Fundamentally, it's about the relationship between producing and understanding the knowledge, or having an insight and saying, “Well, what do I do with that insight? How do I make it useful?” Of course, there is absolutely a place for research that advances the field itself academically - that's what academic life was broadly built on.

But the other side of that, particularly for a university like Middlesex (and for many institutions), is how do you take that and apply it to different understandings in the world - whether that's a community, a policy, an environment, about health and wellbeing, whatever those areas are, it's actually really about what kind of outcomes you want to develop.

Quite often - the classic test for this is – if you ask a new PhD student what their research question is, they will look at you entirely blankly. But if you ask somebody what difference they would like to make in the world, they can probably tell you. The really important thing is if that's the difference you want to make, what's the research you want to do that helps you make that desire real in a very tangible way?

Neelam Raina: That is what makes good research design in my opinion. It’s when you know where you want to get to, at least in the space of arts and humanities. You can see a solution driven approach that might work. You can scale it up, scale it down, or put it back into the community and let them do what they want to do without cascading it.

Very often the research question when you fill in an application for grants is ‘What is your research? What are you trying to find out?’ And when you put collaborative design into it, you might say: ‘I know approximately where I want to go with this, but I would rather sit and work with the community that I want to service, function or operate with, who can then design those research questions collaboratively so we can co-own this piece of work’.

Ownership isn’t something academics are usually interested in. But this approach means that you are looking at building impact from the get-go - rather than chasing a research question that arrives at a method, goes into a space, and then hoping to pull something useful out of a pile of paperwork.  

Anne Boddington: Yes. The other way of looking at that - and I've always had this debate, which may be very much about the creative world that we work in – is: could the output from a research grant be the question? So what you do is you define a rigorous set of iterative methods and the output is the distillation of a question. So the last thing you produce is the question, but it's done through a set of experiences, iterated with a community. I think that's what you were getting at – the sense that you can work that both ways.

I think there is a real need to be able to be open enough to say - and I think this is true for audit structures too - that we may have iterated something or been through that literature process enough times that we can look at all of those case studies and therefore come to a conclusion. A conclusion is a refined question. So now we know what the question is, even if we had a positive question at the beginning.

That sense of being forever curious is what drives people to continue to research, rather than researching cynically - which is: “I've done three quarters of the research, I’ll apply for a grant, I get the grant to keep my research group going”. There's a kind of rather more cynical system that locks us into all sorts of bubbles.

Neelam Raina: Yes. I think it's also an old-school way of looking at research as a linear process. You start with the question and progress to stage two, stage three, and stage four, and at stage four you say, “Oh, I will make policy impact by talking to somebody”. Research is very often seen as an old-school pyramid that existed when I first started in this space: you do the research, you push out the output, and hope somebody in policy and programming will look at it and maybe cite your work at some point in a policy document - and hey presto, you have impact.

Anne Boddington: Research is messy. Like life.

Neelam Raina: Yes. It's very back and forth.

Anne Boddington: One of the things that's interesting about the REF process is it starts to ask people to untangle that mess a little bit. And that is where I think you do REF well, irrespective of results and money and all of the other things, if you do a good REF in an institution, what you are doing is some of the untangling of some of those things.

Quite often, people will say at the end of it: “I’m much clearer about the things I'm doing”, or “This is where I think we make a difference”, or “This is where we can evidence that we make a difference - and this is where we can't”. Those thoughts and discoveries are great discoveries for institutions, for individuals and for communities in that process. The whole process is a process of discovery. It's not that one thing is the formal process and then there's another process; they are all part of the process, and it’s messy - it’s like REF. It doesn't go according to plan, and if it did, it wouldn't be very interesting.

Neelam Raina: Yes, it'd be pretty dull and boring. I also think that it kind of tumbles into the question of value. Very often, those things that are metric driven - clear indicators of what is success and “blah, blah” - all of those things make the value proposition of research a driving factor for everything that you will sift and untangle. If you were to unpack the research and its messiness to actually sift through and come up with what this was about and how does it matter, then the hard wiring has to be changed. We have to be reprogrammed, maybe somewhere, but we also have to step away in many ways from that principal idea of what is valuable and what is not - because we see value as an exchange, a limited-dimension product, saying that this is valuable because it has been cited a million times somewhere. That very tip-of-the-iceberg understanding of the value of research is something we need to distance ourselves from because the messiness of research produces different types of value.

Anne Boddington: I think that's why, for example, it's really important to keep the datasets. The value of having a good dataset, which may have been done for one purpose, is that it can support an open and trusted research environment from the start. I also think that different people and communities will see different kinds of value in data, for example.

That's a very simple thing, but they would also see different things revealed in any form of knowledge - whether we are talking about visual representations, or textual representations - depending on the nature of what it is we are interested in. Given that everyone has a bias – depending on the lens through which we look at something – we will each see and observe different kinds of insights, and that’s healthy.

It's also the best thing about whether it's human nature or indeed artificial intelligence that depending on the question you ask it, you’ll get different answers.

Neelam Raina: Yes. I think there is something to be said about the value of these things too - datasets for other groups of people. I will put out my favourite word that is decolonization of these spaces. Open and trusted research perhaps has, after a very long time, provided that opening of not reproducing hierarchies of knowledge - allowing access to publicly funded research to people who are outside of us, outside of the English-speaking world, outside of the UK - in whichever space they want to use, consume, observe, engage, or immerse themselves in a dataset that is open, and then find meaning from that space that creates for the bodies of knowledge or questions, critiques, or what has been put together.

Could you tell us a little about a perfect case scenario of how you would define open and trusted research? What should we aspire to when we talk about trusted research?

Anne Boddington: I think one of the things that that comes back to - and this is where I think I'm biased because I'm a design researcher in that sense - is that it’s about the rigour of the design research. One of the things that I think REF and a lot of peer review has taught me is it's interesting when you do a PhD it's almost all method, and it's the risk associated. Why have you chosen this method? Why haven't you chosen this method? What are the structures and what are the risks around that?  

It’s interesting because you can usually spot a paper that has come out of a PhD because it's so well methodologically defined. It's interesting when you see the next three papers come out, they're not as rigorous necessarily. They don't go through that same rigour and push because the demand is not as great. One of the things is the methodological frameworks and whether we interrogate those enough and whether peer review interrogates those enough.

For me, that is one of the big questions in open and trusted research - other than the research integrity questions - is that we publish things and methods that don't work far less than things that do. Most people want good news. Research law is not, ‘Well, we got this completely wrong’, so we end up wasting an awful lot of public money because we don't publish the things that don't work, and I think that’s a question around the world.

Millions and millions of pounds, dollars, yen, or any other currency are often spent because when something doesn't quite go according to plan, we don't see it as publishable or open. That's one of the big questions around selection territory that we need to tackle. We still have a problem with that because research that probably doesn't work as well might not be sighted as much as research that does, and we get into that really difficult territory of those two things coming up against each other.

People are too worried about league table positions, QS rankings, or citation indices to actually really think through some of the really core issues around things that we get wrong, things that don't work for us, that we've received an awful lot of public money for. Which is fine provided you then make it open.

Neelam Raina: I think that is a cultural shift, which I am hoping that some amount of cultural reflection in some space - whether it's inside or outside of academia - will allow us to do that. Right now, success is driven by a culture that says you have to be perfect, you have to be good, you have to be successful. Everything has to go right, and when it goes wrong, we don’t talk about it.

After 25 years in this space, I look back at where I went wrong all those years ago and I still see people going wrong in the same space with the same thing. You think, “hello - 25 years of the same mistake being made because nobody's allowed to talk about that mistake”. Nobody's allowed to give a talk about such mistakes because this is not an area that we facilitate dialogue about or in, so you then have a whole bunch of researchers who are afraid.

When you start off in the real research space, you don’t want to make any mistakes or take risks. That entire space of where things have gone wrong is a dark hole - people will come quite close to the edge, they might even fall into it - but it almost feels like a taboo space to talk about things that went wrong that we must tell everybody about so nobody else wastes any more time, money, or resources on repeating those mistakes. Within the science field maybe this is different, but within arts and humanities, it's almost seen as a rite of passage that you fall into some of these holes.

Anne Boddington: I think the idea of “right” and “wrong” is just that it didn't work or didn't work the way we thought it was going to work. I don't think that it's necessarily wrong, it's just that it didn't do what we expected it to, and that seems to be a very difficult thing to document and to track.  

It's very hard, and it becomes even harder as we start to rely on things like AI or generative AI. We're asking for trawls of what is published, because what is published isn’t quite what works. It's that sense of maybe not capturing the nuance of what didn't quite go according to the plan that we put in place. That tends to lead to a sense that we may not necessarily, or we might be, reinforcing the knowledge in particular ways that we need to unpack.

I’d be very cautious about something where AI, in a way, replaces colonialism—as a model that skews knowledge in very particular ways. Our job as humans is to examine our relationship to these systems and to recognise that what’s training all of these large language models may already reflect a skewed sense of what knowledge might be.

There are some really interesting and important debates to be had about how those things are explored, and why I think it's interesting when the form of research changes. Not only the written form, but also the spoken, sound-based, or practice-based forms help us see those things differently and perhaps unlock different ways of seeing, understanding and thinking through what research might be and is, and indeed what learning might be and is.

For me, I started life in a learning and teaching space. I didn't start in research studies; I started with how I learned things myself and then how I taught other people those things - usually from mistakes.

It’s also how you enter research if you haven't been through the traditional route. I started in a professional world, so it was very different coming to research for me.

Neelam Raina: This leads to questions around ethics, which always remains on the side as a present silent ghost that is watching everything you do. With this reinforcement of knowledge being slightly taken out of our hands through LLMs and reasoning models within AI, there’s a tension. There's a space in-between absolute paranoia about what AI can and will do, and a space of ethical responsibility for the knowledge we own, possess and create. Where do we responsibly position ourselves in this space?  

I've done a podcast on this, focusing on the ethics of research within AI and data points. For example, the data points that are picked up by some of these AI bots and spaces, are skewed data points because they are from a specific part of the world, speak a specific language, or are digitally connected - leaving behind all those people who don't matter or don't count as being data points.

So, there's a principal issue around this, and I think as research evolves, we will tread into these spaces and explore them. I was going to ask you a question about the significance of research, but we’ve kind of covered it at this time. I will circle back to the concept of rigour of research - both from the perspective of the researcher and the research itself - because we've spoken about decolonising and how AI is entering this space.

How do we understand rigour in research that is not tied to excellence, that is not tied to trusted research only? There’s rigour as defined through a PhD programme, and there’s also rigour defined through peer review of all kinds. So how would you judge or define rigour in research?

Anne Boddington: I can't remember who said this, but I remember when we were talking about how to define it, and somebody saying, “Rigour is not sloppy”. That’s a kind of shorthand way of doing it. I think the best descriptions that I've come across are about how you work through the chain of reasoning between a question and a finding or output.

It's that sense of have I made assumptions? Where have I made assumptions, and have I questioned enough something that is more than opinion? So where are those differentials? I think that sense of the strength of the chain of reasoning between a question - the question you ask – and of course, that raises all sorts of things about whether the question is good enough. So, you move back and forth between these are the things I found, but did I ask the right question? That 360-degree between those two positions is where rigour best sits. It comes through that kind of sense of, have I looked at it this way? Have I tested it? How do I test it? What are my blind spots and being aware that we all have them. Language introduces them. Culture introduces them.

There are parts of the world where there is no difference between scientific knowledge and knowledge that comes from the world. In other parts of the world, like here, there is a scientific tradition and the idea that things that happen in the world or spiritual things are not part of that world. This sense of separation – or lack of it - raises huge fundamental questions about how we think about knowledge, and indeed rigour and what that means.

It's about being open enough - there isn't a ‘right’ way or a perfect way - but have you been open enough, thoughtful and reflective enough to pick up those differences, catch those differences and reflect on them? I think that's something that maybe comes with age too. You perhaps become increasingly less certain about things and more questioning as you see the world unravel in front of you and reshape itself. I think that's true in terms of little research questions or big research questions, or indeed life. It's life and how we experience it.

Those things for me are the pieces around what rigour looks like. But fundamentally, I would say it's the relationship between the question and the chain of reasoning that gets you to a finding or an output.

Neelam Raina: That's an interesting way of looking at it. It's making me think - I'm going to think about this for some time.

I think the fact that we have arrived in 2025 - I won’t name whoever's in the White House - but the whole point of being at a point of geopolitical rupture, which has come very close on the heels of a big pandemic that we've all lived through. And we reach a position where we start to question not only: what do we know, and did we ever really need to know that, but also: what is it that we don't know now – that we haven't known for a while - that has made us arrive at this point in 2025, where we are sitting in a world that is turning itself upside down, willingly, democratically, with agreement?

We know that all the scientific advancements we’ve put into place over the last hundred, hundred and twenty years are being questioned very quickly and swiftly, because we have put things in place that allow everything to be scrutinised, or replaced entirely.

I'm not just talking about the research space across the pond, but research that is increasingly driven by the desire to embed tech into everything - where digital transformation is supposed to be the keystone of everything you do, and where you cannot have a talk about research (and we are at fault here too) without mentioning AI.

But this whole bandwagon move towards a certain direction, which we think is the right direction. And yet we come to a point where we are fractured, fragmented into very small pieces. And we have a sector in the UK at least, that is really struggling to stay upright on its feet as we go through another government in the country, and we arrive at a point where we are questioning everything that we have done. And also, perhaps going backwards, saying maybe where we have arrived, is a consequence of the journey we have travelled, but then maybe looking forward.

So, as a wrap-up question - and this is a big one: If you were to dial back 35 years, or if you were to give advice to somebody who has just finished their PhD in whichever field, what would you say?

Not only about the expectations of what they would fix in the next 25 years of their life, but also: what would be three or five pieces of key advice you would give to somebody who has just finished their Viva and is celebrating entry into the academic space? What would you ask them to keep at their core as they pursue a career in this field?

Anne Boddington: As an academic, I think the first thing is to be confident of what it is you can do and know - and to remain curious. Always remain curious.

It is really hard - because I don't think I did this at all – but I went from being completely without confidence about things to being overconfident about things sometimes. And you think, actually, it is about keeping that reflective mindset. It's okay to change your mind, but when you change your mind, it is very easy to swing from one thing to the other.

I think there is something about retaining curiosity and being conscious of when you do or say or think something - it's worth testing it out on other people, the impact you can have on other people by the way you behave. Think about that before you do it. I think particularly for people doing PHD’s and you’re looking at lots of things on leadership and what that looks like - be conscious that it reflects on other people.

It's a really hard learning process to understand that some of the decisions you make, which you think you make for all the right reasons, can have negative impacts on people. That might sound incredibly obvious, but it's one of the things that makes you, on one hand, more confident and sometimes less confident as you get older. The decisions you make can be seen and affect people in very different ways. It took me a long time to really learn some of those things, and I learned them first probably as an architect - in the way you might see the choreography of space and living is not necessarily how everybody lives.

It's those sorts of understandings - when you teach, when you have conversations, when you understand the space and human beings in context – the theory is all very well, but there are many, many ways to cut that theory. It's that sense of remaining both reflective and optimistic, recognising that things change - and you can allow them to change. Not everything is permanent. Even the disruptions at the moment are hideous in many ways, and personally, you might not be around to see our world come out of some of that change - but things happen for a reason, and this world is remarkably resilient.

The classic - you need to break up eggs to make an omelette - is maybe an obvious thing to say, but it's the kind of things that disruption brings that is really hard for people. It's about being human and reflective enough to deal with that.

It's also about understanding that out of it will come other things and other opportunities - you need to be able to see both. It comes back to things like rigour. Some of the arguments that we are hearing at the moment are frankly unbelievable. But I think you kind of say, well, okay, somebody sees the world like that.

We have to receive that and take a view on it, and you need to take many views on it before you get that - hindsight is a wonderful thing. In that context, probably leave it there.

Neelam Raina: Thank you so much. That was most useful. I think we might do a follow-up version of this - let's see what's happening with the REF and everything is pinned down. Maybe we'll have more questions once we've reflected as an institution on what we are doing and how we are doing it.

There is something we were talking about a couple of days ago in another meeting. It was an interesting comment where somebody said to me: Do you do research because you want to change the world, or do you do research because you always wanted to be an academic who teaches?

That was a very strange question because the research that we do is what keeps you awake at night thinking - lying in a hospital bed, thinking about what I will do next. But there is a relationship with the people that one works with, not just me. There’s a relationship between the research you do and how it changes things, and there's a bigger, larger layer of what you wish you could do a little bit faster, a little bit more of. That ambition or that curiosity, as you said, is something that doesn't fall asleep, die, go away. It remains inside your head for the rest of your life.

Anne Boddington: Yes. It's curiosity and still thinking why is the world like that, or why do people think that? Sometimes it's a very small thing and, sometimes it's a massive thing that you can't cope with.

Coming back to the kind of advice that you'd give to somebody. There are things you can do nothing about, however much you would like to. But there are also many things you can do a lot about. They might be lots and lots of very small things, but they're actually very important.

For me, one of them was that there is nothing better than supporting the next generation. I come right back to the beginning: an awful lot of what I like doing is teaching, or helping other people learn is probably a better way of putting it. And I think research is part of that.

Neelam Raina: It is definitely part of that. I shall close this here. Thank you so much for joining us Anne, and thank you to everybody who's listening.

You can find this podcast on different platforms but thank you for listening. We welcome your comments, opinions and see you soon.

Thank you very much for joining us for this podcast. This is a series of podcasts you have been listening to from Middlesex University, exploring the research we do here. You can find us on whichever platform you get your podcast from, and you can also find us on the research pages of Middlesex University, which is www.mdx.ac.uk/research.

All of our podcast episodes will be posted here for you to access for free.

Thank you very much for joining us. Goodbye.