EVERYWHERE you look today, someone is talking about AI. Vendors are weaving it into services we’ve been using for years, charging extra for it, and telling us that students and businesses are already using AI for everything, so we might as well get on board. Of course, as librarians we know that it’s not always as simple as just “getting on board”.

AI has a tendency to make everything more extreme. It can make some things much easier and much faster, but it rarely fixes our most persistent problems. Instead, it often amplifies them. Added to that are the well-documented ecological and ethical concerns that sit uncomfortably alongside the hype.

One area where AI has a particularly strong impact on academic librarianship is journals and resource discovery. The perils of using general-purpose large language models, ChatGPT, Claude, and the like, to search for articles and books are now well known. Made-up references, hallucinated citations, and ¬entirely fictional “top 10” reading lists have become familiar territory. But what about the tools designed to be a step up from this: platforms specifically ¬designed for searching and navigating the scholarly literature? These tools are less prone to outright hallucination, but that doesn’t mean they are problem-free.

There have always been layers, middle-men, sitting between researchers and students, and the journal content they want to access. Sometimes this layer is a discovery platform with resources carefully selected and configured by academic librarians. Sometimes it takes the form of external databases or search tools such as Scopus, Web of Science, or Google Scholar. Increasingly, that middle layer is some form of AI. Over the last couple of years, as AI has become embedded in everyday life, librarians have found that not only are new AI-driven discovery platforms appearing, but long-standing services are also adding AI functionality. Google Scholar now has Labs, Dimensions has its own Research GPT, and Elsevier, the owner of Scopus, has introduced LeapSpace. On the surface, these are simply new ways of finding information. Dig a little deeper, though, and a host of headaches for librarians quickly emerges.

I’ve been doing some of that digging recently to see whether any of these tools live up to the hype, most recently through a trial of LeapSpace, Elsevier’s AI discovery platform. The positives are easy to see. ¬Creating a good search strategy is difficult and time-consuming. Using LeapSpace, a simple natural-language question not only returns answers but also generates the corresponding Boolean search for Scopus. It provides a range of additional material: easy-to-read bullet-point summaries, citation analysis indicating whether a paper is broadly supportive, mixed, or negative, and confidence analysis showing how aligned the literature appears to be. The time savings, and the easier access to dense and technical literature, are obvious benefits.

For these additional features to be worth what universities are increasingly being asked to pay for them, however, they need to be trustworthy. The LeapSpace adverts now appearing in my personal social ¬media feeds describe it as a ¬“research-grade AI workspace”. That sounds reassuring, but does it stand up to scrutiny? In testing LeapSpace, I clicked through from one of its confident summaries to an article cited in support of a particular claim. From the title and abstract alone, it was immediately clear that the AI had drawn the opposite conclusion to that of the paper itself.

In other cases, the drive to compress complex research into a single sentence stripped out most of the nuance. The summaries weren’t entirely wrong, but they often weren’t entirely right either. Crucially, they were delivered with such confidence that a time-poor student could easily take them at face value and cite the paper without reading it.

This, of course, is not entirely different from relying on a review article to summarise a field, and critical thinking is required in both cases. Yet I can’t help feeling that the slickness of platforms like LeapSpace actively encourages trust while discouraging a more critical eye.

CILIP members can read the rest of this article in Information Professional.

A new report by Dr Andrew Cox, Senior Lecturer at the University of Sheffield offers fresh insight into the adoption, use and perceptions of artificial intelligence across the library and information profession. The study, which builds on previous research, provides a snapshot of how professionals are engaging with this fast-evolving technology, and the attitudes shaping its future role in the sector and how that impacts users.

This report follows on from the CILIP commissioned, landmark report led by Dr Cox: The impact of AI, machine learning, automation and robotics on the information profession published in 2021.

This new report provides a clearer understanding of where we currently stand as a sector, as employers, and as individual professionals in terms of AI adoption, use, and attitudes in the libraries and information profession.

The findings reveal that AI, as understood by most respondents, is largely synonymous with generative AI tools. While other forms of AI have potential to reshape library services, their development appears limited to a handful of institutions.

Individual librarians are experimenting creatively with generative AI, often to save time on routine tasks rather than to fundamentally transform services. Most institutions have either implemented or are drafting AI policies, though these are often at the broader organisational level rather than specific to library services. Still, the report characterises this as an “early days” moment for AI in libraries.

Ethical concerns are nearly universal, with questions of accuracy, reliability, legality, and environmental impact cited as key reasons for caution. Because many applications rely on external AI platforms beyond the control of libraries, respondents often approach the technology defensively. Limited resources and in-house expertise are typically also slowing adoption.

The most common activity that there is in the area of AI literacy, is training users to understand AI as an aspect of information literacy, rather than direct uses of AI services.

Resource and expertise limitations also seem to be inhibiting take up of AI. But there does appear to be some activity around use of AI for chatbots and knowledge discovery. Follow-up research could usefully explore these cases of useful applications.

Interestingly, fears about job displacement – often central to AI debates – were largely absent from survey responses. While no direct question addressed the issue, it did not surface in open-text comments, suggesting it is not a dominant concern among information professionals at present.

Overall, the report depicts a profession balancing curiosity with restraint. Ethical considerations and accuracy issues are the primary restraints on adoption currently, even as generative AI shows tangible benefits from the user perspective. The central question, Dr Cox suggests, is how libraries can present their cautious approach as a strength while remaining open to AI’s opportunities. 

Read the full report here.

The CILIP Data Protection Conference, on 21 May 2025, is an exciting new addition to the CILIP calendar. Presented in collaboration with Naomi Korn Associates, this conference is designed for all those responsible for managing information and personal data. Here Conference Chair Jess Pembroke looks at what to expect from the latest addition to CILIP’s training portfolio. Book now.

The CILIP Data Protection Conference offers an essential opportunity to stay up to date with GDPR, data privacy, compliance, and regulation while exploring broader topics in data governance, AI, cybersecurity, and children’s privacy.

"Data protection, including information and cyber security, is crucial because the consequences of getting it wrong can be severe..."

The conference has been launched in response to the increasing demand for expert guidance and professional development in the rapidly evolving landscape of data protection, AI, privacy, and governance and with the introduction of the Data Use and Access Bill, which is set to revise data protection legislation: staying informed is more critical than ever.

Library, information professionals and knowledge managers are often at the forefront of managing personal and identifiable information in their organisations. Keeping pace with these developments enables professionals to support innovative ways of managing, securing, and sharing data. It also positions them as advocates and leaders at the intersection of information management, technology and ethics.

Impact across the profession

Data protection, including information and cyber security, is crucial because the consequences of getting it wrong can be severe – not just for the organisations involved but also for the individuals whose data is compromised. A data breach can derail an organisation’s aims by damaging its reputation, eroding public trust, and causing significant financial and operational setbacks. Conversely, getting it right ensures legal compliance, adherence to professional standards, a recognition of ethical professionals and builds trust with users, customers, staff and suppliers.

For organisations, breaches, can lead to the loss of access to vital resources – including digital materials and services that users rely on daily – sometimes causing long-term disruption to operations and harming relationships with stakeholders, such as the public and partners who depend on the availability of secure, reliable services. This can have a ripple effect on the organisation’s ability to achieve its wider objectives.

Beyond organisational impacts, data breaches can deeply affect staff morale. When a breach occurs, library and information professionals, as well as wider teams, often face intense scrutiny and a surge in workload as they deal with the fallout. The loss or disruption of critical information services exacerbates the pressure, creating a stressful environment that can lead to frustration, burnout, or a decline in professional confidence.

However, the most devastating impact of data breaches can be felt by the data subjects themselves. For example, breaches that expose sensitive personal information – such as home addresses, medical records, or financial data – can put individuals at risk. These failures can have significant consequences for those involved, which can impact their daily lives. Data breaches impact on their privacy, and their trust in the services and organisations they rely on. In an increasingly digital society, the responsible management and protection of data is essential for our collective well-being.

Getting data protection right not only prevents such risks but also positions organisations as trustworthy custodians of information. It ensures uninterrupted access to resources, protects the rights of individuals, and reinforces public confidence in the organisation’s ability to safeguard sensitive data. For library and information professionals, strong data security practices support the wider aims of their organisations by upholding transparency, accountability, and ethical standards in managing information.

Adherence to responsibilities under the data protection legislation reinforces an organisation’s ability to achieve its goals. Effective data governance ensures that resources and services remain accessible, builds trust with users, and supports transparency and accountability. For information professionals, strong data ¬security practices highlight their leadership in safeguarding information, protecting the rights of users, and upholding the principles of access, equity and protecting intellectual freedom. This in turn contributes to an organisation’s wider aims, whether it’s providing public access to knowledge, supporting innovation, or delivering exceptional services.

Data protection is everyone’s issue. Senior leaders must grasp the strategic importance of data protection to mitigate risks and ensure compliance. Compliance specialists need to be adept at handling requests for copies of personal information and sharing when necessary. IT professionals need to be supported when tasked with implementing business critical, cutting-edge AI and other technologies which can elevate an organisation’s efficiency, whilst also complying with data security and privacy requirements.

The CILIP Data Protection Conference is an excellent opportunity to remain ahead of the curve in important developments in privacy law – supporting librarians, information professionals and knowledge managers with the CPD they require to feedback and support their colleagues and users with confidence.

The conference will feature several key presentations with more to be announced. Confirmed speakers include:

Dr Kit Good, Data Protection Manager at The Alan Turing Institute, who will ¬explore how AI technologies are reshaping data protection, focusing on governance frameworks and ethical considerations.

Sara Stock, Head of Data Protection and Information Management at the Driver and Vehicle Standards Agency (DVSA), will discuss the unique challenges of managing data protection in the public sector, including regulatory compliance and resources.

Sue White, Information Governance Manager at Naomi Korn Associates, will provide an update on recent changes in data protection legislation including changes from the Data (Use and Access) Bill, focusing on practical implications for organisations in the education and charitable sectors. This session will include information on important changes including an amendment which would make it easier for charities to contact supporters with unsolicited direct marketing messages after a supporter makes a financial donation. This is known as soft opt-in, an exemption to the rule that you need consent to send such messages (read a blog at https://tinyurl.com/NKsoftoptin).

Alongside these sessions, there will be shorter lightning talks and a panel discussion at the end for Q&A.

Delegates can expect to come away with enhanced professional knowledge and skills including an updated understanding of legislation and guidelines; a better understanding of how AI works and its impact on society and strategies for balancing information availability with security and privacy requirements.

Keep an eye out for updates on exciting speakers and topics to be covered, and register at cilip.org.uk/DataConf25.

First published in Information Professional, February 2025. Read more articles from the Information Professional archive online or by downloading the app on Google Play or Apple iTunes.

This video is hosted on the CILIP AI Hub

As a Knowledge and Information Management professional, have you ever assessed AI tools in the context of potential copyright infringement? How does current copyright and intellectual property legislation address AI issues? How do you protect third-party copyrighted works in your AI environment? Are existing copyright exceptions sufficient to implement text and data mining or machine learning?

CILIP K&IM Special Interest Group will explore these issues with specialists from different sectors in our new webinar.

Speakers

Harry Jennings, life sciences lawyer and Partner at law firm VWV, will provide us with current legislative updates on copyright and AI.

Jane Secker, copyright guru, well-known for ALT-CoOlSIG and LIS-Copyseek users, co-founder of Icepops and Senior Lecturer at City University, will discuss copyright and AI in higher education and research.

Susan Smith, NHS copyright specialist, NHS representative on Libraries and Archives Copyright Alliance (LACA) and Knowledge & Library Service Manager at Mid Cheshire Hospitals NHS Trust, will look at AI and copyright issues in the healthcare sector.

Emma Thwaites, Director of Global Policy and Corporate Affairs at the Open Data Institute (ODI) will be appearing at this year’s Rewired conference. Here she looks at the how a National Data Library could work and what is needed for it to be a success.

The idea of a National Data Library (NDL) emerged in the Labour Party Manifesto ahead of this year’s ­General Election. Since then, there’s been ­ongoing discussion about its shape and scope.

My organisation, the Open Data Institute, has contributed thoughts on the NDL to the AI Action Plan, reviewed research and written about the concept more broadly. The original vision from Labour for the NDL was to “bring together existing research programmes and help deliver data-driven public services, whilst maintaining strong safeguards and ensuring all of the public benefit”. This will only be possible if we create a well-thought-through piece of infrastructure. This involves ensuring access to high-value datasets, building infrastructure that the public can trust and ensuring that people have the right skills to use and maintain it. Many of the things that information professionals think about daily, no doubt.

As we’ve argued at the ODI, without data, there is no AI. As a country, we need well-structured and well-governed data to support AI stacks. To illustrate just how critical this point is, one study found that analysts typically spend 80 per cent of their time preparing data for AI use. To ensure the same challenge doesn’t blight the NDL, we must design it to be AI-ready from the outset.

The NDL will require high-quality datasets curated from existing public sector bodies, research organisations, and beyond. However, data often sits in legacy IT systems that don’t “talk” to each other and vary enormously in format and quality. For example, a great deal of data still only exists in PDF documents stored in rudimentary databases.

With these kinds of challenges, it’s tempting to try to design a comprehensive data architecture from the get-go, with accompanying detailed rules and reporting requirements. But it’s important not to stifle good practices where they exist or, indeed, to make the task ridiculously daunting for those who don’t already have high-quality data. It’s also critical to understand the cost of digital (and data) transformation when public finances are under pressure. While the government’s newly published Green (consultation) Paper, Invest 2035: The UK’s Modern Industrial Strategy, has a goal of “using public sector data as a driver of growth”, we must ensure that the approach to implementing the library takes account of the varying data maturity of public sector and research organisations whose data might come within its remit. In the ODI’s recommendations for the AI Action Plan, we suggest that the library initially focuses on three key areas: high-quality public data, federated Trusted Research Environments (TREs), and cultural heritage data. These datasets offer significant benefits and are well-structured for reliable public-sector deployment. Making them available to digital innovators could enhance public service delivery and stimulate the ecosystem necessary for sustainable impact. This would also give the NDL robust foundations on which to build. Regardless of the roadmap to building the library, resources and investment will be needed, just as they are required for building physical infrastructure.

In some ways, as our Senior Policy Advisor, Gavin Freegard, noted back in July the NDL can be likened to a traditional library. He cited a report by Onward proposing that “the Government should establish a British Library for data – a centralised, secure platform to collate high-quality data for scientists and start-ups.” Just as libraries store, organise, and provide access to knowledge, the NDL can curate and make data accessible to researchers, businesses, and public services. However, just as libraries have different access levels depending on the sensitivity of their materials, the NDL will need to offer tiered access to protect personal data. Data stewards will be critical in guiding users through this vast resource, ensuring the NDL becomes a trusted and effective driver of innovation and societal benefit.

Indeed, building and maintaining public trust will be a critical success factor in the project. The Lloyd’s Register Foundation’s World Risk Poll 2024 reveals that only 11 per cent of global citizens trust their governments to protect personal data, and the problem is particularly acute when it comes to personal or sensitive data, like benefits information or health records. The 2021 GPDPR campaign (https://tinyurl.com/odiNHSdata), where patient opt-outs spiked, shows the risks of ignoring public concerns. In building the National Data Library, we can avoid this by exploring the potential of new Privacy Enhancing Technologies (PETs), which include personal data stores, federated learning models and multi-party computation. These technologies, which the ODI is researching, offer ways to ensure data security and empower individuals to control data about them.

For example, in a Federated learning model, the model travels to the data rather than the data being ‘pooled’ by the user for the algorithm to be deployed. The model is trained locally on each device; the data never leaves its original location. This technology is already being successfully used in projects such as the EU’s MELLODDY and the secure healthcare research collaboration between Moorfields Eye Hospital and Bitfount, which aims to improve the early detection of eye diseases. Enabling algorithms to be trained across multiple local datasets without exchanging the underlying data presents great potential to unlock value from data that might traditionally have been kept closed.

With personal data stores, individuals control their personal data and decide how it can be used across the Web. There are several examples of this, including Solid. Solid is an open-source project, community and standard (or protocol), originated by the ODI’s co-founder Sir Tim Berners-Lee. Stewarded by the ODI since October 2024, Solid’s model is already proving effective. For example, in Flanders, a company called Inrupt (www.inrupt.com), has provided enterprise-grade Solid software for projects, including with the government of Flanders – through the Flanders’ Data Utility Company. This has given 6.8 million citizens their own Personal Online Data Stores (Pods) through which they can share data with government services, demonstrating the potential for trusted, transparent public sector data exchanges.

Multi-party computation (MPC) is a cryptographic protocol that enables individual stakeholders who hold sensitive data to pool it with others for joint computations without revealing the underlying data itself. Again, this offers a potential solution to some of the challenges around sharing sensitive public sector information via the NDL. Earlier this year, the ODI looked at how the Boston Women’s Workforce Council (BWWC) has partnered with Boston University to use an MPC to enable companies from the Greater Boston area to collectively compute the sum of their payroll data without revealing their individual contributions. The MPC protocol was produced to benchmark employers and produce a pay equity report covering the Greater Boston area. Introducing MPC technologies to the NDL could enable researchers to conduct similar analyses using UK public sector data.

A further significant consideration for the NDL is the data skills gap. AI tools can help the public sector and researchers work with complex datasets, but there’s an urgent need for digital and data skills training across all sectors. It would be a poor outcome for the NDL if we built it but did not provide people with the skills they need to use it. Multiverse recently found that new technology often goes under-utilised in organisations, and data is frequently mistrusted. Their Mapping the Data Skills Gap Intelligence Report 2024 revealed that while businesses invest substantial amounts in software, they aren’t matching that investment with skills training, leaving employees at a disadvantage.

Keeping up with the pace of technological change is hard, but it’s essential if the NDL is going to be able to reach its potential. Forrester’s 2023 Your data culture is in crisis (https://tinyurl.com/odiCulture) report indicated that 41 per cent of employees often mistrust the data available to them for decision-making, and according to Multiverse, as much as a third of the time spent by employees working with data is unproductive, with just under two-thirds reporting that they don’t even have basic Excel skills. In its design and inception, the NDL must encourage and develop data literacy, ensuring that those building it and using it can get the maximum possible value.

There is potential for the National Data Library – as an essential part of our national data infrastructure – to act as a showcase for responsible data practices while being stewarded and governed in the national interest. It can become a valuable resource for the public and private sectors and the UK’s research community. Using the country’s considerable expertise in cutting-edge technologies and well-designed data governance models, including those that allow for safe and ethical data sharing, we can make the UK’s NDL a global leader. As our Executive Chair, Sir Nigel Shadbolt has said, “With open minds, we can build platforms, services, and ways of working to realise the social and economic value in health, education, and other public data.”

Photo of Mia Ridge, Digital Curator at the British Library

Digital Curator at the British Library, Mia Ridge reflects on the ‘AI and the digital humanities’ session at CILIP Conference 2024.

This article is featured on the CILIP AI Hub

I was invited to chair a session on 'AI and the digital humanities' at CILIP's 2024 conference with Ciaran Talbot (Associate Director AI & Ideas Adoption, University of Manchester Library) and Glen Robson (IIIF Technical Co-ordinator, International Image Interoperability Framework Consortium). Here's a quick post with some reflections on themes in the presentations and the audience Q&A.

I presented a brief overview of some of the natural language processing (NLP) and computer vision methods in the Living with Machines project. That project and other work at the British Library showed that researchers can create innovative Digital Humanities methods and improve collections data with current AI / machine learning tools. But is there a gap between 'utilities' and 'cutting edge research' that AI can't (yet) fill for libraries?

AI opens up collections

AI (machine learning) makes library, museum and archive collections more accessible in two key ways. Firstly, more and better metadata and links across collections can make individual items more discoverable (e.g. identifying places mentioned in text; visual search to find similar images). Secondly, thinking of 'collections as data' and sharing datasets for research lets others find insights and inspiration.

Some of the value in AI might lie in the marketing power of the term - we've had the technical ability to view collections across silos for some time, but the institutional will might have lagged behind.

Breaking out of our silos

Identifying the real gaps that AI can meet is hard, cross-institutional work - you need to understand what time-consuming work could be automated with ML/AI. Ciaran's talk gave a sense of the collaborative, co-creative effort required to understand actual processes and real problems and devise ways to optimise them. An 'anarchy' phase might be part of that process, and a roadmap can help set a shared vision as you work out where AI tools will actually save time or just create more but different work.

Glen gave some great examples of how IIIF can help organisations and researchers, and how AI tools might work with IIIF collections. He highlighted the intellectual property questions that 'open access' collections being mined for AI models raises, and pointed people to HaveIBeenTrained to see if their collections have been scraped.

Keep learning and collaborating

I was struck by the delicate balance between maintaining trust and secure provenance while also supporting creative and playful uses of AI in collections. Labelling generative AI images and texts is vital. Detecting subtle errors and structural biases requires effort and expertise. As a sector, we need to keep learning, talking and collaborating to understand what generative AI means for users and collection holders.

The environmental impact

The first question from the audience was about the environmental impact of AI. I was able to say that our work-in-progress principles for AI at the British Library ask people to consider the environmental impact of AI (not just its carbon footprint, but also water usage and rare minerals mining) in balance with other questions of public value for proposed experiments and projects. Ciaran said that Manchester have appointed a sustainability manager, which is probably something we'll see more of in future. There was a question about what employers are looking for in library and informatics students; about where to go for information and inspiration about AI in libraries (AI4LAM is a good start); and about how to update people's perceptions of libraries and the skills of library professionals.

Thanks to everyone at CILIP for all the work they put into the conference, and the fantastic AV team working in the keynote room at the Birmingham Hilton Metropole.

The original version of this article appears on the British Library digital scholarship blog

External Links

IIIAF

Living with Machines

Have I been trained

British Library's Research Repository

AUSTRALIA’S Peter Miller, aka Scribbletronics. has been working with digital art for two decades. It’s a long way from his early days, tinkering with mathematical rules to generate procedural images, but Peter’s approach has evolved for the age of Midjourney, DALL-E, et al. Prompt-crafting today involves directing software to dip into its training database, using tags to identify and evoke concepts – what, say, does “worried” look like if we ask for a depiction of a worried man?

This article is featured on the AI Hub

"Sometimes fate is only a failure of imagination. The trajectory of automation is not only uncertain, it is unwritten."

“A lot of people forget that AI tools don’t have a spatial awareness like that of humans,” Peter explains. Instead of representing imaginary objects, the AI is exploring concepts.

“Ask for a dinosaur, it’s pretty straightforward – it looks for everything that’s tagged as a dinosaur and tries to make some kind of confluence of those things. If you ask it for Italian marble, it goes and looks at lots of images tagged as depicting Italian marble. The real power comes when you give it an instruction to make a dinosaur from Italian marble, and it looks at the intersection of the tagged materials, and does its very best to make a dinosaur that has the characteristics of marble. If you ask it for a worried dinosaur made of Italian marble in a supermarket, it may struggle because the intersecting concepts are that little bit more complex – but it’ll do pretty well.”

Peter acknowledges that AI training sets have appropriated work by other artists, representing something of a “poisoned chalice”, but connects it to wider histories around intellectual property and the sampling of music: “The question of appropriation was as significant then as it is for AI now, but most people were unaware of it because it affected only musicians.”

Where does that leave us ethically? Sci-fi author and social scientist Malka Older argues that “AI can serve as a cover for the choices of human authorities, slotting into a “role previously held by other amorphous concepts like ‘bureaucracy’” – a way of depersonalising decisions and avoiding accountability.

The late Joanna Russ, another science fiction writer, argued technology itself had become a guise for something else since the age of mass production:

“It is the entire social system that surrounds us; hence the sense of being at the mercy of an all-encompassing autonomous process that we cannot control. If you add the monster’s location in time (during and after the Industrial Revolution), I think you can see what is being discussed when most people say ‘technology.’ They are politically mystifying a much bigger monster – capitalism in its advanced, industrial phase.”

Carissa Véliz, a philosopher at Oxford’s Institute for Ethics in AI, argues against putting all the blame on capitalism, arguing that we should consider data and privacy “in terms of power, rather than money. Power isn’t just about wealth.”

She explains: “That connection between data and power is often obfuscated by those who benefit from it. Bertrand Russell had this insight that power behaves like energy, transforming from one kind to another. Enough economic power buys you political power; enough political power gets you military power; and so on. In the digital age, we have a new manifestation of power.”

Professor Véliz notes that we could have had a different kind of AI.

“Though machine learning is very impressive and has caused some truly exceptional advancements, it has limits and disadvantages. It’s not very good at identifying causation rather than correlation, for example. It can also leave us stuck in the past, as it typically uses historical data, perpetuating tendencies and biases we might prefer to get rid of. If we had more symbolic AI, or hybrid AI, we would need less data.”

The world could always be otherwise than it is. Sometimes fate is only a failure of imagination. The trajectory of automation is not only uncertain, it is unwritten. The fundamental question is less about technical capacity than ethics, authority, and accountability: Who will create the future – and with what ends in mind?

By Gary Horrocks

THE summer 2023 issue of UKeiG’s open access journal, eLucidate, featured my reflections on the implications of a Members’ Day presentation by Ken Chad on the “library technology ecosystem".

This article is featured on the CILIP AI Hub

"For every rally cry to embrace AI in education, there are concerns for student literacy and the demise of academic skills."

He has worked on trends in the library tech market for several years and his focus on this occasion was on the higher education sector. He recently delivered an equally thought-provoking seminar on the directions for library technology in FE, and in both cases I was struck by the complexities and contradictions that the profession faces navigating this challenging teaching and learning support landscape.

Ken notes there is “a dichotomy between the provision of library resources and learning resources.” It’s increasingly difficult to identify what constitutes a resource that is managed (and paid for) by the library and what is an educational resource paid for from a learning budget. Within this context, what future does a traditional library management system have in delivering an increasingly diverse and eclectic portfolio of learning resources when Google (and now ChatGPT or Google Gemini – previously known as Bard) dwarf conventional library systems?

Are the LMS we know and love dead in the water? Ken poses a challenging question as these learning systems and services assimilate. Who pays for the ecosystem? “LibTech” becomes “EdTech”. He argues that some current library vendors are mired in supporting legacy systems preventing the visionary transformative approach needed. “Only those that can deliver new higher value platform solutions will thrive in the longer term.”

Supporting excellence

The library and information services role in supporting teaching excellence and the student experience is integrating into a much wider learning paradigm. We can all list information innovations and disruptive technologies; the opportunities and possibilities are endless. AI and machine learning, virtual reality, open educational textbooks and open access resources, enhanced personalisation and student engagement using mobile apps, for example.

However, we’re experiencing a proliferation of mobile educational apps and learning tools at a time when parents are calling for a “smartphone-free childhood”. We’re all acutely aware of Brianna Ghey’s mother calling for social media safeguarding. Nearly a third of UK secondary pupils avoid school due to anxiety. The Covid-19 pandemic that helped create this situation also catalysed the transition to online learning and reliance on mobile technology. How do we get that balance right?

Concerns

For every rally cry to embrace AI in education, there are concerns for student literacy and the demise of academic skills. A recent survey reported that “more than half of UK undergraduates say they use AI to help with essays… One in four are using applications such as Google Gemini or ChatGPT to suggest topics and one in eight are using them to create content. Just five per cent admitted to copying and pasting unedited AI-generated text into their assessments.”

A US headline screamed “College Students Basically Don’t Know How To Read Anymore. This Is Grim” . This “marked decline” in “reading resilience” started before the pandemic but was significantly hot-housed by it; a reticence to read large volumes of prescribed text, partially down to the mobile tech culture and “smartphones becoming the main medium for reading.”

The e-book market still stumbles along riddled by prohibitive pricing, bundling reducing choice and unstable access. One study concluded that: “Reading print texts improves comprehension more than reading digital materials does…”, so perhaps we are barking up the wrong digital tree?

Potential

The data and learning analytics that underpin most of these EdTech developments offer huge potential to map student behaviour, tracing digital footprints to determine support needs but also to evidence the value and impact of library and information services on student achievement and success. However, privacy and cyber security issues loom if a student is unaware that their data is being processed, regardless of the purpose or outcome. “Libraries will need to navigate the ethical implications of collecting and managing user data while ensuring user privacy and data security,” says Ken.

And of course, the greatest contradiction of all, articulated by former CILIP Chief Executive Nick Poole in the January-February 2024 issue of Information Professional, is that the information and data age, the ‘fourth industrial revolution’, is an age of mis- and dis-information, where technologies like deepfake audio and voice cloning have the potential to influence millions.

On Safer Internet Day in February 2024 students listed “cyberbullying, mis-information and inadequate regulation of apps” as key concerns when it came to their online safety. Never have the opportunities to support teaching, learning and the student experience been greater, but the terrain is perilous.

CILIP has launched a new research report into the impact of AI, machine learning, automation and robotics on the information profession.

The independent research, authored by Dr Andrew Cox from Sheffield University, examines how these new technologies are impacting the daily work of healthcare information professionals or how they are likely to do so in the near future.

Nick Poole, CILIP CEO said, “This is a real call to arms for the information profession, our sector really is the best placed to maximise the benefits of AI, machine learning, automation and robotics for information users while mitigating the emerging risks.

The publication of this report is a milestone in the development of our professional community. There is a tremendous opportunity ahead for the information professions and thanks to this research we are well-placed to capitalise on it.

Working together, we can help our users look ahead with confidence to a near future that is being transformed by AI, machine learning, automation and robotics.”

The full research report, published by CILIP with the support of Health Education England, sets out a detailed and methodical analysis of the challenges and opportunities presented by this new generation of technologies. Full details are available here: https://www.cilip.org.uk/page/researchreport

Trevor Huddleston is Chief Analyst and Chief Scientific Adviser at the Department for Work and Pensions (DWP). He oversees the department’s analysis function, supporting wider decision-making and policy formulation. Here he speaks to Information Professional Editor Rob Green about how librarians and library services are now a central part of that function, and how the library service is supporting the wider needs of the Department.

Trevor Huddleston has been with the Department for Work and Pensions for 31 years, beginning his career as a graduate analyst, with an economics background. He has been closely involved with a number of key reports and development for the department – including leading the analysis for the Pensions Commission, Chaired by Adair Turner and which reported in 2005. He has since moved away from that direct involvement in analysis work to oversee the department’s team of analysts, becoming Chief Analyst in 2011. Chief Scientific Adviser was added to his role three years ago.

Policy development

This role is to lead the Department for Work and Pensions’ analysis function – ensuring the best evidence is available when it is needed in order to facilitate the best decisions to be made about policy. There are currently around 650 analysts in the department, and Trevor oversees this function – both as a direct manager for cross-departmental analysis (focusing on internal areas such as HR); and with ultimate responsibility for analysis work that supports policy development and implementation. It is this second strand of the analysis function that has the widest impact - informing policy teams and, ultimately ministers. The aim is to ensure decisions are evidence-based, and that that evidence is as reliable as possible.

Trevor recently incorporated the department’s library services more directly within the DWP’s Central Analysis and Science Directorate. The move is in recognition of the close synergies between the analysis function and the library service, as well as an understanding that better integration will lead to better outcomes.

Trevor says: “I am particularly interested in a subset of information, which we call evidence. My job, and the job of the analysis function, is to improve the quality and consistency of the decisions being taken by ministers. The focus is on ‘what are the decisions that people have to make?’ to change policy or change the way we organise services, or whatever it happens to be. “Once we understand what those decisions are, then we need to be aware of what evidence you would need to have access to in order to make an informed decision that is the area that my teams are working in.”

Profiting from synergies

There is little surprise then that this focus on evidence sees a clear link to library services and librarians. Analysts have always made use of the library service, but Trevor was keen to make the relationship closer. He says that a speech he gave at a conference on Informed Parliaments, Engaged Citizens, Effective Governments, organised by IFLA’s Government Libraries Section, helped to crystallise his views on this relationship.

“One of the things that I found important when I was researching for my speech was thinking about the location of the specific functions in an organisation. Organisational design can help or hinder the influence of different functions,” Trevor says. “The idea of bringing the library into its new home sitting alongside folk who are doing things which are not dissimilar – organisationally that is important for the visibility of the library. Also if you put two functions together like this you see lots of synergies emerge just by having them sitting together and in the same meetings – although not physically these days.” Under Trevor’s leadership, the DWP analysis function has mirrored that idea with teams working closely together. “We have a slightly unusual set up in the DWP in that we have analytical teams embedded in different areas – whether that is policy, finance or operational areas. That model of having them embedded in the function means they are intimately involved at the earliest stage.”

Fostering a similarly close relationship between the analysts and librarians is seen as an important part of ensuring a successful transition and future for the library service within the DWP’s Central Analysis and Science Directorate. Some of the synergies between the two teams are already clear, and it is easy to see why Trevor has pushed for a change that he believes will see more emerge.

Analysis for evidence

Trevor says: “We have professional analysts who have been recruited to generate evidence, or to use evidence that has been generated elsewhere.

“There is an external research programme that generates reports and that will look at external organisations which have expert areas – whether that’s the Institute of Fiscal Studies, the Bank of England, research functions in universities, or looking at what other governments are doing.

“Then we have a whole section of analysis that we do internally and this includes secondary analysis on data sets to generate new findings.

“This is quite a complex infrastructure of evidence being generated in different places, and what we have to do as the analysis function is to weigh up all of that evidence coming from these different places, and producing a consolidated piece of advice.”

Within that analysis function there is a clear role for library support. Trevor says: “The role of the literature review – either a formal literature review that might be published, or a less informal literature review – is absolutely vital. We want people to be on top of the evidence, and part of that is knowing what the key evidence is, as well as understanding what is less strong or less relevant. One of the things we have been looking at is trying to increase the capacity of the library team to help us with those literature reviews and evidence synthesis.”

Trevor realises that resources are important, saying: “In terms of ensuring success occurs, we have secured funding for the library – both in terms of academic journals and books, as well as a smaller increase in capacity of the library staff themselves to do more of those literature searches.”

Trevor says that the nature of the research and analysis function means that journals are a valuable additional resource. The majority of data used to support policy- and decision-making within government departments is administrative data that is created internally and not widely available outside of the department. The fact that the department generates much of its own data, means that books can be limited in their usefulness as part of the policy development framework. Instead, journal articles, particularly those that are associated with Government research programmes, can supplement that internal data analysis and provide additional context.

Professional Development

Increasing access to journals has a clear impact on the analysis function, but Trevor points out that the library is also there to support the wider department. He says: “This is not just about evidence decisions – there is knowledge and information that you need to do your job better, and I am talking about the wider Civil Service.”

This puts the library service squarely in the realms of facilitating continuing professional development (CPD), through collection building to support access to the kinds of books people need to learn new techniques, master skills or keep informed about the latest developments in their professional field.

“Any book, paper, or information that helps the Civil Service become more effective has to be part of the library’s role,” says Trevor. “We’re talking about things like communication skills, negotiating skills, listening skills that are common to civil servants wherever they are working.”

Those cross-departmental skills within the Civil Service are an established part of how the service works. Common standards mean that people can move between departments more easily. Managers will know what they are getting and those moving do not need to learn new systems. This consistency and fluidity delivers benefits for the individual as well as the organisation, and Trevor says that professionalism and continuing professional development underpin that. Recognising the role of the library in supporting CPD to the department underlines the importance of a well-resourced library service.

There are around 80,000 staff across the DWP – many of these are frontline workers and these need support as well as those central staff members. Pre-Covid the library held a series of pop-up events, and these have been successfully replicated in a digital environment. Books can be sent out physically by post and returned to the London-based library at Caxton House. The department’s central function is also split across six regional sites, with more than half of analysts based outside of London. This dispersed nature of DWP presents issues for the library. There is a distinction between the physical experience and the online experience and the key is making sure everyone gets the same level of service – however they access it. The library team’s experience with a de-centralised workforce has meant they were ahead of the curve when the pandemic hit and forced many people to work from home.

The value of accreditation

The Civil Service is built on professionalism and the Central Analysis and Science Directorate is no different – as Trevor points out, professionalism is about standards. The data analysts will typically have a minimum of a first degree in a numerate subject, but most will also hold a higher degree. Out of around 650 analysts, some 100 will have a PhD and a further 250 will have Masters level degree.

Trevor says: “Having an accredited professional is really important for a number of reasons – and that can be an economist, another type of analyst or a librarian. Firstly, you want to know what you are getting and accreditation is critical in assuring you that you are going to get the quality and the skills that you need.

“We encourage our analysts to target 100 hours per year of CPD. This might be reading relevant papers or attending seminars, because in every discipline things move on. It is the same in the library service and we want librarians to be aware of the changes that are taking place in their sector – whether that is in technology or how services can be delivered.”

Again, this commitment to professionalism and CPD is not just about improving the organisation. Trevor says: “Professional accreditation is important to both the quality of the department, and professional development because it allows you to move around. If someone is seeking promotion, we will always ask them about their CPD over the last 12 months – what they have done, what they have learnt from it and what difference has it made.”

The importance of keeping up to date professionally is highlighted by changes in the analysis function over the three decades since Trevor joined the DWP. Key to that is how access to data has been transformed.

“Evidence-based policy-making goes back decades in the UK. We are excellent at it, and certainly compared to some countries we are very, very mature in the use of evidence and the generation of evidence,” says Trevor. “The big difference now is the speed at which evidence can be generated – the ‘real-timeness’ of evidence that you can use – has changed. If I look back to the 1990s, if I wanted to look at trends in say incapacity benefits, I would need to look at printed tables and enter them into spreadsheets. Once a year you would get the annual Social Security stats publications – but you would be using data that could be 12-18 months out of date.

“Even when we first got access to administrative data electronically – it still took about 12 months for that data to be loaded-up and accessible. Now, if you are looking at data for people on Universal Credit, for instance (which is of particular interest in the pandemic), then that data is only 48 or 24-hours old – in some cases it is almost real time.

“The availability of data further increases the demand for evidence because the technology opens up more opportunities. Speed to market of data has been a big change.”

Additional data, particularly real-time data, has enabled the analysis teams to move beyond a role that is focused purely on policy development. Analysis now has an increased role in the delivery and implementation of policies, where data can indicate how well things are working and if policy needs to be adapted.

While (near) real-time access to data is having a positive impact on the work of the Civil Service, there is still a need for context – this could be how current policy sits within long-term trends, or comparing outcomes to previous policies. This is another area where the library has a central role, through the preservation and organisation of corporate memory. The synergies continue to present themselves.

Showing your worth

Trevor says that it is important to make others aware of the skills and value of the team – at all levels of the organisation. He says: “I report into the Permanent Secretary and he is aware that this is something I have been pushing hard on, and he has been receptive to it. A really important step for any function is to go out and talk to senior people to establish what their user need is – both for that individual, and for their teams.

“So you have to ask ‘what is the need and how can you meet that need?’. Part of that is making people aware of what functions you can fulfil. I have used many libraries over the years, but I haven’t necessarily thought about what they can do for me. So, it is important for a service to spell that out to its users and show them how it can meet their needs. You need to be having those conversations across the department and with your users to a) let them know what is available, and b) to let them know what could be available.

“Then you have to think about making sure you deliver – and that’s about having clear objectives and also the resources to fulfil those objectives. All this means that organisationally the library is going to be more visible than it has been.” The journey to closer align the department’s library and analysis function began with Trevor speaking at an IFLA conference, and it is no coincidence that the library service has now become an organisational member of IFLA. It brings an international perspective that can often be overlooked, as Trevor points out: “[The conference] showed me was how important it is to have relationships with libraries that operate in different languages. We will naturally work with an evidence base that we can read in English – so we will look at what the Americans, Australians, or New Zealanders are doing but there is a vast literature that might be relevant to policy in the UK that might be written in a different language. Part of my motivation is to try to make sure we develop good relationships with the relevant libraries in other countries so that they can point us to stuff that we ought to be mindful of – I see IFLA membership as a real opportunity to access that information. It will also help us to understand best practice elsewhere and to provide examples of best practice that others will find useful.”

Interview arranged by Anoja Fernando, Research Library Manager, Department for Work and Pensions.

CILIP believes that we are living and working in the early days of an economy and a society that are being transformed by data. If appropriately managed by ethical professionals, this transformation creates extraordinary opportunities to improve people’s daily lives, their health and wellbeing and the services on which they depend.

We therefore welcome the Government’s initiative to develop a comprehensive and far-reaching National Data Strategy (NDS) in consultation with industry partners, including CILIP.

We believe that there is a clear line of continuity from the existing role of information professionals as ethical stewards of information-based services to a future role as accountable data stewards on behalf of the communities we serve.

We are pleased to publish our response to the National Data Strategy Consultation, developed in discussion with our professional community and following an open round-table of industry experts in November 2020.

Download the CILIP NDS Consultation response

Key points

• Data has the potential to be a tremendous force for positive social and economic development
• Information professionals are ideally positioned to help unlock this potential
• Any Government policy must promote the interests of empowered, data-literate citizens
• We call on Government to invest in data and information literacy alongside digital skills
• The implementation of the NDS must reinforce and protect the data rights of the individual
• We encourage HM Government to promote the adoption of Open Data across all industry sectors
• We support calls to embed a ‘data culture’ into the delivery and improvement of public services

Proposed programmes

We have used our response to the National Data Strategy consultation as an opportunity to propose a number of key programmes to Government which will help create the conditions for our future data economy:

In the years ahead, CILIP will actively be exploring our role as a professional association for information professionals including people working in data and informatics. If you would like to find out more, or be involved in these developments, contact nick.poole@cilip.org.uk.

Understanding digital archiving

Crown Records Management have shared content from their Vault Podcast Series, and we'll be publishing more content in the coming weeks. This episode episode sees Crown's digital transformation sales lead interview Commercial Account Director from Preservica Nathan Vooght.

You can listen now to a podcast version here.

Nathan: Digital archiving is really new area of IT and its only really in the last decade or so that it’s become an established requirement in its own right. This tails back to the national archives who ten years ago came to us with a project. They realised that they had a lot of content and wanted to make sure that they could avoid file obsolescence. They needed a way of transitioning, taking these old file formats and really making sure that they’re constantly preserved for the future.

Nathan: So recently we worked with Transport for London (TFL), they needed to store content on a number of pieces of equipment. So something that hundreds of thousands – if not millions – of people use on a daily basis is their escalators. The blue prints and health and safety documents need to be stored for over sixty years, as that’s the lifespan of an escalator in London. Being able to put this information into Preservica, make sure that its constantly revalidated and is fully accessible in the future is absolutely critical. It may not need maintenance for twenty years and by this point the original engineers could no longer be working for TFL. So you need to make sure the manuals, all the work around it, right down to the geological surveys are preserved and can be accessed at any point as well.

Archiving new technology

Nathan: When it comes to new forms of communication, these are areas we are moving into and are looking at ways to preserve. Historically the main requirement has been around traditional content and by traditional I mean business documents, Jpegs and PDFs. The recognised formats like TIF right the way through to AV files, so MP4, MP5 we are also looking at. Content harvesting is the really interesting area of how preservation will adapt and evolve over time. This is for instance if you look at a great example of Twitter. There are very important, very senior, incredibly powerful people who love Twitter, maybe love it a bit too much! Essentially there are regional legislations which are in place so you have to preserve online communications. You can’t delete it and so being able to harvest from Twitter and keep that communication is important. If you’re in the public eye you’re never essentially off the clock. I wouldn’t say the whole twitter harvesting thing is absolutely critical at the moment. When you look at different content management systems, how do you ingest from different document management systems already have in place? How do you pull that information into Preservica so it becomes more of a seamless process?

Currently digital archiving is a man managed process but if we’re looking at compliance and criticality you need to automate that. You need to make sure that it is as slick and quick as possible without too much intervention.

And so then the archivist or the records manager is curating and their taken care of. So their role becomes much more business value orientated rather than business critical.

The evolution of the archive manager

Kevin: Is this a parallel to the evolution of the role of the records manager, with new young born digital archive managers coming into the organisation?

Nathan: I would definitely agree with that. I think archivists as a group are going through an evolution. Working within document automation or document capture there’s a predisposed idea that they don’t have much exposure outside of that. There’s a perception that their value isn’t necessarily recognised even within large organisations. However, I think that is changing hugely, especially around the newer ideas coming into archival digital archivists and GXPR archivists. These are people not afraid of IT, not afraid of new technology and they’re coming in and shaking things up.

They’re coming in and saying ‘well look these solutions exist now and we need to get them in place. This isn’t something that we can wait for a decade we should already be dealing with this’.

That is helping massively with building the profile of an archivist and how critical their role is to an organisation. Likewise the actual role of archivists as you said is evolving, that role is bleeding into records management now. There is a blending of roles happening. You do come across more and more people who have a hybrid role. They’re now not only dealing with the immediate one to seven year records, records retention and disposition of those. They’re also answering the question ‘what do we do with anything that has a lifetime over ten years?’. Look at mortgage documents or life insurance policies, anything around financial communications, this needs to be stored. So they’re having to learn from both sides. Its a really exciting time. The more businesses recognising the value of an archivist and digital archiving, the greater their profile. More and more jobs are being advertised for this and more and more archives themselves are being started within an organisation. They’re no longer outsourcing, no longer just sending dump files of content to the national archives. There’s also more saying ‘we should be doing this ourselves’ and actually gaining the value from it, while also protecting themselves in the long term.

2019 buzzwords and trends

Kevin: So just finally, we touched on many different areas of digital transformation and digital archiving. GDPR has been a big buzzword certainly this year, what can we look forward to in 2019? Around the work that you do, what is going to be the big buzzwords and trends?

Nathan: I definitely agree that GDPR has been done to death but I don’t think it’s going away anytime soon! We’ve now transitioned to a period where we’re seeing the results of GDPR. The Information Commissioner’s Office (ICO) is now releasing figures on how many people they have taken to court. How many people they’ve warned and how many organisations they’ve supported. They’re not the big bad guys in this scenario, it’s much more about supporting records managers and archivists. Their role is educating, so now we’re actually going to be looking at what that means in the real world. There are organisations which are quite nefarious in their methods who just don’t care about GDPR. They will be found out and dealt with.

On a global scale because an organisation deals with Europe or the UK there’s still going to be more around the immediate impacts.

Likewise you see many regions which are now taking on the aspects of GDPR. Creating their own legislation based off of the GDPR principles and the articles within. In terms of buzzwords, I tend not to look too far out. I’m not technical so my understanding is much more of a laments understanding but ideas around robotic process automation is really interesting. You have companies that have digitised all of their documents, some of them are now paper free. They’re now looking at how they can automate quite complex human processes using mirrored process automation. How that relates in to AI and actually mimicking the human to teach the robot, that for me is really interesting. We are looking at these areas as well and seeing what lessons can be learned from the technology. This is further out from the adoption cycle but I think those two at least are the main ones. Then we’ll see what happens around GDPR and what the actual results are. I think everyone is now waiting to really see what the fall out was. Whether it was – I don’t want to go too far down it – was it Y2K or did it actually have a genuine impact.

Kevin: Watch this space! Nathan Vooght thanks very much

Nathan: Thank you.

Banner image: Northern Line Extension: Lowering of the tunnel boring machines. Copyright John Zammit

AI Ethics: “When the robots do take over, at least they could be nice.”

Kriti Sharma (@sharma_kriti), AI technologist, business executive and humanitarian, gave the opening keynote at CILIP Conference 2019 with a five point guide to keeping your future AI projects ethical.

After running through the many ethical problems AI has run into in areas ranging from face recognition to recruitment, Kriti said anyone looking at AI would do well to check:

1.That AI should reflect the diversity of the users: is it working for everyone? Who is being left behind?

2. Is AI being held to account? Don’t let AI do things you wouldn’t let a human do. Whether it’s discriminating against women in a hiring process or discussing how self-driving cars can be made accountable.

3. Transparency: Show what’s working. AI can be very complicated and we need to make them accessible. But also users need to know simple things, early on, like are they talking to a human or a machine.

4. Positive: What are you using it for? One of the biggest uses of AI in the real world is helping people click more ads. Or driving digital addiction. Are you doing something better? Kriti is a founder of AI For Good.

5. Jobs: AI is going to replace some jobs and create new ones. Need to see the impact and understand what effect it’s going to have.

On this last point Kriti gave an example of AI designed to help a customer support queries. It aimed to take over 80% of the mundane work, leaving humans to work on thee 20% complex problems. However a survey showed that workers saw the change as “My job used to be 80% easy, 20% difficult. Now it’s 100% difficult.”

Kriti recommended employers looking at upskilling and working policy alongside AI projects that might fundamentally change working practice.

In her talk Kriti also said that if we can get the ethics in place “When the robots do take over, at least they could be nice!”

Emerging Technologies: The Library of Everything

With limited space on shelves and hard drives alike, librarians can spend significant amounts of time “weeding” – archiving or disposing of unpopular books, journals and data. What if we didn’t have to do that? What if we could utilise something like Jorge Luis Borges’ Library of Babel (https://bit.ly/2ARcjrL) in real life – a (practically) infinite library. Surprisingly, there are teams of researchers around the world working to make this a reality.

In 2012, genomics maven George Church first demonstrated that DNA could be used to store data,1 and since then a number of companies have been working to create a commercially viable product. Microsoft sees DNA storage as a future part of its Azure cloud services,2 and has recently announced significant advances in automation of DNA-based data storage and retrieval. DNA is synthesised artificially by combining the four amino acids adenine (A), cytosine (C), guanine (G), and thymine (T), in a similar way to the binary code that underlies today’s computers.

This sounds like it could be expensive and cumbersome, and indeed today it is – but that will change. And the rewards for the first team to make the process practical are potentially enormous. In 2017, a team from Columbia University and the New York Genome Center managed to cram 215 petabytes of data into a single gram of DNA.3 That’s around 250,000 times as much data as will fit on to a typical 1 terabyte PC hard drive, or the equivalent of about 900 billion pages of text.

But we are habituated to a rationing mindset, based on drive capacity and shelving space for collections, so could we actually cope with the idea that we might never need to throw anything away? It’s one of the most profound changes imaginable and goes against all our curatorial instincts. Perhaps “hoarding” data and information will no longer be seen as slightly aberrant behaviour. Instead, perhaps complete data retention – for everyone, all the time – will become the norm.

When we move beyond the specialised domains of research and education, it becomes clear that being able to keep everything forever might not always be welcome. We’re starting to see the chilling effects of being followed around for life by youthful hijinks exposed on social media, and how risks for vulnerable or marginalised groups can be amplified by the internet. But many of today’s blockchain start-ups are positively excited about a world where nothing is ever forgotten, and DNA-based data storage could be the technology to make this practical.

Perhaps the emergence of the digital vegan4 tells us everything we need to know about the direction of travel of privacy and data rights. Will a generation that has been online since their first ultrasound increasingly reject pervasive screens, data and internet-connected devices – choosing instead to carefully curate the information they share about themselves? Arguably it’s what we should expect from kids who have grown up with Instagram and Snapchat filters, and the parents and grandparents among us should be proud that they are seizing control of their own narrative. The kids are alright!

So, are we all set to toss hard drives and USB sticks into the recycling bin? Not quite; the cost of DNA sequencing has fallen exponentially over the last 10 years as the technology has been commoditised, at the same time as the speed of sequencing has increased. This is great for getting the data back from the DNA, but it is still slow and expensive to put that data there in the first place. So, don’t get rid of your library collections or data centres just yet, but do plan for a future where data will be cheaper and more abundant, and start thinking about the implications of finding needles in haystacks.

1 Service, R. F. “DNA could store all of the world’s data in one room”. Science 02/03/17. https://bit.ly/2RJy8Pw

2 With a “hello” Microsoft and UW demonstrate first fully automated DNA data storage. Microsoft, 21/03/19. https://bit.ly/2TlJ4Uh

3 Cornish, C. “How DNA could store all the world’s data in a semi-trailer” FT, 05/02/18. https://on.ft.com/2nIF838

4 Hill, K. “I Cut The ‘Big Five’ Tech Giants From My Life. It Was Hell.” Gizmodo 08/02/19. https://bit.ly/2Ue05VY

“I’m a trained computer scientist and all my degrees and all my jobs have been in computing. But I went through my entire education process and much of my career without ever hearing about ethics, even though technology impacts society in so many ways,” says Kriti Sharma, a United Nations Young Leaders For Sustainable Development Goals. Kriti, who is also the founder of AI for Good, a social enterprise that uses artificial intelligence to tackle social issues, will be delivering a keynote speech at CILIP Conference 2019.

I’ve had to re-educate myself. I’ve had to really understand what non-geek speak looks like. And I think it’s a responsibility of the technology industry to be more open and explain what we do a lot better.”

Powerful as it is, one of the problems is that artificial intelligence is not aimed at the worthiest of the world’s problems. “Today, if you look at the very successful AI applications at scale they are in the field of making people click more ads. There’s a lot about driving digital addiction, where content is designed to get your attention. And it’s making people buy more products saying: ‘If you’re interested in this then maybe you would like this’ and ‘people like you bought this’.

“But there’s an opportunity to solve social justice issues using this technology. That’s where I would love to see more action going forward, beyond these fields, to see society and the technology industry mature to a point where we would focus on wider problems. This technology could be used to drive behaviour change, changing our pathways to the information that we are exposed to. This combination of behavioural science and machine learning is a real example of knowledge being power. So, if you have that sort of data, it’s critical to create ethical frameworks around it. And I don’t want to look back in three years and think that the best technology of our time was used to make people click more ads.”

“A big part of the problem is that the people who create this technology lack diversity. That manifests itself in the technology and also the data.”

One manifestation is how male and female personas are given to particular AI assistants.

“A lot of these assistants are given female personalities if they’re doing things like ordering the shopping or playing your favourite music – Siri or Alexa – rather than Ross the robot lawyer, where a male persona is making important decisions.”

It’s a pervasive problem where children shout demands at female bots and listen to male ones. And then the problem goes deeper.

“Let’s say, if you were building something like Siri and you trained it on Wikipedia to build its knowledge. If you do that, it’s going to learn that only 17 per cent of notable people’s biographies are of women.

“The point is that if the data is skewed, the machine does not have a better way of knowing it. Another example is the MIT study of facial recognition systems where there is a 0.8 per cent error rate for light-skinned men, and a 34.7 per cent error rate for darker skinned women. They failed to recognise Michelle Obama and Oprah Winfrey. It’s a big issue because this technology is being used in policing and criminal justice. So there are a lot of biases because AI is trained on a certain kind of data set.”

Misleading datasets and poor representation can produce misguided artificial intelligence. It means the positive potential of AI is offset, even negated by its ability to amplify old prejudices. AI For Good, an organisation Kriti founded, runs projects where machines are created to provide friendlier, less judgemental sources of information than their human counterparts.

“In one of my projects in India we are working on sexual and reproductive health information for young people. We’re using AI to provide access to the right information to the right people at the right time. Young people historically have struggled to get access to vital information about it. It’s very awkward and sometimes socially unacceptable to access this information but algorithms, designed with the right controls and the right experts in the room, with a safety first approach, can bridge that gap between the young and information.

“Another very interesting example is a bot we launched in November, called rAInbow, to help victims of domestic violence in South Africa or those at risk of it. Historically victims would have to call a helpline and talk to a human.

“There were major issues about things like stigma and judgement and victim blaming and helplines not being open all the time – the fact that victims or survivors wanted to take action at their own pace in their own time. So we built this non-judgemental machine that was there to give them access to the information and designed to be empathetic. It doesn’t have empathy, but it is designed to be empathetic, and the result is that we had over 150,000 conversations in 90 days from launch and it’s really working as a system which the people suffering actually want rather than something that is given to them. It’s not a replacement to human advisers, but is there to augment their capabilities.”

The evidence suggests that engagement with people and issues beyond technology is not just a nice idea, it is a necessity. “Let me be very clear about it”, Kriti says. “The answer to solving any technical or business problem is not just throwing a bunch of data scientists into a room with clever degrees and PhDs. You need to work with domain experts and that’s where a big opportunity for re-skilling the work force lies, and it’s quite often overlooked. If you look at the next revolution and how the jobs will change, often they say ‘oh we need to create more data scientists’ – yes we need to do that, but I don’t think that’s the only answer. There’s a huge opportunity to re-skill and improve collaboration because we need that domain expertise. It’s not as easy to train a machine learning system without that domain knowledge.”

Having said that, she believes that professionals should brace themselves for the arrival of data scientists. “What Google did to information and search was very exclusive technology at the time, very expensive to build. What we’re seeing now is an extrapolation of that, a democratisation of it. With advancements in technology this is all more easily available now and AI is already being used in many ways in our daily lives, and in our business lives.

“Some companies are starting to release their ethical AI frameworks on how to deal with this but I think every company should have chief ethics officer roles in place. A person who is not a lawyer, not a compliance professional, but someone who works closely with the people creating the technology, and also anthropologists and social scientists, to be able to find the optimal solution. It’s not just a technical problem.”

CILIP Conference takes place in Manchester on 3 & 4 July. Join Kriti and other keynotes, Liz Jolly, Patrick Lambe, Hong-Anh Nguyes and Aat Vos at this year's conference.

Full information on this year's programme and booking details are available at the CILIP Conference 2019 website.

Book now for early bird deals

The reference interview - or consultation - is a well established technique used by librarians and information professionals. This helps establish the issues that a researcher wishes to cover. More fundamentally it also defines the kind of relationship a researcher will have with those offering advice. Now that the use and management of research data is so important to academics it has become an important potential element of the reference interview. This extract from The Data Librarian's Handbook shows how it can be used to bridge traditional librarianship and the expertise required by the data librarian.

Established researchers often know what datasets are available in their field of study, or at least the main sources and providers from which to seek them. They are immersed in the literature and knowledgeable about the activities of their peers, they know the institutes and principal investigators that are producing research relevant to their work. This is usually not the case with postgraduate students, even less so with undergraduate students, and nor is it the case with researchers exploring the boundaries of their disciplines or doing cross-disciplinary research. 

Fortunately for data librarians, it is possible to become knowledgeable about sourcing datasets in a given field without being an expert in that field. How do data librarians match users to the data required?  – through a reference consultation or interview.

Since every research question is unique (unlike every classroom assignment), it is easy to feel daunted when confronted by a new one. These are some helpful hints to guide you through a difficult reference interview:

1. Buy yourself time by asking more questions before trying to come up with a source; avoid making assumptions about the user’s requirements, prior knowledge or viewpoint.
2. Find out if the user is basing their query on a published article; ask for the citation or a copy to help you with the context. If a student, ask who is their teacher or supervisor.
3. Ask the user to explain acronyms and jargon they use in their language; you do not have to pretend to be an expert in their area of study to help them.
4. Take notes and write down key phrases as the user speaks (if meeting in person or on the telephone).
5. Even if you are unable to find the perfect source for your user, you can probably give them some useful starting points for their search, based on your knowledge of data sources, or that of your peers.
6. Do not be afraid to take time to think, search and consult others; always take the user’s e-mail address for future contact or to follow up.
7. If you remain stumped, resort to asking others: immediate colleagues, peers at other institutions, government statistical agencies, data providers and publishers. Once you have done your homework and ruled out obvious contenders you can also post to a library mailing list or the member list of IASSIST.
8. If the query is about using a dataset rather than finding one, take time to read the documentation, try out the interface yourself or reproduce the problem before turning to others for help.
9. If the user does not voluntarily let you know their query has been satisfied, follow up in a reasonable amount of time to see if you can offer further assistance.

Interest in data has been growing in recent years. Support for this peculiar class of digital information – its use, preservation and curation, and how to support researchers’ production and consumption of it in ever greater volumes to create new knowledge, is needed more than ever.  Many librarians and information professionals are finding their working life is pulling them toward data support or research data management but lack the skills required. The Data Librarian’s Handbook, written by two data librarians with over 30 years’ combined experience, unpicks the everyday role of the data librarian and offers practical guidance on how to collect, curate and crunch data for economic, social and scientific purposes.

About the author

John Southall is Data Librarian for the Bodleian Libraries at the University of Oxford. He is based in the Social Science Library and is subject consultant for Economics, Sociology and Social Policy & Intervention.

Robin Rice is Data Librarian at EDINA and Data Library, an organisation providing data services for research and education based in Information Services at the University of Edinburgh.

The relatively new exception to copyright law that we enjoy in the UK, permitting text and data mining (TDM) for the purposes of non-commercial research, offers potential to further knowledge and make scientific and medical breakthroughs. Importantly, the new exception states that any contractual clause which purports to restrict this exception is automatically null and void. Librarians who manage electronic resources and datasets can assist researchers greatly. However, in order to do this they need a robust understanding of the law and to be assertive in their ­relationships with publishers when negotiating or interpreting licence agreements. This article examines the ways in which librarians can facilitate the work of researchers who want to use TDM. It also argues that librarians need to encourage researchers to exploit the new copyright exceptions as key partners in the research process.

Walls come tumbling down

As Seth Godin told us back in 2011, librarians are no longer the gatekeepers to knowledge, they are gate-openers; the walls have come tumbling down and researchers can get access to vast amounts of data and knowledge without ever needing to step into a library. However, as librarians we are well aware that open access and data sharing do not currently deliver all the resources an academic might require to undertake research in almost all disciplines. Although debate continues to rage over how to optimise and sustain systems of scholarly communication, advocates for libraries and research successfully argued for changes to the law to allow researchers to make full use of information they have already paid for access to. In 2014, amendments to UK copyright legislation following the Hargreaves Review of intellectual property law, removed legal barriers for researchers who wish to perform text and data mining (TDM) for non-commercial research purposes, providing they have legitimate access to the materials they wish to mine. This change holds the promise of being able to answer entirely new research questions that were previously not amenable to enquiry. 

Exploiting copyright exceptions requires librarians to be bold about the advice they give researchers, to encourage them to use the facilities such as TDM. This may at times mean challenging or ultimately contravening the terms and conditions of a licence with an electronic resource supplier. However, it is possible to do this responsibly through considered institutional approaches and working with colleagues to share best practice. 

Data sharing: panacea or holy grail 

The concept of data sharing is gaining a lot of attention currently as being a way of potentially offering solutions to many of the world’s problems, from transport to healthcare. Data sharing is strongly linked to, but not synonymous with, the concept of open data which is defined as data or ­content that can be 'freely used, modified, and shared by anyone for any purpose’. It is not always possible for all data to be open for ethical, practical or commercial reasons, but the principles behind data sharing are to encourage re-use of research outputs as widely as possible. Just as we increasingly have open access to publicly-funded research outputs, advocates of the open data movement such as Nigel Shadbolt, who keynoted at the CILIP 2016 conference, believe that open data will eventually become the norm. However, many researchers recognise that much of the world’s data is not open or available for sharing, but owned by commercial organisations and kept behind paywalls. Licensing and copyright are potentially restricting our ability to use data in new and interesting ways which is why the text and data mining exception is so important. 

Content Mine, is a project led by Peter Murray-Rust that is doing a lot of work to highlight the TDM issue. They strongly advocate that publishers who restrict researcher’s ability to mine through technical protection measures should be challenged. Content Mine have been leading the way with a number of TDM projects, for example a project to mine the research literature for information about the Zika virus. They offer software, which allows you to interrogate the research literature and process it to search for keywords or phrases. They also run workshops and training which include more information about copyright law and the TDM exception.

What might researchers want to mine?

TDM is generally performed on corpora of text or data in electronic form. For example, a body of chemistry research literature might be mined by running TDM software on it to discover then create a database of molecular structures, a valuable resource for molecular chemists, crystallographers and other scientists. Another example is the 'Robots Reading Vogue' project at Yale which uses the ProQuest licensed archives of Vogue magazine to provide fascinating insights into the changing use of imagery and language in fashion throughout the 20th century. 

What problems might researchers encounter?

There are various problems associated with TDM where researchers might call on librarians for advice. These include:

• Reaching a download limit (often arbitrary and relating to contractual stipulations made by the publisher) for papers from a particular publisher or database, at which point access to a resource might be cut off.
• Being served with an ‘unusual behaviour’ report from publishers, for example when systematically downloading large amounts of material.
• Encountering Digital Rights Management (DRM) technologies that prevents them using TDM methods, which then require a request for permission to have the DRM removed. 

In the first instance, librarians can deal with these issues on a case-by-case basis, but as the library builds up more expertise in this area they might consider writing FAQs or  developing toolkits to assist researchers.

How else can librarians help?

There are a number of additional things that librarians can do to assist with TDM, including:

• Highlighting the relevance and application of the TDM exception as part of any copyright training they offer to researchers and advertising TDM support as a service. This could be promoted alongside related research support services. 
• Encouraging the development of partnerships with academic colleagues to work on TDM projects, through highlighting institutional collections that might be suitable for mining.
• Being clear about licensing and terms & conditions under which resources are made available to colleagues at your institution, and ensuring they fully understand the copyright exception that permits TDM for non-commercial research regardless of contractual stipulations.
• Being firm with publishers and other content suppliers in the case of ‘unusual behaviour’ reports or DRM blocks to protect the legitimate interests of researchers, as defined by law.
• Advising on next steps should local support not be enough, up to and including appealing to the Intellectual Property Office (IPO) if a rights-holder will not remove DRM locks. Information on making a complaint can be found on the IPO’s website. 
• Being aware of data protection issues: TDM is not exempted from data protection law, so content identifying a living individual must be processed in compliance with the Data Protection Act (1998).
• Being mindful of clauses in any new licence agreements for resources that might restrict TDM activities. 

TDM, copyright risk and anxiety

Librarians understandably might feel ­anxious about sanctioning or undertaking action that is legally ambiguous. The nature of UK copyright exceptions is that they are defences to accusations of infringement rather than rights, and this puts the onus of responsibility on the person doing or facilitating the copying to ensure it is legal. Similarly, the hard-won changes to the law that mean that contracts cannot override exceptions are fundamental to equal access to information in today’s digitally connected world. However, these require reviewing a legal contract drawn up by a well-funded commercial organisation and deciding to ignore parts of it.

It is essential that information professionals are supported by senior managers to take a measured, yet assertive approach to use of TDM. This requires a mature attitude to risk, which balances the institution’s reputation and continuity of its business against its ultimate mission. Allowing a situation where the default position is to acquiesce to the demands of commercial publishers because of a lack of institutional capacity (for example not having a properly resourced copyright support provision) potentially creates a bottleneck in the system.

In 2015 The Publishers Licensing Society undertook research which suggested that the TDM exception had brought ­little value to researchers and that commercial publishers had responded to the challenge by ­creating innovative licensing solutions. These findings seem premature and the research question and methodology also ­appears to be framed to reinforce their members’ commercial interests rather than seek an objective view. However libraries should ask themselves what role they are playing (or not playing) in changing the ­status quo and advancing the research agenda.

The Libraries and Archives Copyright Alliance (Laca) and Universities UK have been collecting evidence of problems researchers run into when trying to use the TDM exception and in September 2015 Laca appealed to the IPO on behalf of a UK academic to have DRM removed from a site he wished to mine. In this instance the academic was trying to mine a publicly accessible website, which used Captcha technology that prevented him from downloading more than a few records at a time.

The case was eventually referred to the IPO who, after some deliberation, decided the case was not within scope of the exception. They argued the exception did not apply to work ‘made available to the public on agreed contractual terms in such a way that members of the public may access them from a place and at a time individually chosen by them.’ You can read more about the case on the Laca website, however it seems likely then that this would apply to any web-based resource.  Although the outcome of the application was not successful, it is hoped that with continued referral of similar cases the library community will be able to create an environment where publishers ­respect the new exception and researchers are more easily able to access the information they need.

As Lauren Smith said in the closing keynote of this year’s CILIP conference (in the context of public library closures), it is important to keep fighting battles even if you think it’s unlikely you will win. Without further applications to government to address this issue on a case-by-case basis there will be no possibility to change the status quo.

Exploring strange new worlds

Researchers will increasingly look to librarians for assistance with TDM, and they should be ready and willing to assist with these projects, and actively promoting the opportunities that technology provides. Librarians are well placed to help with rights issues and to create case studies which can influence the legislative reform required to use TDM for the wider benefit of society. The Hague Declaration is a powerful statement, which advocates removal of legal barriers to knowledge creation in the digital age and the declaration web site includes a number of case studies from across Europe. The future of the UK’s involvement in efforts to further harmonise TDM laws across the EU may now be in a state of legislative limbo following Britain’s EU referendum result but there are still moves afoot to bring member countries in line with the recent UK legislation. This may still allow use of TDM in multi-national research partnerships across Europe, something not currently permitted where licences are not available or practical to acquire.

Ultimately, librarians should be mindful of the right to mine, and recognise the ­important role they can play in liberating data and knowledge. Their continuing mission: to explore strange new worlds, to support research and discovery of new scientific breakthroughs, to boldly go where no one has gone before. 

About the authors

Dr Jane Secker is Senior Lecturer in Educational Development at City University. London. She is Chair of the CILIP Information Literacy Group and the author of several books.
Chris Morrison is Copyright Compliance Officer at the University of Kent.
Neil Stewart is Digital Library Manager at LSE.
Laurence Horton is Data Librarian at LSE.

References

Original illustration by Davide Bonazzi. Source: http://copyrightuser.org/topics/text-and-data-mining/

Update Magazine

This article was originally published in CILIP Update Magazine, September 2016.