Conference Agenda

In support of research, teaching and learning, the Stanford Digital Repository (SDR) is a network of systems and services that house the digital collections of Stanford University Libraries (SUL). Collections in SDR include Google-scanned books, student dissertations and theses, University Archives, Allen Ginsberg's papers, Parker Library, and the Fugitive U.S. Agencies Web Archive, to name a handful. As of late 2022, SDR holds over 5 million digital objects composed of more than 530 million content files. SDR is extremely heterogeneous along several facets, including content types (e.g., books, images, web archives, GIS datasets) and file types.

For the past 4 years, SUL has been working towards migrating SDR to a new datastore and data model. We successfully completed the migration this year. In this presentation, we will describe the motivations for this work and the strategies used to accomplish the migration. These strategies may be repurposeable in other production system migrations: adopting a validatable data model, abstracting the datastore behind an API, separating concerns, testing metadata mappings against production, writing reports to understand complex data, templating unit tests, performing a rolling migration, and incorporating migration into ongoing project work.

In January 2023 the national harvester for The Netherlands (Narcis) announced to stop their services within a few months, leaving The Netherlands without a central portal for their scholarly output. Even though the content of the larger repositories may be well indexed in scholarly search engines, the national portal was important for the visibility of Dutch research output for several reasons. First, the content was not limited to major universities; smaller research organizations and universities were connected and delivered unique materials (e.g. research reports). Second, the audience of Narcis is not just academics: due to its focus on national output and added value such as the experts guide, it was a source of knowledge for journalists, companies and policy makers that are less likely to use scholarly search engines. Third, the content was not limited to publications, but also covered datasets, software and funding information and connected these items through a PID-graph. A taskforce was created to quickly prevent the loss of visibility of the national output. By setting clear goals, monitoring and having a good collaboration with OpenAIRE, a new research portal based on OpenAIRE CONNECT and a Monitor dashboard were set up before July: the Netherlands Research Portal.

This presentation will describe the Harvard Library DRS Futures Project. It will then detail the initial exploratory phase and principles that informed the requirements and design philosophies that underpin the development phase. Finally, it will share lessons learned and upcoming challenges as the project enters its third year.