Creating a Sustainable Future for Scratchpads

Over the course of this year, we've been working towards making Scratchpads a more stable service. As this work is coming close to fruition we wanted to give you an update on recent improvements, the impending move to a new server architecture, and our future development plans.

Scratchpads 2.9.3: First new release since 2016

Over the past months, the Natural History Museum’s Informatics Group have been working to address some of the major problems concerning Scratchpad stability and performance that many of you have been facing over the past 2 years. These efforts are at last being deployed, and on the 1st November, the Scratchpads were migrated to the latest 2.9.3 release. This fixes many outstanding bugs and simplifies the system architecture, replacing ancillary services with a cloud-based issue queue and help wiki. For the moment, you will require a Github account to post issues on the queue but were are currently working on the software to seamlessly integrate this without needing a separate login, and hope to release this shortly.

More powerful servers - 13th November 2018.

We have also been hard at work on the server infrastructure, and the Scratchpads will move to new, more powerful servers on 13th November 2018. This will improve performance and stability across the platform, resulting in less downtime for your Scratchpad. Our new DevOps deployed server architecture will allow us to quickly release further iterations of Scratchpads without service interruption, and drop in new servers as additional capacity is required.

To perform this server migration, we will need to put the Scratchpads into read-only mode whilst data is synchronised. Your Scratchpad will still be online and the content available, but from Friday 9th November to Tuesday 13th November you will be unable to add or edit content. With over 1000 active Scratchpads still in use, this will provide enough time to move the sites, test the performance and address any issues. We apologise for any inconvenience this will cause to you and your Scratchpad users, but we are confident this will vastly improve your experience with the Scratchpad system.

Once the new servers are deployed, the Scratchpads will be fully editable again. At this time we will remove the moratorium on new Scratchpads, and we can start accepting applications again.

The future of Scratchpad support and development

We are determined to never let the Scratchpads service level regress to that of the past two years. Fundamental to this situation was the loss of all funding and the key staff who understood and could work on the platform. Once the new server architecture and stabilisation project concludes, the Scratchpads will adopt a new development process where all members of the Informatics Group development team will take turns in working on the Scratchpads, ensuring we have a pool of developers with the knowledge to continuously improve and support the platform. To improve the transparency and accountability—and to hopefully rebuild some trust in our neglected platform—each six-weekly development cycle will culminate in a new release and blog post announcing the improvements.

Many of the fundamental problems that the Scratchpads sought to address remain relevant today and for the foreseeable future. We believe the best long-term future for the Scratchpads, and the many similar systems that have come to be known as Virtual Research Environments (VRE’s) is if the global community of software developers working on these systems come together and develop a common roadmap. This will lead to greater alignment between these systems, and ultimately a much wider pool of experts, ready, able and resourced to support the natural sciences community in their need to share and manage thematic collections of biodiversity data. Throughout 2019, the NHM team hope to convene discussions with these VRE developer communities to generate a common development roadmap that safeguards the efforts of the contributors adding data to these systems.

Thank You

Thank you for using Scratchpads. We appreciate your patience during this long period and look forward to continuing the development of Virtual Research Environments for the natural sciences community into the future.