Bigpicture: Seamless data upload with the Data Transporter

Secure data submission with the Data Transporter

One of the biggest challenges in pathology data sharing is ensuring that each contributor follows the same standards while maintaining security and compliance. “We needed a tool that would allow for an automated, seamless process while ensuring full control for the data contributors,” explains Caitríona Lyons, PhD, Bigpicture WP3 node-coordinator for non-clinical safety, and Research Projects Manager at Deciphex.

The Data Transporter is a secure software tool that allows non-clinical data contributors to securely prepare, validate, and upload pathology datasets to the Bigpicture platform while maintaining full control over the sensitive information. Unlike traditional data-sharing methods, which can be inefficient and inconsistent, the Data Transporter ensures that data is extracted, standardized, sensitive data is encoded, and encrypted before submission.

Dr. Pierre Moulin, Chief Scientific Officer at Deciphex, has been involved in the Bigpicture project since the start, and highlights the strategic importance of tools such as the Data Transporter: “Ensuring efficient, secure, and standardized data transfer is critical to Bigpicture’s long-term success,” he explains. “The Data Transporter is a key enabler of this vision, offering a professionally developed solution for data upload while maintaining strict control over access, authentication, and data integrity.”

How it works

The Data Transporter operates within the firewall of each non-clinical data contributor and works independently, meaning that no institution can see what others are uploading. The tool ensures that only data which they have selected for upload is submitted while removing sensitive, proprietary and identifiable elements to comply with strict data protection regulation. 

To support contributors in using the tool, Deciphex provides each data contributor with two demo studies:

1. A failure case, which highlights errors that need fixing.
2. A success case, showing how the workflow should operate smoothly.

This hands-on approach allows contributors to test the system and understand the required data formatting before submitting real datasets. “We designed the process to be as user-friendly as possible,” says Caitríona. “By actively asking for input and feedback, listening to what the non-clinical beneficiaries communicated and actively collaborating across the various work-packages, we’ve created this application which will streamline the upload of data from the non-clinical beneficiaries directly to the Bigpicture platform.” 

Once a dataset is ready for upload, users must approve the submission, ensuring that nothing is sent without validation. This level of control is crucial for maintaining data quality and contributor confidence.

Interface of the Data Transporter

Why is this important for digital pathology?

Standardization and security are two of the biggest hurdles in pathology data management. Without clear metadata guidelines and compliance measures, datasets can become difficult to navigate, reducing their research value.

With the Data Transporter, contributors benefit from:

• Automated standardization: data is formatted consistently before submission.
• Pre-conversion validation: any errors in metadata are flagged prior to conversion, enabling data submitters to correct their data upfront ensuring a more efficient workflow.
• Full control over data: contributors can review their data before submission, ensuring sensitive / proprietary data is removed or encoded, as well as ensuring compliance with privacy regulations such as GDPR.

For researchers and AI developers who will use this data, these features translate into better searchability, improved data integrity, and enhanced interoperability across different institutions. “By ensuring all submissions follow a clear, standardized approach, we are laying the groundwork for more accurate, high-quality AI models in pathology,” says Caitríona. 

Pierre expands on this point, adding, “The regulatory landscape for AI development in healthcare is evolving rapidly. The design of both Bigpicture and the Data Transporter anticipated these regulatory shifts, ensuring flexibility and adaptability to meet current and future requirements.”

A multidisciplinary approach as a cornerstone

Developing the Data Transporter required close cooperation between multiple disciplines and work packages within Bigpicture. WP3 defined the metadata standards, work package 4 (WP4) defined the implementation of the standard, while work package 2 (WP2) handled the development of the platform and its services. Ensuring alignment between these elements was essential.

“It was a true team effort,” says Caitríona. “We worked across disciplines (pathology, metadata standardization, and software engineering) to develop a tool that met non-clinical’s needs. Open communication and adaptability were key to overcoming the challenges we faced.”

Pierre adds: “The collaboration between experts from different disciplines was essential in making the Data Transporter both technically robust and practically aligned with the needs of the pathology and AI research communities. By bringing together expertise from pathology, metadata structuring, and software engineering, we created a solution that not only ensures data accessibility but also future-proofs our approach to digital pathology.”

One of the main hurdles was ensuring the metadata framework and tool development progressed in sync. The teams took an agile approach, refining the system based on ongoing feedback rather than waiting for a final version. The experiences gained by non-clinical have provided valuable insights that have also supported the clinical implementation. This collaboration ultimately created a flexible and effective tool that serves the entire project and Bigpicture platform!

Data Transporter transports Bigpicture to the next phase

With the Data Transporter fully operational, Bigpicture is reaching an exciting phase; soon, all 10 non-clinical beneficiaries will begin uploading data to the platform. This is a major milestone in creating Europe’s largest, high quality data platform in pathology that will drive the development of AI in pathology (ai) innovation in digital pathology worldwide.

For Caitríona, the real reward is seeing the platform take shape: “Being part of a global collaboration that is changing the future of pathology is an amazing experience. The sense of community and impact makes this a truly rewarding project to work on.”

Pierre concludes: “Bigpicture operates on the foundational values of catalyzing, trustworthiness, collaboration, and inclusiveness. The Data Transporter is a perfect example of these principles in action, bringing different experts together to build a secure and standardized approach that benefits the entire community.”