Bridging The “Last Mile” of Data Access: How Digital Twin Technology Could Help Enable NOAA’s Mission in Revolutionary Ways

Jeff Steward, Ph. D.

When a user wants to access data from the National Oceanic and Atmospheric Administration (NOAA), there is usually a reason for the inquiry. For example, they may request the nightly temperature forecast for weeks at a time but what they are really seeking to know is when the nightly temperatures will be above 50 degrees Fahrenheit at night in order to lay concrete. While the data provided is useful, the user may become frustrated due to the overwhelming volume of data and the amount of work necessary to uncover the answer to a relatively simple question. Bridging this gap, known as the last mile, is a significant but challenging part of NOAA’s mission. To address this challenge, Orion recently completed a concept study for NOAA’s Satellite and Information Services (NESDIS) Joint Venture Partnerships (JVP) program to explore a new approach. Our concept study focused on applying digital twin technology to enhance NOAA’s weather monitoring and modeling systems, aiming for a solution that's not only accurate but also efficient and user-friendly for all of NOAA's service beneficiaries.

UNLOCKING THE POWER OF DIGITAL TWINS

Digital twins have been used successfully in sectors ranging from manufacturing to health care. The core idea is to create a digital replica of a system that is easier to interact with than the physical system. As part of a concept study awarded through the JVP Broad Agency Announcement, our team set out to prototype an Earth observation digital twin, which is an ambitious fusion of advanced software engineering and machine learning for the Earth as a whole. At its core, this digital twin acts as a virtual representation of Earth’s dynamic systems, capable of ingesting, analyzing and visualizing enormous streams of environmental data from a wide array of sensors. 

The vision is a unified, intuitive digital environment that is prime for delivering actionable insights to NOAA’s diverse community of users from operational forecasters to the general public. Our prototype digital twin solution demonstrated promise for delivering detailed information at the final stage of a data processing workflow, ensuring that end-users receive highly relevant, precise, error-quantified, and easily accessible insights. By making critical information easier to access and interact with, this technology empowers users to model not just current or historical (“what now”) and future (“what next”) conditions, but also hypothetical (“what if”) scenarios. These capabilities are key tools for addressing the last mile of what users are actually looking for, and each of these capabilities is necessary for delivering on this promise. 

STRENGTHENING COLLABORATIONS THROUGH JVP

The NESDIS Joint Venture Partnerships (JVP) program is designed to foster meaningful collaborations between the private sector, academia and other federal agencies. This collaborative endeavor is at the heart of driving innovation, utilizing external expertise to champion the development of state-of-the-art Earth observation and ground system capabilities. 

Through vehicles like Broad Agency Announcements (BAA), JVP funds pilot and demonstration projects to assess the feasibility of emerging technologies, innovative instruments and new mission concepts. This proactive strategy allows NESDIS to identify and integrate cutting-edge solutions, ensuring NOAA’s mission needs are met efficiently. The successful integration of these solutions amplifies the scientific community's capacity and elevates the quality of environmental data and services, ultimately resulting in the efficient use of taxpayer dollars and the enhancement of national weather and climate monitoring infrastructures. 

KEY FINDINGS FROM THE STUDY

In a recent announcement released by NOAA, the agency shared key findings of the NESDIS JVP study. Specifically, the release stated that the study determined the NOAA weather monitoring and modeling could improve with digital twin technology. Highlighting Orion’s contributions on the Earth Observations digital twin project, the release elaborated that in the course of developing a digital twin system for Earth observation data processing, our study highlighted several best practices to optimize performance, accessibility and scalability. Here's a summary of the recommendations that emerged from our research and prototyping efforts: 

1. Adopt Open-Source Software: Use open-source software tools, processes and engines that adhere to the standards set by the geospatial community. This ensures compatibility and facilitates easier collaboration and innovation across various disciplines and organizations 

2. Embrace Open-Source Data Formats: The adoption of open-source formats for data storage and streaming is recommended, with a preference for the Open Geospatial Consortium 3D Tiles format. This format is highly adaptable and supports the capacity for scaling and customization in data sets 

3. Leverage Web-Based Visualization Tools: By using web-based tools for data visualization, we eliminate the necessity for users to download and install client software. This approach greatly simplifies the user experience and makes the data more accessible to a broader audience 

4. Automate Data Processing Pipelines: Automation is key in ensuring that the processing and integration of data are not just swift but also consistent, enabling the continuous delivery of updated information as soon as it becomes available 

5. Maintain Data Integrity: The integrity of original scientific data should be maintained in its primary format to ensure authenticity. For efficient storage and streaming, a secondary format is suggested where modeled data points are stored within a hierarchical grid structure, enabling easy access while upholding scientific fidelity 

   
   

Adherence to these recommendations supports the creation of a digital twin environment that is robust, efficient, and able to evolve with the growing needs of the Earth observation field. 

LOOKING AHEAD

Digital twin technology offers powerful new possibilities for improving how we understand and respond to Earth’s complex weather systems. The insights gained from our concept study for NOAA NESDIS mark just the beginning. 

Looking ahead, the focus is on making these tools even more user-friendly, efficient and capable. The vision is a seamless integration of advanced technologies into everyday weather forecasting and climate research, helping scientists, forecasters, and decision-makers as well as the general public access the insights they need, when they need them.  

To further build on this work, we are working on an additional project with NOAA known as the Knowledge Mesh Natural Language Processing project. This project utilizes large language models and natural language processing with digital twins to directly solve the user’s queries. Each user will have an associated “persona” so that the system can tailor answers to an understanding of what the user is looking for. After all, a scientist studying ground temperature and a farmer looking to lay concrete will have very different expectations of a system. The users will use natural language conversational text to query against a comprehensive knowledge mesh of data and processes with digital twins providing the “what now,” “what next,” and “what if” system capabilities. This style of interface is of course becoming increasingly prevalent with the rise of ChatGPT and other generative AI tools, so this project represents a great opportunity to tie these different streams together. 

Stay tuned to the KnowledgeXchange for updates as this important work evolves, and as we continue to explore the future of digital twin innovation in weather and climate science.