Commit be6122ff authored by Jonathan Minz's avatar Jonathan Minz
Browse files

remove dmp.md

parent 075ee33e
Loading
Loading
Loading
Loading
+0 −254
Original line number Diff line number Diff line
# LAFI Working Data Management Plan

| Data | Management | plan 28.08.2025 |
| --- | --- | --- |
| ## Aims | And | Objectives: |
| The key aims | and objectives | of the Research | Data | Management | (RDM) | team, | within | the broader |
| context | of the LAFI | project, | are as follows |
| ## 1. To prepare | harmonized , standardized , and easily | accessible | datasets | to facilitate | effective |
| scientific | collaboration | within | LAFI | and beyond. |
| ## 2. To publish | these | datasets | in accordance | with | the FAIR | principles | (Findable, | Accessible, |
| Interoperable, | and Reusable), | thereby | supporting | Earth | System | Science | research, | education, |
| and evidence -based | environmental | policy -making. |
| ## 3. To document | RDM | workflows | and research | software, | archive | datasets, | and develop |
| tutorials—establishing | benchmarks | for FAIR | data practices | within | the wider | Earth | System |
| Science | (ESS) | community. |
## Tasks:
| The aims | and objectives | of the LAFI | RDM | team | can be organized | into the following | high -level |
| focus | areas: |
| ## 1. Data | Harmonization | and Standardization |
| ## 2. FAIR | data storage, | archiving | and publication. |
| ## 3. Development | of a web-based | user interface | for the LAFO | server |
## 4. Documentation
## 
## 
| These | categories | can be thought | of as broad | focus | areas | that require | attention | for realization | of |
| the intended | data management | aims. | Each | category | is further | divided | into specific | tasks | designed |
| to ensure | that all data generated | within | the LAFI | project | aligns | with | the FAIR | principles | (go- |
fair.org/fair -principles ).
| ## Timelines, | Milestones | & Deliverables: |
![LAFI RDM Timeline](lafi_dmp_images/page_1_img_1.png)

| LAFI | timeline | and milestones |
## 
| Data | Management | plan 28.08.2025 |
| The Gantt | chart | above | outlines | the timeline | for RDM | tasks | over the 2025–2027 | period, | which |
| corresponds | to the remaining | duration | of the LAFI | project. | It is anticipated | that a proposal | for a |
| potential | extension | of LAFI | will need | to be finalized | by the end of Q2 2027. | This represents | a key |
| deadline | for RDM -related | activities | (Milestone | 5 – M5). | Other | significant | milestones | include | the |
| proposed | Fall Schools | in 2025 | (M1) | and 2026 | (M3), | the planned | commencement | of converting |
| LAFI | data from | all contr ibuting | groups | into obs4MIPs -compliant | netCDF | files (M2), | and the |
| launch | of the FLAIR | project | (M4), | with | LAFI | serving | as the Living | Use Case | (LUC) | for |
NFDI4Earth.
| During | 2025 | and early | 2026, | the primary | focus | will be on preparatory | work | required | to enable |
| the conversion | of LAFI | datasets | into obs4MIPs -compliant | netCDF | formats. | These | efforts | will |
| involve | developing | and testing | Python | scripts | to generate | CF-compliant | netCDF | files from | native |
| machine | outputs, | incorporating | error | estimates, | implementing | quality | flag as well as aligning |
| metadata | with | CF and obs4MIPs | standards. | The subsequent | project | phase | will emphasize | data |
| conversion, | the development | of a RESTful | API or web portal | for direct | access | to LAFI | data, | and |
| the publication | of datasets | across | multiple | platforms, | including | NFDI4Earth’s | OneStop4All | ( a |
| centralized | web portal | providing | unified | access | to NFDI4Earth | data, | tools, | services, | and training |
| resources ), obs4MIPs , World | Data | Center | for Climate | (WDCC) , and PANGAEA. |
| While | the authoring | of user-facing | documentation | and tutorials | is expected | to begin | in earnest |
| following | the start of the FLAIR | project, | the internal | documentation | of Python | scripts | will be an |
| ongoing | activity | throughout | the 2025–2027 | period | to facilitate | effective | code | sharing | within | the |
| LAFI | consortium. |
| Each | milestone | is associated | with | a concrete | deliverable. | For Milestone | 1 (M1), | a real-world | use |
| case involving | Doppler | Lidar | (DL) | data conversion | and its documentation | is planned . This will |
| allow | testing | of the script | with | data generated | by other | groups | at the 2025 | Fall School. | A working |
| script | and associated | documentation | for DL data conversion | has been | created | and successfully |
| tested . These are already | available | on a working | Gitlab | repository | (link). By Milestone | 2 (M2), | it |
| is expected | that the DL datasets | will have | been | successfully | processed | using | an updated | CMOR |
| script, | enabling | the historical | LAFO | Doppler | Lidar | data to be converted | into the obs4MIPs | format. |
| By Milestones | 3 and 4 (M3 and M4), | a dedicated | LAFI | GitLab | repository, | along | with | script |
| documentation | and initial | tutorials, | should | be available. | Furthermore, | a web interface | will be |
| developed. | By the final | milestone | (M5), | selected | LAFI | datasets | should | be published | via the |
| NFDI4Earth | service | portfolio | and obs4MIPs, | accompanied | by comprehensive | documentation | and |
| user tutorials | as well as web access . |
| ## Tools | And | Standards : |
| A successful | outcome | of the tasks | described | above | depends | on the consistent | use of specific | tools |
| and standards. | In general, | data analysis, | processing, | and conversion | scripts | developed | by the LAFI |
| RDM | team | will be written | in Python. | These | scripts | will be stored | and version -controlled | using |
| GitLab, | which | will also serve | as a platform | for documenting | their | functionality | and usage. |
| Additionally, | GitLab | is expected | to support | software | issue | tracking | throughout | the project. |
| Data | within | the LAFI | project | is generated | by a wide | range | of instruments, | each | producing | output |
| in various | plain | text formats. | This heterogeneity | necessitates | the conversion | of these | datasets | into |
| a standardized | format, | ensuring | broader | usability | within | the Earth | System | Science | (ESS) |
| community. | The ultimate | objective | of LAFI | RDM | is to convert | all relevant | datasets | from | their |
## 
| Data | Management | plan 28.08.2025 |
| original | text formats | into netCDF | files, | which | are widely | supported | and easily | manipulated | using |
| a variety | of programming | tools | and languages. |
| To facilitate | broad | access, | the converted | datasets | will be published | online, | ensuring | availability |
| to researchers, | policymakers, | educators, | and the general | public. | We will adopt | the latest | versions |
| of key data standards —Climate | and Forecast | (CF) metadata | conventions | v1.13 | and obs4MIPs |
| (Observations | for Model | Intercomparison | Projects) | Data | Specifications | ODS | 2.5—both | of which |
| provide | detailed | guidelines | for metadata, | variable | naming, | and file structure. | Compliance | with |
| these | standards | is required | by most | data publishing | and archiving | platforms, | including | obs4MIPs, |
| OneStop4All, | WDCC, | DOKU, | and PANGAEA. |
| To verify | adherence | to the CF conventions, | we will use the CEDA | CF-checker | tool |
| (https://help.ceda.ac.uk/article/4160 -cf-checker -command -line-tool). Final | conversion | into |
| obs4MIPs -compliant | netCDF | files will be performed | using | the CMOR | tool |
(https://pcmdi.github.io/obs4MIPs/cmor.html).
| All finalized | LAFI | datasets | will be temporarily | stored | on the LAFO/I | server | at the University | of |
| Hohenheim | before | being | published | online. | However, | due to memory | limitations | on this server, |
| individual | LAFI | research | groups | are responsible | for managing | the storage | of their | raw and |
| intermediate | data. | Groups | based | outside | the University | of Hohenheim | may access | the LAFO/I |
| server | via the university | VPN, | which | requires | the creation | of a guest | account, | university - |
| affiliated | email | address, | and two-factor | authenticati on. The complete | access | procedure | has been |
| documented | separately | (see Protocol | for LAFI | RDM | Meeting | – 27.05.2025). |
| Each | research | group | is responsible | for the conversion | and publication | of its own datasets, | with |
| the LAFI | RDM | team | providing | standardized | scripts | and guidance | on the application | of CF and |
| obs4MIPs | conventions. | While | the RDM | team | can offer | support | and adviso ry services, | it bears |
| direct | responsibility | only for the data produced | by the Institute | of Physics | and Meteorology | at |
| the University | of Hohenheim. |
| ## Key | Stakeholders | - Roles | And | Engagement : |
| In addition | to employing | the right | tools | and standards, | achieving | the objectives | of LAFI | RDM— |
| and contributing | to the establishment | of research | data management | best practices | within | the |
| broader | Earth | System | Science | (ESS) | community —requires | active | collaborat ion with | key |
| stakeholders, | both | within | and beyond | the LAFI | project. |
| Decision -making | authority | regarding | data management | within | LAFI | resides | with | the LAFI |
| Speaker | and Principal | Investigators | (PIs). | The LAFI | RDM | team, | in collaboration | with |
| NFDI4Earth | technical | support, | is responsible | for developing | data conversion | scripts, | clarifying |
| the application | of CF, obs4MIPs, | and FAIR | standards, | maintaining | the LAFI | GitLab, | and |
| authoring | documentation | and tutorials. | The primary | users | of these | outputs | are the various | LAFI |
| research | groups, | who will apply | them | to convert | and publish | their | own datasets. |
| To ensure | alignment | and continuity | across | all stakeholder | groups, | regular | communication | will be |
| maintained. | This includes | progress | updates | through | Data | Management | (DM) | meetings, | tutorials |
| during | the Fall Schools, | and up-to-date online | documentation | for LAFI participants. | For external |
| audiences —including | the broader | ESS research | community, | educators, | and policy -makers—key |
| findings | and updates | will be disseminated | via conference | presentations | and posters | (e.g., | at EGU, |
| AGU, | PyData, | or PyCon) | and publications | in both | peer-reviewed | data science | journals | and |
| general | scientific | publications. |
## 
| Data | Management | plan 28.08.2025 |
| It is envisioned | that open | access | to LAFI | datasets | will be facilitated | through | a web server | or |
| RESTful | API, | in addition | to publication | on established | online | data repositories | such | as obs4MIPs, |
| WDCC, | PANGAEA, | and OneStop4All. | This will enable | a broad | spectr um of users—from |
| researchers | to the public—to access | and work | with | the data. |
| Maintaining | close | contact | with | the CF and obs4MIPs | working | groups | is essential—not only to |
| ensure | that LAFI | datasets | remain | compliant | with | evolving | standards, | but also to contribute |
| meaningful | feedback | toward | improving | those | standards. | This engagement | may take the form | of |
| direct | communication | via email | or GitHub | discussions, | as well as presentations | at steering |
| committee | meetings | of these | respective | working | groups. |
| LAFI | is expected | to generate | numerous | best practices | in research | data management | throughout |
| its duration. | To ensure | that these | insights | are captured | and contribute | to future | standards, | they |
| should | be shared | regularly | with | international | organizations | such | as GLASS, | GLAFO, | and ESMO, |
| as well as local | initiatives | like AI & Data | Science | Certificate | Hohenheim | (AIDAHO ) at the |
| University | of Hohenheim. | AIDAHO | provides | a low-effort | opportunity | to leverage | excellent |
| AI/ML | and Data | Science | expertise | within | University | of Hohenheim | to develop | applications | using |
| the finalized | LAFI | datasets. |
| Engagement | may include | presentations | at key organizational | meetings, | conference | participation, |
| and publications | in both | peer-reviewed | journals | and science | communication | outlets. |
| A summary | of the key stakeholders, | their | roles, | and modes | of engagement | is provided | below. |
## 
| ## Table | 1: Stakeholder | Roles, | Responsibilities | And | Engagement |
| Stakeholder | Role | Responsibilities | / Interests | Type | of Engagement |
| LAFI | Speaker | & |
| PIs Project | Leadership |
| / Decision -makers | Oversee | data management |
| strategy, | approve | standards, |
| guide | overall | RDM | direction | Participation | in Data |
| Management | (DM) |
| meetings, | strategic |
| planning | discussions, |
| feedback | loops |
| LAFI | RDM | Team | Implementation | & |
| Coordination | Develop | data conversion | scripts, |
| ensure | CF/obs4MIPs/FAIR |
| compliance, | maintain | GitLab, |
| produce | documentation | and |
| tutorials | Internal | collaboration, |
| GitLab | management, | DM |
| meetings, | Fall School |
tutorials
NFDI4Earth
| Technical | Support | Technical |
| Guidance | & |
Infrastructure
| Support | Provide | expertise | on FAIR |
| principles, | metadata | standards, |
| and web-based | data publication |
| platforms | Coordination | meetings, |
| feedback | on tools | and |
documentation
| LAFI | Research |
| Groups | Primary | Data |
| Producers | & Users | Apply | RDM | scripts | and |
| standards | to process | and publish |
| their | datasets | Use of GitLab | resources, |
| Fall School | participation, |
| DM meetings, | access | to |
| online | documentation |
CF & obs4MIPs
| Working | Groups | Standards |
| Authorities | Define | and update | metadata | and |
| data formatting | standards; |
| receive | feedback | from | data users | GitHub | discussions, |
| direct | email |
communication,
| presentations | at steering |
| group | meetings |
## 
| Data | Management | plan 28.08.2025 |
| External | ESS |
| Community | (e.g., |
| researchers) | Broader | Scientific |
| Users | Use LAFI | data for Earth | system |
| science | applications, |
| reproducibility, | and meta - |
| analysis | Access | through | data |
| repositories | and APIs, |
| uptake | of published |
| datasets, | conference |
sessions
| Educators | & |
| Policy | Makers | Indirect | Users | / |
| Beneficiaries | Use LAFI | data for teaching, |
| public | communication, | and |
| policy | decisions | Open -access | platforms, |
simplified
documentation,
| presentations | at broader |
| science | forums |
International
Organisations
| ## (Glass, | Glafo, |
| ESMO) | Global | Knowledge |
Exchange
| Networks | Promote | adoption | of data |
| management | best practices | and |
| incorporate | feedback | into global |
| RDM | frameworks | Presentations, | white |
| papers, | participation | in |
| working | groups | and |
meetings
| Local | Initiatives |
| (e.g., | AIDAHO) | Institutional |
| Collaboration | & |
| Outreach | Share | learnings | locally, |
| integrate | LAFI | RDM | practices |
| into institutional | policies | Workshops, | internal |
| seminars, | collaboration |
| through | institutional |
forums
## 
## 

---

## Extracted Figures

![page_1_img_0.jpeg](lafi_dmp_images/page_1_img_0.jpeg)

![page_1_img_1.png](lafi_dmp_images/page_1_img_1.png)

![page_1_img_2.png](lafi_dmp_images/page_1_img_2.png)

![page_2_img_0.jpeg](lafi_dmp_images/page_2_img_0.jpeg)

![page_3_img_0.jpeg](lafi_dmp_images/page_3_img_0.jpeg)

![page_4_img_0.jpeg](lafi_dmp_images/page_4_img_0.jpeg)

![page_5_img_0.jpeg](lafi_dmp_images/page_5_img_0.jpeg)