Generate small website for events

README.pdf

events/*/docs/ai_generated/papers/

public/

# You can override the included template(s) by including variable overrides

# SAST customization: https://docs.gitlab.com/user/application_security/sast/#available-cicd-variables

# Secret Detection customization: https://docs.gitlab.com/user/application_security/secret_detection/pipeline/configure/

# Dependency Scanning customization: https://docs.gitlab.com/user/application_security/dependency_scanning/dependency_scanning_sbom/#customizing-analyzer-behavior

# Container Scanning customization: https://docs.gitlab.com/user/application_security/container_scanning/#customizing-analyzer-behavior

# Note that environment variables can be set in several places

# See https://docs.gitlab.com/ci/variables/#cicd-variable-precedence

# GitLab CI for "Analyse Open Data"

#

# Assumptions

# - Runners: a Docker executor (shared runners) that can pull images from

#   mcr.microsoft.com and registry.gitlab.com (security analyzers). With a

#   shell executor the `image:` lines are ignored and the pages job needs

#   python3 + playwright + chromium installed on the runner.

# - GitLab Pages is enabled on the instance and for this project

#   (Settings > General > Visibility > Pages). Visibility/access control of the

#   published site is set there.

# - Default branch is `main`; the site is only deployed from it.

# - The pages job is named `pages` and publishes `public/`, which every GitLab

#   version recognises. The newer `pages: publish:` keyword (GitLab >= 17.9) is

#   deliberately NOT used: older instances reject it as an unknown key.

#

# Why Auto-DevOps was replaced by explicit templates

# - The repository has no Dockerfile and no application to deploy, so the

#   Auto-DevOps build job (buildpack detection) would fail or do nothing useful,

#   and the review/staging/canary/production/rollout jobs need a Kubernetes

#   cluster. Only the security scanners were used, so they are included

#   directly (same templates Auto-DevOps includes internally).

#

# DATA LICENCE CAVEAT

# - events/20261103/extreme_weather embeds EM-DAT-derived country-level

#   aggregates. EM-DAT terms of use may forbid redistribution (see

#   docs/ai_generated/open_data_licences.md). It is published unless the owner

#   adds it to scripts/site_exclude.txt (a commented line is ready there).

#

# Customisation docs:

# SAST: https://docs.gitlab.com/user/application_security/sast/#available-cicd-variables

# Secret Detection: https://docs.gitlab.com/user/application_security/secret_detection/pipeline/configure/

# CI variable precedence: https://docs.gitlab.com/ci/variables/#cicd-variable-precedence

stages:

  - build

  - test

- deploy

- review

- dast

- staging

- canary

- production

- incremental rollout 10%

- incremental rollout 25%

- incremental rollout 50%

- incremental rollout 100%

- performance

- cleanup

  - secret-detection

sast:

  stage: test

  - deploy

include:

- template: Auto-DevOps.gitlab-ci.yml

  - template: Jobs/SAST.gitlab-ci.yml

  - template: Jobs/Secret-Detection.gitlab-ci.yml

variables:

  SECRET_DETECTION_ENABLED: 'true'

sast:

  stage: test

secret_detection:

  stage: secret-detection

pages:

  stage: deploy

  # Pin the tag; keep the pip pin below in sync with it.

  image: mcr.microsoft.com/playwright/python:v1.49.1-noble

  variables:

    PLAYWRIGHT_VERSION: "1.49.1"

  script:

    # The image ships browsers; install the matching python package if absent.

    - python3 -c "import playwright" 2>/dev/null || pip install --no-cache-dir "playwright==${PLAYWRIGHT_VERSION}"

    - python3 scripts/build_site.py --out public

    # Safety net: never publish build scripts, templates or docs (papers PDFs).

    - "! find public \\( -name '*.py' -o -name 'template.html' -o -name '*.pdf' -o -path '*/docs/*' \\) | grep ."

  artifacts:

    paths:

      - public

    expire_in: 1 week

  rules:

    - if: $CI_COMMIT_BRANCH == "main"

# Licence review of the open_data repository

AI generated by Claude Sonnet and Claude Opus 5.5 on 2026-10-21. Not legal advice.

Scope: the datasets in the `open_data` repository, folders `carbon_pricing`,

`renewable_energy` and `extreme_weather`. Licence terms were checked on the web on

2026-10-21 and in the OWID metadata files. Anything not confirmed is marked

"unverified". Datasets whose terms do not allow redistribution are not kept in the

repository and are not listed here.

## Summary

| File(s) | Dataset | Provider / original source | Licence | Size | Verdict |

|---|---|---|---|---|---|

| `carbon_pricing/owid_carbon-tax-instruments.csv` (+ metadata) | Carbon tax instruments | OWID / Dolphin and Merkle (2024), RFF | CC BY 4.0 (OWID processing); upstream terms unverified | 244 KB | Publish with attribution file |

| `carbon_pricing/owid_carbon-emissions-trading-system.csv` (+ metadata) | ETS presence | same | same | 194 KB | Publish with attribution file |

| `carbon_pricing/owid_carbon-tax-trading-coverage.csv` (+ metadata) | Share of CO2 covered | same | same | 200 KB | Publish with attribution file |

| `carbon_pricing/owid_emissions-weighted-carbon-price.csv` (+ metadata) | Emissions-weighted carbon price | same | same | 157 KB | Publish with attribution file |

| `carbon_pricing/owid-co2-data.csv` | OWID CO2 data | OWID / Global Carbon Project and others | CC BY 4.0 (OWID); third-party columns keep upstream terms | 33.2 MB | Do not commit (size); give download instructions |

| `carbon_pricing/datahub_carbon-pricing-initiatives.csv`, `datahub_carbon-prices.csv`, `datahub_datapackage.json` | World Bank carbon pricing (datahub mirror) | World Bank, State and Trends of Carbon Pricing, via datahub.io | CC BY 4.0 (package); NZ ETS rows PDDL | 14 KB, 112 KB, 8 KB | Publish with attribution file |

| `renewable_energy/ember_yearly_generation_global.csv` | Ember yearly electricity data | Ember | CC BY 4.0 | 16.1 MB | Licence fine; git LFS or download instructions (size) |

| `renewable_energy/owid_solar-pv-prices-vs-cumulative-capacity.csv` (+ metadata) | PV module price vs capacity | OWID / Nemet (2009), Farmer and Lafond (2016), IRENA | CC BY 4.0 (OWID); IRENA terms unverified | 2 KB + 4 KB | Publish with attribution file |

| `renewable_energy/owid_levelized-cost-of-energy.csv` (+ metadata) | LCOE | OWID / IRENA | CC BY 4.0 (OWID); IRENA terms unverified | 36 KB + 15 KB | Publish with attribution file |

| `renewable_energy/owid_cumulative-installed-wind-energy-capacity-gigawatts.csv` (+ metadata) | Wind capacity | OWID / IRENA | same | 93 KB + 1 KB | Publish with attribution file |

| `extreme_weather/worldbank_NY.GDP.MKTP.CD.json` | World Bank GDP | World Bank WDI | CC BY 4.0 | 3.7 MB | Publish with attribution file |

| `extreme_weather/worldbank_SP.POP.TOTL.json` | World Bank population | World Bank WDI (UN and other sources) | CC BY 4.0 default; third-party terms unverified for this series | 3.5 MB | Publish with attribution file |

| `extreme_weather/worldbank_countries.json` | World Bank country list | World Bank | CC BY 4.0 | 114 KB | Publish with attribution file |

| `extreme_weather/owid_continents.csv` | OWID continents | OWID | CC BY 4.0 | 8 KB | Publish with attribution file |

| `extreme_weather/noaa_billion_dollar_events_US_1980_2024.csv` | US billion-dollar disasters | NOAA NCEI | US Government work (public domain, unverified on the NCEI page) | 36 KB | Publish with attribution file |

| `*/SOURCES.md` | Documentation | repository authors | n/a | 4 KB each | Publish |

## General point: OWID's CC BY and third-party data

OWID states (FAQ, and the `owid/co2-data` README) that its CC BY covers only data OWID

produces itself. Third-party data stays "subject to the license terms of those

providers", and reuse must credit both OWID and the original provider. So each OWID file

below needs two checks: OWID's CC BY and the upstream terms.

## Carbon pricing: OWID World Carbon Pricing Database files

Files: `owid_carbon-tax-instruments`, `owid_carbon-emissions-trading-system`,

`owid_carbon-tax-trading-coverage`, `owid_emissions-weighted-carbon-price` (each `.csv` +

`.metadata.json`).

- Provider: Our World in Data (last updated 2026-06-10).

- Upstream: Dolphin and Merkle (2024), World Carbon Pricing Database / Emissions-weighted

  Carbon Price, Resources for the Future (RFF). Citation in metadata: "Dolphin and Merkle

  (2024) – with minor processing by Our World in Data."

- Licence: CC BY 4.0 for OWID processing, https://creativecommons.org/licenses/by/4.0/.

  The metadata JSONs contain no licence field, only citations.

- Upstream terms: unverified. No licence statement was found on an RFF page. The GitHub

  repository g-dolphin/WorldCarbonPricingDatabase has a LICENSE file listed as "other"

  (not read). OWID republishes the data under CC BY with "minor processing", which

  suggests the authors allow it.

- Attribution: "Dolphin and Merkle (2024), World Carbon Pricing Database, Resources for

  the Future; with minor processing by Our World in Data. Source:

  https://ourworldindata.org/grapher/<slug>".

- Restrictions: none beyond attribution. No registration.

**Suggestion:** Fine to include in a public repository, with attribution in

`DATA_LICENSES.md`. Before the first push, read the LICENSE of

g-dolphin/WorldCarbonPricingDatabase to confirm it does not forbid redistribution; if it

does, remove these four files and give download instructions instead:

```

cd open_data/carbon_pricing

for s in carbon-tax-instruments carbon-emissions-trading-system carbon-tax-trading-coverage emissions-weighted-carbon-price; do

  curl -L -A "Mozilla/5.0" -o "owid_$s.csv" "https://ourworldindata.org/grapher/$s.csv?v=1&csvType=full&useColumnShortNames=true"

  curl -L -A "Mozilla/5.0" -o "owid_$s.metadata.json" "https://ourworldindata.org/grapher/$s.metadata.json?v=1&csvType=full&useColumnShortNames=true"

done

```

## Carbon pricing: owid-co2-data.csv

- File: `carbon_pricing/owid-co2-data.csv`, 33.2 MB.

- Provider: OWID, https://github.com/owid/co2-data. Upstream sources: Global Carbon

  Project (CO2), Jones et al. (other greenhouse gases, temperature), Energy Institute

  Statistical Review and US EIA (energy), Maddison Project Database (GDP), World Bank

  (income groups), various sources for population.

- Licence: OWID's own work is CC BY 4.0; "third-party data is subject to the license

  terms from the original third-party authors" (README). The Global Carbon Budget paper is

  CC BY 4.0; a separate data licence for the dataset was not found (unverified). Energy

  Institute and EIA terms for the energy columns are unverified.

- Attribution: Our World in Data and Global Carbon Project (Friedlingstein et al., Global

  Carbon Budget, Earth System Science Data), plus the sources of any other column used.

**Suggestion:** Do not commit the raw file, because of its size and the mixed upstream

terms of the energy columns. Add it to `.gitignore` and give download instructions:

```

cd open_data/carbon_pricing

curl -L -A "Mozilla/5.0" -o owid-co2-data.csv https://owid-public.owid.io/data/co2/owid-co2-data.csv

```

Check: about 33 MB; first columns `country,year,iso_code,population,gdp,...`; contains

`co2` and `share_global_co2`; latest year 2024 (as of 2026-10-20). A reduced file with

only `country,year,iso_code,co2,share_global_co2` (under 1 MB) could be committed under

CC BY with Global Carbon Project attribution.

## Carbon pricing: World Bank Carbon Pricing Dashboard via datahub.io

- Files: `datahub_carbon-pricing-initiatives.csv`, `datahub_carbon-prices.csv`,

  `datahub_datapackage.json`.

- Provider: datahub.io package https://datahub.io/climate-and-environment/carbon-pricing

  (updated 2026-03-16). Original: World Bank, State and Trends of Carbon Pricing, data as

  of 2024-04-01.

- Licence: CC BY 4.0, stated in `datahub_datapackage.json` and on the datahub page. The

  NZ ETS series comes from a separate source under ODC PDDL (public domain dedication).

  World Bank datasets are CC BY 4.0 by default

  (https://datacatalog.worldbank.org/public-licenses); the World Bank dashboard's own

  terms could not be read (HTTP 403): unverified.

- Attribution: "World Bank, State and Trends of Carbon Pricing (data as of 1 April 2024),

  CC BY 4.0, via datahub.io; NZ ETS: NZ Ministry for the Environment, via datahub.io

  (PDDL)."

- Restrictions: none beyond attribution.

**Suggestion:** Fine to include. Add the attribution to `DATA_LICENSES.md` and keep

`datahub_datapackage.json` next to the CSVs, since it documents the licence.

## Renewable energy: Ember yearly electricity data

- File: `renewable_energy/ember_yearly_generation_global.csv`, 16.1 MB.

- Provider: Ember, https://ember-energy.org/data/yearly-electricity-data/.

- Licence: CC BY 4.0, stated on the Ember page.

- Attribution: "Ember – Yearly Electricity Data", with URL and access date.

- Restrictions: none beyond attribution. No registration.

**Suggestion:** The licence allows redistribution. Because of the size, either track it

with git LFS, or add it to `.gitignore` and give download instructions:

```

cd open_data/renewable_energy

curl -L -A "Mozilla/5.0" -o ember_yearly_generation_global.csv https://files.ember-energy.org/public-downloads/generation/outputs/release_generation_yearly_global.csv

```

Check: about 16 MB; sha256

`91d30fb170dac8a62130f7b39e11ecbb301f902e08400e9561510882eadcd701` for the version of

2026-10-20 (changes when Ember updates).

## Renewable energy: OWID solar PV prices, LCOE, wind capacity

Files: `owid_solar-pv-prices-vs-cumulative-capacity`, `owid_levelized-cost-of-energy`,

`owid_cumulative-installed-wind-energy-capacity-gigawatts` (each `.csv` +

`.metadata.json`).

- Provider: OWID. Upstream: PV prices from Nemet (2009) for 1975–2003, Farmer and Lafond

  (2016) for 2004–2009, IRENA Renewable Power Generation Costs from 2010, with processing

  and inflation adjustment by OWID. LCOE and wind capacity: IRENA.

- Licence: CC BY 4.0 for OWID processing. IRENA terms unverified (the IRENA terms page

  returned HTTP 403); IRENA's usual notice allows reuse with acknowledgement of IRENA as

  source and copyright holder.

- Attribution: "IRENA, Renewable Power Generation Costs (2024, 2025, 2026); Nemet (2009);

  Farmer and Lafond (2016); with processing by Our World in Data", CC BY 4.0.

- Restrictions: none known beyond attribution. All files under 100 KB.

**Suggestion:** Fine to include, with the attribution in `DATA_LICENSES.md`. Confirm

IRENA's current terms page in a browser for certainty.

## Extreme weather: World Bank WDI files

- Files: `worldbank_NY.GDP.MKTP.CD.json` (3.7 MB), `worldbank_SP.POP.TOTL.json` (3.5 MB),

  `worldbank_countries.json` (114 KB).

- Provider: World Bank API v2 (World Development Indicators; country metadata).

- Licence: CC BY 4.0 is the default for datasets produced by the World Bank

  (https://datacatalog.worldbank.org/public-licenses). Exceptions apply where a third

  party requires other terms. Total population is compiled from the UN Population

  Division and other sources; a custom licence for this series is unverified.

- Attribution: "World Bank, World Development Indicators (NY.GDP.MKTP.CD; SP.POP.TOTL),

  CC BY 4.0, accessed 2026-10-20", noting that the files are unmodified API responses.

- Restrictions: none beyond attribution. No registration.

**Suggestion:** Fine to include, with the attribution above. If the population series

turns out to carry third-party terms, replace the file with download instructions:

```

cd open_data/extreme_weather

curl -L -o worldbank_SP.POP.TOTL.json "https://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?format=json&per_page=20000&date=1960:2025"

```

## Extreme weather: OWID continents

- File: `owid_continents.csv` (8 KB): OWID entity names, ISO3 codes and continent

  (OWID definition).

- Licence: CC BY 4.0 (classification produced by OWID).

- Attribution: "Our World in Data, Continents according to Our World in Data, CC BY 4.0."

**Suggestion:** Fine to include, with the attribution above.

## Extreme weather: NOAA billion-dollar disasters

- File: `noaa_billion_dollar_events_US_1980_2024.csv` (36 KB). Provider: NOAA NCEI, U.S.

  Billion-Dollar Weather and Climate Disasters; the product was retired in May 2025.

- Licence: no explicit licence on the NCEI pages. As a US federal government work it is

  in the public domain in the US (17 U.S.C. 105); this is general knowledge, not stated

  on the page (unverified).

- Attribution: "NOAA National Centers for Environmental Information (NCEI), U.S.

  Billion-Dollar Weather and Climate Disasters (2025), https://doi.org/10.25921/stkw-7w73".

**Suggestion:** Fine to include, with the citation above and a note that the product is

archived and no longer maintained by NOAA.

## Proposed repository layout

```

open_data/

  README.md          what the repository holds, how to fetch the ignored files

  DATA_LICENSES.md   per file: provider, licence, attribution text, access date

  fetch_missing.sh   downloads the ignored files listed below

  .gitignore

  carbon_pricing/  renewable_energy/  extreme_weather/   each with SOURCES.md

```

Do not put one licence on the whole repository: the content mixes CC BY 4.0, PDDL and

public domain, and the repository owner is not the rights holder. Use

`DATA_LICENSES.md` with per-file attribution, and license only the repository's own

README, SOURCES.md files and scripts.

`.gitignore`:

```

carbon_pricing/owid-co2-data.csv

renewable_energy/ember_yearly_generation_global.csv   # remove this line if using git LFS

```

`fetch_missing.sh`: `set -e`, then the `curl` commands from the sections above, followed

by a size and `sha256sum` check against the values in `SOURCES.md`.