Changes for docs/production_doppler_lidar_conversion/other_scan_types/non_vertical_dataset_overview.md: 131 added lines, 0 removed lines.
Original line number
Diff line number
Diff line
# Dataset Overview: Non-Vertical Scan Products
## Purpose
This page summarises the non-vertical Doppler lidar datasets currently identified for production conversion, with present focus on the **DL141 User5 scans**.
The purpose of this overview is to capture the **observed structure of the source data**, the **main scan families present**, and the **key properties relevant to conversion**.
## Current scope
At present, only one non-vertical scan family has been formally taken forward for production processing:
Other non-vertical or ambiguous cases can be added later as this documentation grows.
## Source inventory summary
### DL141 dataset context
Within the broader `DL_141_vertical_until_17_june_2025` source tree, the HPL files separate naturally into two broad groups:
1.**Stare-based files** (`Stare_141_*.hpl`) — treated in the vertical-scan workflow.
2.**User5 files** (`User5_141_*.hpl`) — treated here as non-vertical scans.
### User5 inventory
The User5 inventory identified:
-**1676** files matching `User5_141_*.hpl`
- concentrated on **2025-06-17**
- two regular scan families:
-`DL_DIAL_north - csm`
-`DL_DIAL_south - csm`
The regular inventory supports the interpretation that the User5 data consist primarily of **fixed-azimuth elevation scans**.
## Geometry overview
### Regular scan geometry
For the regular User5 files, the beam geometry is characterised by:
-**Azimuth:** approximately `330°` (constant within a file)
-**Elevation:** progressive scan from near-horizontal values up to roughly `9.74°`
-**Range:** 100 gates at 30 m gate length
A representative file shows:
- azimuth repeated at `330°` for all rays,
- elevation increasing from approximately `-0.5°` through positive values to around `9.74°`,
- range-gate centres from `15 m` to `2985 m`.
This makes the User5 scans structurally very different from the vertical-stare products, for which elevation is close to `90°`.
## Scan-family breakdown
The regular User5 files are grouped operationally into two scan families:
### 1. North scan family
-**scan_type:**`DL_DIAL_north - csm`
- treated as one homogeneous scan family for aggregation
- aggregated into a single daily product for `2025-06-17`
### 2. South scan family
-**scan_type:**`DL_DIAL_south - csm`
- treated as one homogeneous scan family for aggregation
- aggregated into a single daily product for `2025-06-17`
## Structural characteristics relevant for conversion
The regular User5 files typically show:
-`Number of gates = 100`
-`Range gate length = 30.0 m`
-`Gate length (pts) = 10`
-`Pulses/ray = 3000`
-`No. of waypoints in file = 1`
-`Resolution = 0.0382 m s-1`
- around `61–62` rays per file, with some normal variation
These are sufficiently regular to support a consistent preservation-oriented Level-1 conversion.
## Aggregation logic
The daily aggregation for the User5 scans is performed by grouping files by:
1.**UTC date**, and
2.**scan family** (`north` or `south`).
This is different from the vertical-stare aggregation, where grouping is typically by UTC date alone.
For the regular User5 files, this produced:
- one **north** daily aggregate for `2025-06-17`, and
- one **south** daily aggregate for `2025-06-17`.
## Important limitations
### No gate geolocation at this stage
The current conversion does **not** compute per-gate latitude, longitude, or altitude.
This is intentional. The Level-1 and CF-oriented products retain:
- instrument latitude,
- instrument longitude,
- instrument altitude,
- azimuth,
- elevation,
- pitch,
- roll,
- slant range.
However, transformation of individual gates into geographic coordinates is deferred to later scientific analysis.
### Why geolocation is deferred
The main reasoning is:
- the source azimuth convention should not be over-interpreted without dedicated confirmation,
- different studies may require different geometric assumptions,
- uncertainty introduced by the geometric transformation should be documented in the context of the intended analysis.
## Relation to later documentation
- For workflow differences relative to vertical scans, see [`processing_differences_from_vertical_scan.md`](./processing_differences_from_vertical_scan.md).
- For known outliers and deferred issues, see [`exceptions_and_follow_up.md`](./exceptions_and_follow_up.md).
