Commit 454628d2 authored by Jonathan  Minz's avatar Jonathan Minz
Browse files

Upload New File

parent 62b667ed
Loading
Loading
Loading
Loading
+893 −0
Original line number Diff line number Diff line
# LAFI External SFTP User Provisioning — Technical Documentation

## Purpose

This document is the code-level reference for the registry-driven LAFI external-user provisioning system.

It is intended for administrators and developers who need to:

- understand how the scripts work;
- troubleshoot failures;
- modify or extend the provisioning logic;
- validate account state;
- understand dry-run and logging behavior;
- preserve idempotency and safety.

For the operational overview, see:

`LAFI_external_SFTP_user_provisioning.md`

---

## 1. Components

### THREDDS VM

```text
/home/admin120/lafi/rdm/scripts/sftp_user_creation/
├── sync_lafi_external_users_vm.sh
└── external_users_working.csv
```

Runtime log:

```text
/var/log/lafi-user-provisioning.log
```

### LAFO server

```text
/home/jminz/server_access/user_creation/
├── sync_lafi_external_users_lafo.sh
└── external_users.csv
```

Runtime log:

```text
/var/log/lafi-user-provisioning.log
```

The scripts intentionally modify only the host on which they are executed.

---

## 2. Registry schema

Required header:

```csv
full_name,username,uid,gid,ssh_public_key,date_added
```

The scripts expect this header exactly.

### `full_name`

Human-readable user identity.

### `username`

Unix/SFTP username used on both systems.

### `uid`

Numeric UID. Must be within:

```text
20000-20999
```

### `gid`

Numeric primary GID. Must be within:

```text
20000-20999
```

### `ssh_public_key`

Public SSH key used on THREDDS.

Example:

```text
ssh-ed25519 AAAAC3... user@computer
```

### `date_added`

Initially blank for new users.

Filled by the LAFO script after a successful real provisioning pass.

---

## 3. Why the scripts use Python for CSV parsing

A naive Bash parser such as:

```bash
IFS=, read ...
```

is not robust CSV parsing.

CSV can contain quoted fields and punctuation. The scripts therefore use Python's standard `csv` module and hand the parsed values to Bash as tab-separated fields.

Pattern:

```bash
while IFS=$'\t' read -r full_name username uid gid ssh_public_key date_added; do
    ...
done < <(
    python3 - "$REGISTRY" <<'PY'
    ...
PY
)
```

Bash performs system administration; Python performs CSV syntax handling.

---

## 4. Bash safety settings

Both scripts use:

```bash
set -euo pipefail
```

### `-e`

Stops on an unhandled command failure.

### `-u`

Stops when an undefined variable is used.

### `pipefail`

A pipeline fails if any command in the pipeline fails.

These settings reduce the chance of silent partial provisioning.

---

## 5. Argument handling and dry-run

Both scripts accept:

```bash
script.sh [--dry-run] registry.csv
```

Argument handling:

```bash
DRY_RUN=false

if [[ "${1:-}" == "--dry-run" ]]; then
    DRY_RUN=true
    shift
fi

REGISTRY="${1:-}"
```

`shift` removes `--dry-run`, leaving the registry path as the first positional argument.

---

## 6. Common validation behavior

Both scripts verify:

- script is run as root;
- registry exists;
- Python 3 exists;
- UID is numeric;
- GID is numeric;
- UID is within the reserved range;
- GID is within the reserved range.

Fatal errors use:

```bash
die() {
    echo "ERROR: $*" >&2
    exit 1
}
```

The core safety rule is:

```text
missing state    -> create
correct state    -> verify
unexpected state -> stop
```

The scripts deliberately avoid silently repairing unexpected account state.

---

## 7. THREDDS script

Script:

```text
sync_lafi_external_users_vm.sh
```

Important constants:

```bash
SFTP_GROUP="lafi_sftp"
UID_MIN=20000
UID_MAX=20999
LOG_FILE="/var/log/lafi-user-provisioning.log"
```

---

## 8. Existing-user path on THREDDS

The main check is:

```bash
if getent passwd "$username" >/dev/null; then
```

For an existing account the script verifies:

```bash
id -u "$username"
id -g "$username"
```

against registry UID/GID.

It then checks `lafi_sftp` membership:

```bash
id -nG "$username" | tr ' ' '\n' | grep -Fxq "$SFTP_GROUP"
```

Expected home:

```text
/home/<username>
```

Expected shell:

```text
/usr/sbin/nologin
```

Expected key file:

```text
/home/<username>/.ssh/authorized_keys
```

The registry key must be present as an exact line:

```bash
grep -Fqx "$ssh_public_key" "$auth_keys"
```

A mismatch stops the script.

---

## 9. New-user checks on THREDDS

Before account creation:

```bash
getent passwd "$uid"
```

checks for a conflicting numeric UID.

```bash
getent group "$gid"
```

checks for a conflicting numeric GID.

A conflicting group name is also rejected.

This is essential because the same numeric identities must later exist on LAFO for NFS ownership.

---

## 10. Public-key checks

For a new account, the script requires:

- non-empty key;
- no placeholder;
- recognized key prefix.

Accepted prefixes currently include:

```text
ssh-ed25519
ssh-rsa
ecdsa-sha2-*
```

This is a basic syntax check, not full cryptographic validation.

---

## 11. THREDDS account creation

Private group:

```bash
groupadd --gid "$gid" "$username"
```

User:

```bash
useradd \
    --uid "$uid" \
    --gid "$gid" \
    --groups "$SFTP_GROUP" \
    --create-home \
    --home-dir "/home/$username" \
    --shell /usr/sbin/nologin \
    "$username"
```

SSH directory:

```bash
install -d \
    -o "$username" \
    -g "$username" \
    -m 700 \
    "/home/$username/.ssh"
```

`authorized_keys` is created as:

```text
owner: <username>:<username>
mode: 600
```

---

## 12. Effective SSH-policy check

After creating a THREDDS user, the script runs:

```bash
sshd -T -C user="$username",host=localhost,addr=127.0.0.1
```

and checks the relevant values.

Expected:

```text
pubkeyauthentication yes
passwordauthentication no
kbdinteractiveauthentication no
x11forwarding no
allowtcpforwarding no
allowagentforwarding no
forcecommand internal-sftp -d /lafi -u 022
chrootdirectory /srv/sftp
permittunnel no
```

This confirms the `Match Group lafi_sftp` configuration applies to the new account.

---

## 13. LAFO script

Script:

```text
sync_lafi_external_users_lafo.sh
```

Important constants:

```bash
DATA_ROOT="/lafi/srv_data"
UID_MIN=20000
UID_MAX=20999
LOG_FILE="/var/log/lafi-user-provisioning.log"
```

---

## 14. Existing-user path on LAFO

For an existing identity the script verifies:

- UID;
- GID;
- home field `/nonexistent`;
- shell `/usr/sbin/nologin`.

The LAFO account is an NFS identity, not an interactive login account.

---

## 15. LAFO account creation

Private group:

```bash
groupadd --gid "$gid" "$username"
```

User:

```bash
useradd \
    --uid "$uid" \
    --gid "$gid" \
    --no-create-home \
    --home-dir /nonexistent \
    --shell /usr/sbin/nologin \
    "$username"
```

The UID and GID must match THREDDS exactly.

---

## 16. LAFO data directory

Canonical path:

```text
/lafi/srv_data/<username>
```

Creation:

```bash
mkdir "$user_dir"
chown "$username:$username" "$user_dir"
chmod 2755 "$user_dir"
```

Expected result:

```text
drwxr-sr-x <username> <username>
```

Mode check:

```bash
stat -c '%a' "$user_dir"
```

must return:

```text
2755
```

Existing directories with unexpected ownership or permissions cause the script to stop rather than silently correcting them.

---

## 17. `date_added`

The LAFO script collects users successfully processed during a real run.

Only after the complete LAFO pass succeeds does it update blank `date_added` fields.

Existing dates are left unchanged.

Date format:

```text
YYYY-MM-DD
```

The update uses a temporary file followed by an atomic replacement. Existing registry ownership and mode are preserved.

---

## 18. Dry-run implementation

In dry-run mode, scripts still perform read-only checks:

- CSV parsing;
- numeric validation;
- account lookup;
- UID/GID conflict detection;
- existing-account verification;
- directory ownership/mode verification.

For missing objects they print:

```text
CREATE: would create ...
```

Dry-run does not:

- run `groupadd`;
- run `useradd`;
- create `.ssh`;
- write `authorized_keys`;
- create LAFO directories;
- modify `date_added`;
- create or append provisioning logs.

---

## 19. Provisioning log

Real runs use:

```text
/var/log/lafi-user-provisioning.log
```

Permissions:

```text
root:root
600
```

The logging helper records:

- ISO timestamp;
- hostname;
- target system;
- action;
- username;
- UID;
- GID;
- optional detail.

THREDDS actions include:

```text
VERIFY
CREATE
```

LAFO actions include:

```text
VERIFY
CREATE_IDENTITY
CREATE_DIRECTORY
```

Dry-run never writes the log.

---

## 20. Idempotency

The scripts are designed to be idempotent.

A repeated real run with an unchanged registry should:

- verify existing users;
- verify existing directories;
- make no account changes;
- make no directory changes;
- leave `date_added` unchanged.

This behavior was validated with the test users.

---

## 21. Registry synchronization model

Current process:

```text
LAFO external_users.csv
       |
       v
copy to THREDDS
       |
       v
external_users_working.csv
       |
       v
add new rows
       |
       v
VM dry-run + real run
       |
       v
copy to LAFO
       |
       v
LAFO dry-run + real run
       |
       v
date_added finalized
       |
       v
copy completed registry back to THREDDS
```

The registry copies should be identical at the beginning and end of a provisioning cycle.

A simple byte-level comparison can be performed with SHA-256 checksums after ensuring both files are compared in equivalent form.

---

## 22. Troubleshooting

### `ERROR: Registry not found: --dry-run`

Cause: an older script version without dry-run support is being used.

Check that the script contains:

```bash
if [[ "${1:-}" == "--dry-run" ]]; then
```

---

### UID already in use

Check:

```bash
getent passwd <UID>
```

Do not choose a new UID based only on one host. It must be free and consistent on both THREDDS and LAFO.

---

### GID already in use

Check:

```bash
getent group <GID>
```

---

### Existing user has wrong UID/GID

Inspect:

```bash
id <username>
getent passwd <username>
getent group <username>
```

Do not automatically modify an existing UID/GID without understanding file-ownership consequences.

---

### THREDDS user is not in `lafi_sftp`

Check:

```bash
id <username>
getent group lafi_sftp
```

The script intentionally stops rather than silently altering an existing account.

---

### Public key missing or mismatched

Inspect:

```bash
sudo cat /home/<username>/.ssh/authorized_keys
```

Permissions:

```bash
sudo ls -ld /home/<username>/.ssh
sudo ls -l /home/<username>/.ssh/authorized_keys
```

Expected:

```text
700 .ssh
600 authorized_keys
```

---

### SFTP authentication succeeds but session fails

Validate/reload SSH configuration:

```bash
sudo sshd -t
sudo systemctl reload ssh
```

Inspect the effective policy:

```bash
sudo sshd -T -C user=<username>,host=localhost,addr=127.0.0.1
```

Expected:

```text
forcecommand internal-sftp -d /lafi -u 022
chrootdirectory /srv/sftp
```

Changing only `authorized_keys` does not require an SSH reload.

---

### User cannot write to their own data directory

Check identity on both systems:

```bash
id <username>
```

Check LAFO directory:

```bash
ls -ldn /lafi/srv_data/<username>
```

Check THREDDS NFS view:

```bash
ls -ldn /srv/sftp/lafi/<username>
```

Numeric ownership should match the registry on both sides.

---

### Unexpected LAFO directory mode

Check:

```bash
stat -c '%a %U %G %n' /lafi/srv_data/<username>
```

Expected mode:

```text
2755
```

The script deliberately stops instead of silently correcting unexpected state.

---

### `date_added` remains blank

Confirm:

1. the real LAFO run was used, not dry-run;
2. the script completed successfully;
3. the registry was writable by root;
4. the username matches exactly.

Inspect:

```bash
grep '<username>' external_users.csv
```

---

### Provisioning log missing

The log is created only on real runs.

Check:

```bash
sudo ls -l /var/log/lafi-user-provisioning.log
sudo tail -n 50 /var/log/lafi-user-provisioning.log
```

---

## 23. Development rules

Future changes should preserve these design principles.

### Fail on unexpected state

Do not silently repair:

- UID mismatches;
- GID mismatches;
- incorrect account home/shell;
- unexpected directory ownership;
- unexpected directory mode.

### Preserve dry-run purity

Any new feature must ensure `--dry-run` does not modify:

- accounts;
- groups;
- directories;
- keys;
- registries;
- logs.

### Preserve idempotency

A second real run with the same registry should make no changes.

### Keep responsibilities separated

THREDDS manages:

- SFTP login identity;
- public keys;
- SSH group membership;
- SFTP policy.

LAFO manages:

- matching NFS identity;
- canonical storage directory;
- final `date_added`.

### Never store private keys

Only public SSH keys belong in the registry.

---

## 24. Possible future enhancements

Potential later additions include:

- automatic registry hash comparison;
- helper script for registry transfer/synchronization;
- dedicated admin group for registry editing;
- explicit username-format validation;
- stronger public-key validation using `ssh-keygen`;
- log rotation;
- user disable/decommission workflow;
- automated external SFTP connectivity test;
- automated consistency report across THREDDS and LAFO.

Any extension should preserve the core behavior:

```text
missing    -> create
correct    -> verify
unexpected -> stop
```