Render token scan as compact token table
This commit is contained in:
@@ -156,13 +156,11 @@ Podstawowe komendy tokenow:
|
|||||||
./rvctl tokens stats --repo ~/dev/workspace/rv/series/inf/03
|
./rvctl tokens stats --repo ~/dev/workspace/rv/series/inf/03
|
||||||
```
|
```
|
||||||
|
|
||||||
`tokens scan` wypisuje waska tabele. Dla jednego endpointu zobaczysz
|
`tokens scan` wypisuje tabele `tokens`: jeden wiersz na logiczny token.
|
||||||
osobny wiersz `source=repo` oraz osobny wiersz `source=tokens.json`.
|
`token_ref` laczy nazwe tokena z markerem po prawej stronie: `*` oznacza, ze
|
||||||
Ten sam numer `item` laczy oba wiersze w pare dla jednego endpointu.
|
remote i `tokens.json` sa zgodne oraz uprawnienia zostaly wczytane, `R` oznacza
|
||||||
Kolumna `token` pokazuje, gdzie jest token, a `sync` stawia `*` przy zrodle
|
token tylko w remote, a `S` token tylko w `tokens.json`. Kolumny `scope`, `org`
|
||||||
prawdy. Kolumna `id` pokazuje `r1` dla tokena w repo albo `t1` dla tokena w
|
i `repo` sa maskami uprawnien; bez `*` maja wartosc `?????` albo `????`.
|
||||||
`tokens.json`, bez pokazywania sekretu. Endpoint jest rozbity na `server`,
|
|
||||||
`scheme`, `host` i `port`.
|
|
||||||
|
|
||||||
## Fetch i switch
|
## Fetch i switch
|
||||||
|
|
||||||
|
|||||||
+21
-26
@@ -292,44 +292,40 @@ Przyklady:
|
|||||||
## `tokens scan`
|
## `tokens scan`
|
||||||
|
|
||||||
Skanuje remote URL-e w repo oraz lokalny `tokens.json`, laczy wpisy w pary po
|
Skanuje remote URL-e w repo oraz lokalny `tokens.json`, laczy wpisy w pary po
|
||||||
endpoincie i pokazuje, gdzie przechowywany jest token.
|
endpoincie, userze i wartosci tokena, a potem pokazuje jeden logiczny wiersz
|
||||||
|
na token.
|
||||||
|
|
||||||
Przelaczniki:
|
Przelaczniki:
|
||||||
|
|
||||||
- `--repo PATH`
|
- `--repo PATH`
|
||||||
Sciezka wewnatrz docelowego repo. Domyslnie biezacy katalog.
|
Sciezka wewnatrz docelowego repo. Domyslnie biezacy katalog.
|
||||||
- `--verbose`
|
|
||||||
Pokazuje dodatkowo sekcje `context` i `scan`.
|
|
||||||
|
|
||||||
Typowy wynik:
|
Typowy wynik:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
items
|
tokens
|
||||||
item source id server scheme host port credential repo_user store_user token sync
|
item server host org repo user remote token_ref token scope org repo
|
||||||
---- ----------- ---------- ------ ------ --------------- ----- ---------- --------- ---------- ----- ----
|
---- ------ ------------- --------- ----------- ---- ------ --------- ------------ ------ ------ ------
|
||||||
1 repo r1 gitea http 77.90.8.171 3001 auth u1 u1 yes
|
1 gitea 77.90.8.171 edu-tools rv-launcher u1 r1 t1 * e59cc...13be +---+ +++++ ++++
|
||||||
1 tokens.json t1 gitea http 77.90.8.171 3001 store u1 u1 yes *
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Jesli remote istnieje, ale ma URL bez `LOGIN:TOKEN@`, wynik bedzie mial
|
`token_ref` jest komorka stalej szerokosci: nazwa tokena jest po lewej, a marker
|
||||||
w wierszu `repo` wartosci `credential=plain` i `token=no`. Jezeli ten sam
|
po prawej. Marker `*` oznacza, ze token w remote i `tokens.json` jest zgodny
|
||||||
endpoint istnieje w `tokens.json`, drugi wiersz tego samego `item` pokaze
|
oraz uprawnienia zostaly wczytane. Marker `R` oznacza token tylko w remote, a
|
||||||
`source=tokens.json`, `id=t1`, `store_user=u1` i `token=yes`.
|
`S` token tylko w `tokens.json`.
|
||||||
|
|
||||||
Ten sam numer `item` oznacza jedna pare logiczna dla jednego endpointu. Wiersz
|
Maski uprawnien:
|
||||||
`source=repo` pokazuje stan remote, a wiersz `source=tokens.json` pokazuje stan
|
|
||||||
lokalnego store. Kolumna `sync` pokazuje `*` przy aktualnym zrodle prawdy dla
|
- `scope` ma pozycje `awrop`: `all`, `write:repository`, `read:repository`, organization scope, `public-only`
|
||||||
pary repo/store. Przy konflikcie pokazuje `!`.
|
- `org` ma pozycje `oawrc`: owner, admin, write, read, create repo
|
||||||
Kolumna `id` pokazuje identyfikator miejsca przechowywania tokena: dla repo jest
|
- `repo` ma pozycje `oawr`: owner, admin, write, read
|
||||||
to nazwa remota, na przyklad `r1`, a dla store nazwa tokenu, na przyklad `t1`.
|
- `+` oznacza wlaczone, `-` wylaczone, `?` nie wczytano, `!` blad wczytania
|
||||||
Endpoint jest rozbity na `server`, `scheme`, `host` i `port`.
|
|
||||||
|
|
||||||
Przyklad:
|
Przyklad:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./rvctl tokens scan
|
./rvctl tokens scan
|
||||||
./rvctl tokens scan --repo ~/dev/workspace/rv/series/inf/03
|
./rvctl tokens scan --repo ~/dev/workspace/rv/series/inf/03
|
||||||
./rvctl tokens scan --verbose
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## `tokens read`
|
## `tokens read`
|
||||||
@@ -385,15 +381,14 @@ item<TAB>value
|
|||||||
repo_root<TAB>...
|
repo_root<TAB>...
|
||||||
token_path<TAB>...
|
token_path<TAB>...
|
||||||
|
|
||||||
items
|
tokens
|
||||||
item source id server scheme host port credential repo_user store_user token sync
|
item server host org repo user remote token_ref token scope org repo
|
||||||
---- ----------- ---------- ------ ------ --------------- ----- ---------- --------- ---------- ----- ----
|
---- ------ ------------- --------- ----------- ---- ------ --------- ------------ ------ ------ ------
|
||||||
1 repo r1 gitea http 77.90.8.171 3001 plain u1 no
|
1 gitea 77.90.8.171 edu-tools rv-launcher u1 r1 t1 * e59cc...13be +---+ +++++ ++++
|
||||||
1 tokens.json t1 gitea http 77.90.8.171 3001 store u1 u1 yes *
|
|
||||||
|
|
||||||
status
|
status
|
||||||
item<TAB>value
|
item<TAB>value
|
||||||
store_ahead<TAB>1
|
in_sync<TAB>1
|
||||||
```
|
```
|
||||||
|
|
||||||
Przyklad:
|
Przyklad:
|
||||||
|
|||||||
+27
-28
@@ -88,49 +88,48 @@ Dzialanie:
|
|||||||
- laczy oba zrodla po endpoincie serwera
|
- laczy oba zrodla po endpoincie serwera
|
||||||
- zapisuje znalezione tokeny do `tokens.json`
|
- zapisuje znalezione tokeny do `tokens.json`
|
||||||
- zapisuje je per endpoint serwera
|
- zapisuje je per endpoint serwera
|
||||||
- wypisuje domyslnie tylko waska tabele `items`
|
- wypisuje tabele `tokens`
|
||||||
- z `--verbose` dopisuje diagnostyczne sekcje `context` oraz `scan`
|
|
||||||
|
|
||||||
Tabela `items` porownuje dwa zrodla dla tego samego endpointu. Ten sam `item`
|
Tabela `tokens` pokazuje jeden logiczny wiersz na token. Remote i `tokens.json`
|
||||||
moze miec dwa wiersze:
|
sa laczone po endpoincie, userze i wartosci tokena.
|
||||||
|
|
||||||
- `source=repo` - stan remote URL-i w repo
|
Znacznik w `token_ref`:
|
||||||
- `source=tokens.json` - stan lokalnego store tokenow
|
|
||||||
|
|
||||||
Dzieki temu widac, dla ktorego endpointu token jest w remote, a dla ktorego
|
- `*` - token jest w remote i `tokens.json`, uprawnienia zostaly wczytane
|
||||||
jest tylko w `tokens.json`.
|
- `R` - token jest tylko w remote
|
||||||
|
- `S` - token jest tylko w `tokens.json`
|
||||||
|
- `!` - blad wczytania uprawnien dla sparowanego tokena
|
||||||
|
|
||||||
Kolumny w tabeli `items`:
|
Kolumny w tabeli `tokens`:
|
||||||
|
|
||||||
- `item` - numer porownywanego endpointu
|
- `item` - numer wiersza tokena
|
||||||
- `source` - `repo` albo `tokens.json`
|
|
||||||
- `id` - identyfikator miejsca tokena: remote `r1` dla repo albo token `t1` dla `tokens.json`
|
|
||||||
- `server` - typ serwera, na przyklad `gitea`
|
- `server` - typ serwera, na przyklad `gitea`
|
||||||
- `scheme` - schemat URL, na przyklad `http`
|
|
||||||
- `host` - host endpointu, na przyklad `77.90.8.171`
|
- `host` - host endpointu, na przyklad `77.90.8.171`
|
||||||
- `port` - port endpointu, na przyklad `3001`
|
- `org` - organizacja z remote URL
|
||||||
- `credential` - typ wpisu: `auth`, `plain`, `store` albo `missing`
|
- `repo` - repo z remote URL
|
||||||
- `repo_user` - login odczytany z remote URL-i repo
|
- `user` - login wlasciciela tokena
|
||||||
- `store_user` - login odczytany z `tokens.json`
|
- `remote` - nazwa remota, na przyklad `r1`
|
||||||
- `token` - `yes` albo `no`, czyli czy dane zrodlo ma token dla endpointu
|
- `token_ref` - nazwa tokena z markerem po prawej stronie
|
||||||
- `sync` - `*` przy zrodle prawdy albo `!` przy konflikcie
|
- `token` - zamaskowana wartosc tokena
|
||||||
|
- `scope` - maska scope tokena `awrop`
|
||||||
|
- `org` - maska praw w organizacji `oawrc`
|
||||||
|
- `repo` - maska praw w repo `oawr`
|
||||||
|
|
||||||
Tabela nie wypisuje sekretu tokena. Jezeli token jest w remote, `id` pokazuje
|
Maski uprawnien:
|
||||||
nazwe remota, na przyklad `r1`. Jezeli token jest w `tokens.json`, `id`
|
|
||||||
pokazuje nazwe tokenu, na przyklad `t1`.
|
|
||||||
|
|
||||||
Zasada znacznika `sync`:
|
- `+` - flaga wlaczona
|
||||||
|
- `-` - flaga wylaczona
|
||||||
|
- `?` - nie wczytano, na przyklad dla `R` albo `S`
|
||||||
|
- `!` - blad wczytania
|
||||||
|
|
||||||
- `in_sync`, `store_only`, `store_ahead` - `*` przy `tokens.json`
|
Tabela nie wypisuje sekretu tokena wprost. Kolumna `token` pokazuje skrot, na
|
||||||
- `repo_only`, `repo_ahead` - `*` przy `repo`
|
przyklad `e59cc...13be`.
|
||||||
- `diverged` - `!`, bo nie ma jednoznacznego zrodla prawdy
|
|
||||||
|
|
||||||
Przyklad:
|
Przyklad:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./rvctl tokens scan
|
./rvctl tokens scan
|
||||||
./rvctl tokens scan --repo ~/dev/workspace/rv/series/inf/03
|
./rvctl tokens scan --repo ~/dev/workspace/rv/series/inf/03
|
||||||
./rvctl tokens scan --verbose
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### `tokens read`
|
### `tokens read`
|
||||||
@@ -170,7 +169,7 @@ Wynik ma ten sam model porownania co `tokens scan`, ale nie zapisuje zmian:
|
|||||||
- liczbe endpointow w store
|
- liczbe endpointow w store
|
||||||
- laczna unie endpointow
|
- laczna unie endpointow
|
||||||
- statusy zgodnosci, na przyklad `in_sync`, `store_ahead`, `repo_ahead`
|
- statusy zgodnosci, na przyklad `in_sync`, `store_ahead`, `repo_ahead`
|
||||||
- tabele `items`, w ktorej repo i `tokens.json` sa osobnymi wierszami tego samego itemu
|
- tabele `tokens`
|
||||||
|
|
||||||
Przyklad:
|
Przyklad:
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import base64
|
||||||
import datetime as dt
|
import datetime as dt
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
@@ -13,6 +14,8 @@ import subprocess
|
|||||||
import sys
|
import sys
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from urllib import error as urlerror
|
||||||
|
from urllib import request as urlrequest
|
||||||
from urllib.parse import quote, unquote, urlsplit, urlunsplit
|
from urllib.parse import quote, unquote, urlsplit, urlunsplit
|
||||||
|
|
||||||
|
|
||||||
@@ -244,6 +247,17 @@ def server_type_from_split(config: WorkspaceConfig, split_url) -> str:
|
|||||||
return "unknown"
|
return "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def repo_parts_from_url(remote_url: str) -> tuple[str, str]:
|
||||||
|
split_url = urlsplit(remote_url)
|
||||||
|
repo_path = unquote(split_url.path).strip("/")
|
||||||
|
if repo_path.endswith(".git"):
|
||||||
|
repo_path = repo_path[:-4]
|
||||||
|
path_parts = [part for part in repo_path.split("/") if part]
|
||||||
|
if len(path_parts) < 2:
|
||||||
|
return "", ""
|
||||||
|
return path_parts[0], path_parts[1]
|
||||||
|
|
||||||
|
|
||||||
def server_info_from_url(config: WorkspaceConfig, remote_url: str) -> dict | None:
|
def server_info_from_url(config: WorkspaceConfig, remote_url: str) -> dict | None:
|
||||||
split_url = urlsplit(remote_url)
|
split_url = urlsplit(remote_url)
|
||||||
if split_url.scheme not in {"http", "https"}:
|
if split_url.scheme not in {"http", "https"}:
|
||||||
@@ -252,12 +266,15 @@ def server_info_from_url(config: WorkspaceConfig, remote_url: str) -> dict | Non
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
endpoint, port_value = endpoint_from_split(split_url)
|
endpoint, port_value = endpoint_from_split(split_url)
|
||||||
|
org_name, repo_name = repo_parts_from_url(remote_url)
|
||||||
return {
|
return {
|
||||||
"endpoint": endpoint,
|
"endpoint": endpoint,
|
||||||
"type": server_type_from_split(config, split_url),
|
"type": server_type_from_split(config, split_url),
|
||||||
"scheme": split_url.scheme,
|
"scheme": split_url.scheme,
|
||||||
"host": split_url.hostname,
|
"host": split_url.hostname,
|
||||||
"port": port_value,
|
"port": port_value,
|
||||||
|
"org": org_name,
|
||||||
|
"repo": repo_name,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -455,8 +472,11 @@ def sync_tokens_from_repo(config: WorkspaceConfig, repo_path: Path) -> dict:
|
|||||||
"remote": remote_name,
|
"remote": remote_name,
|
||||||
"endpoint": "",
|
"endpoint": "",
|
||||||
"type": "unsupported",
|
"type": "unsupported",
|
||||||
|
"org": "",
|
||||||
|
"repo": "",
|
||||||
"url_kind": "unsupported",
|
"url_kind": "unsupported",
|
||||||
"user": "",
|
"user": "",
|
||||||
|
"token_value": "",
|
||||||
"result": "ignored",
|
"result": "ignored",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
@@ -471,8 +491,11 @@ def sync_tokens_from_repo(config: WorkspaceConfig, repo_path: Path) -> dict:
|
|||||||
"remote": remote_name,
|
"remote": remote_name,
|
||||||
"endpoint": server_info["endpoint"],
|
"endpoint": server_info["endpoint"],
|
||||||
"type": server_info["type"],
|
"type": server_info["type"],
|
||||||
|
"org": server_info["org"],
|
||||||
|
"repo": server_info["repo"],
|
||||||
"url_kind": "plain",
|
"url_kind": "plain",
|
||||||
"user": "",
|
"user": "",
|
||||||
|
"token_value": "",
|
||||||
"result": "no_credentials",
|
"result": "no_credentials",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
@@ -486,8 +509,11 @@ def sync_tokens_from_repo(config: WorkspaceConfig, repo_path: Path) -> dict:
|
|||||||
"remote": remote_name,
|
"remote": remote_name,
|
||||||
"endpoint": server_info["endpoint"],
|
"endpoint": server_info["endpoint"],
|
||||||
"type": server_info["type"],
|
"type": server_info["type"],
|
||||||
|
"org": server_info["org"],
|
||||||
|
"repo": server_info["repo"],
|
||||||
"url_kind": "auth",
|
"url_kind": "auth",
|
||||||
"user": user_name,
|
"user": user_name,
|
||||||
|
"token_value": token_value,
|
||||||
"result": "found",
|
"result": "found",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
@@ -575,6 +601,8 @@ def collect_repo_server_entries(config: WorkspaceConfig, repo_path: Path) -> tup
|
|||||||
"auth_urls": 0,
|
"auth_urls": 0,
|
||||||
"plain_urls": 0,
|
"plain_urls": 0,
|
||||||
"users": {},
|
"users": {},
|
||||||
|
"tokens": [],
|
||||||
|
"plain_remotes": [],
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
entry["remotes"].add(remote_name)
|
entry["remotes"].add(remote_name)
|
||||||
@@ -586,10 +614,26 @@ def collect_repo_server_entries(config: WorkspaceConfig, repo_path: Path) -> tup
|
|||||||
credentials = remote_credentials(remote_url)
|
credentials = remote_credentials(remote_url)
|
||||||
if credentials is None:
|
if credentials is None:
|
||||||
entry["plain_urls"] += 1
|
entry["plain_urls"] += 1
|
||||||
|
entry["plain_remotes"].append(
|
||||||
|
{
|
||||||
|
"remote": remote_name,
|
||||||
|
"org": server_info["org"],
|
||||||
|
"repo": server_info["repo"],
|
||||||
|
}
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
entry["auth_urls"] += 1
|
entry["auth_urls"] += 1
|
||||||
user_name, token_value = credentials
|
user_name, token_value = credentials
|
||||||
|
entry["tokens"].append(
|
||||||
|
{
|
||||||
|
"remote": remote_name,
|
||||||
|
"org": server_info["org"],
|
||||||
|
"repo": server_info["repo"],
|
||||||
|
"user": user_name,
|
||||||
|
"token_value": token_value,
|
||||||
|
}
|
||||||
|
)
|
||||||
user_entry = entry["users"].setdefault(user_name, {"tokens": {}})
|
user_entry = entry["users"].setdefault(user_name, {"tokens": {}})
|
||||||
tokens = user_entry.setdefault("tokens", {})
|
tokens = user_entry.setdefault("tokens", {})
|
||||||
if token_value not in tokens.values():
|
if token_value not in tokens.values():
|
||||||
@@ -657,7 +701,7 @@ def print_token_context(
|
|||||||
print(f"token_path\t{config.token_path}")
|
print(f"token_path\t{config.token_path}")
|
||||||
print(f"repo_endpoints\t{len(repo_servers)}")
|
print(f"repo_endpoints\t{len(repo_servers)}")
|
||||||
print(f"tokens_endpoints\t{len(store_servers)}")
|
print(f"tokens_endpoints\t{len(store_servers)}")
|
||||||
print(f"items\t{endpoint_count}")
|
print(f"tokens\t{endpoint_count}")
|
||||||
|
|
||||||
|
|
||||||
def print_status_counts(status_counts: dict[str, int]) -> None:
|
def print_status_counts(status_counts: dict[str, int]) -> None:
|
||||||
@@ -667,19 +711,21 @@ def print_status_counts(status_counts: dict[str, int]) -> None:
|
|||||||
print(f"{status_name}\t{status_counts[status_name]}")
|
print(f"{status_name}\t{status_counts[status_name]}")
|
||||||
|
|
||||||
|
|
||||||
TOKEN_ITEM_COLUMNS = [
|
TOKEN_REF_WIDTH = 9
|
||||||
("item", 4),
|
|
||||||
("source", 11),
|
TOKEN_COLUMNS = [
|
||||||
("id", 10),
|
("item", "item", 4),
|
||||||
("server", 6),
|
("server", "server", 6),
|
||||||
("scheme", 6),
|
("host", "host", 13),
|
||||||
("host", 15),
|
("owner", "org", 9),
|
||||||
("port", 5),
|
("repo_name", "repo", 11),
|
||||||
("credential", 10),
|
("user", "user", 4),
|
||||||
("repo_user", 9),
|
("remote", "remote", 6),
|
||||||
("store_user", 10),
|
("token_ref", "token_ref", TOKEN_REF_WIDTH),
|
||||||
("token", 5),
|
("token", "token", 12),
|
||||||
("sync", 4),
|
("scope_mask", "scope", 6),
|
||||||
|
("org_mask", "org", 6),
|
||||||
|
("repo_mask", "repo", 6),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -691,24 +737,23 @@ def fit_cell(value: str, width: int) -> str:
|
|||||||
return f"{value[:width - 3]}..."
|
return f"{value[:width - 3]}..."
|
||||||
|
|
||||||
|
|
||||||
def print_token_item_table(rows: list[dict[str, str]]) -> None:
|
def print_fixed_table(title: str, columns: list[tuple[str, str, int]], rows: list[dict[str, str]]) -> None:
|
||||||
print("items")
|
print(title)
|
||||||
print(" ".join(fit_cell(name, width) for name, width in TOKEN_ITEM_COLUMNS).rstrip())
|
print(" ".join(fit_cell(label, width) for _, label, width in columns).rstrip())
|
||||||
print(" ".join("-" * width for _, width in TOKEN_ITEM_COLUMNS).rstrip())
|
print(" ".join("-" * width for _, _, width in columns).rstrip())
|
||||||
for row in rows:
|
for row in rows:
|
||||||
print(" ".join(fit_cell(row.get(name, ""), width) for name, width in TOKEN_ITEM_COLUMNS).rstrip())
|
print(" ".join(fit_cell(row.get(name, ""), width) for name, _, width in columns).rstrip())
|
||||||
|
|
||||||
|
|
||||||
def sync_mark(status_name: str, source: str) -> str:
|
def short_secret(secret_value: str) -> str:
|
||||||
if status_name == "diverged":
|
if len(secret_value) <= 12:
|
||||||
return "!"
|
return mask_secret(secret_value)
|
||||||
if status_name in {"in_sync", "store_only", "store_ahead"}:
|
return f"{secret_value[:5]}...{secret_value[-4:]}"
|
||||||
truth_source = "tokens.json"
|
|
||||||
elif status_name in {"repo_only", "repo_ahead"}:
|
|
||||||
truth_source = "repo"
|
def format_token_ref(token_name: str, marker: str) -> str:
|
||||||
else:
|
name_width = TOKEN_REF_WIDTH - 2
|
||||||
truth_source = ""
|
return f"{fit_cell(token_name, name_width)}{marker:>2}"
|
||||||
return "*" if source == truth_source else ""
|
|
||||||
|
|
||||||
|
|
||||||
def users_label(server_entry: dict | None) -> str:
|
def users_label(server_entry: dict | None) -> str:
|
||||||
@@ -741,6 +786,191 @@ def endpoint_column_values(endpoint: str, repo_entry: dict | None, store_entry:
|
|||||||
return values
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def token_store_rows(store_servers: dict[str, dict]) -> list[dict[str, str]]:
|
||||||
|
rows: list[dict[str, str]] = []
|
||||||
|
for endpoint, server_entry in sorted(store_servers.items()):
|
||||||
|
users = server_entry.get("users", {})
|
||||||
|
for user_name in sorted(users):
|
||||||
|
user_entry = users[user_name]
|
||||||
|
if not isinstance(user_entry, dict):
|
||||||
|
continue
|
||||||
|
for token_name, token_value in sorted(user_entry.get("tokens", {}).items()):
|
||||||
|
if isinstance(token_value, str):
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"endpoint": endpoint,
|
||||||
|
"user": str(user_name),
|
||||||
|
"token_name": str(token_name),
|
||||||
|
"token_value": token_value,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def repo_token_rows(
|
||||||
|
endpoint_names: list[str],
|
||||||
|
repo_servers: dict[str, dict],
|
||||||
|
scan_rows: list[dict[str, str]] | None,
|
||||||
|
) -> list[dict[str, str]]:
|
||||||
|
if scan_rows is not None:
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"endpoint": row["endpoint"],
|
||||||
|
"remote": row["remote"],
|
||||||
|
"owner": row.get("org", ""),
|
||||||
|
"repo_name": row.get("repo", ""),
|
||||||
|
"user": row.get("user", ""),
|
||||||
|
"token_value": row.get("token_value", ""),
|
||||||
|
}
|
||||||
|
for row in scan_rows
|
||||||
|
if row.get("endpoint") and row.get("url_kind") == "auth" and row.get("token_value")
|
||||||
|
]
|
||||||
|
|
||||||
|
rows: list[dict[str, str]] = []
|
||||||
|
for endpoint in endpoint_names:
|
||||||
|
repo_entry = repo_servers.get(endpoint)
|
||||||
|
if repo_entry is None:
|
||||||
|
continue
|
||||||
|
for row in repo_entry.get("tokens", []):
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"endpoint": endpoint,
|
||||||
|
"remote": row.get("remote", ""),
|
||||||
|
"owner": row.get("org", ""),
|
||||||
|
"repo_name": row.get("repo", ""),
|
||||||
|
"user": row.get("user", ""),
|
||||||
|
"token_value": row.get("token_value", ""),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def fallback_project_by_endpoint(
|
||||||
|
endpoint_names: list[str],
|
||||||
|
repo_servers: dict[str, dict],
|
||||||
|
scan_rows: list[dict[str, str]] | None,
|
||||||
|
) -> dict[str, dict[str, str]]:
|
||||||
|
fallback: dict[str, dict[str, str]] = {}
|
||||||
|
for row in scan_rows or []:
|
||||||
|
endpoint = row.get("endpoint", "")
|
||||||
|
if endpoint and endpoint not in fallback:
|
||||||
|
fallback[endpoint] = {
|
||||||
|
"remote": row.get("remote", ""),
|
||||||
|
"owner": row.get("org", ""),
|
||||||
|
"repo_name": row.get("repo", ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
for endpoint in endpoint_names:
|
||||||
|
if endpoint in fallback:
|
||||||
|
continue
|
||||||
|
repo_entry = repo_servers.get(endpoint)
|
||||||
|
if repo_entry is None:
|
||||||
|
continue
|
||||||
|
for row in repo_entry.get("tokens", []) + repo_entry.get("plain_remotes", []):
|
||||||
|
fallback[endpoint] = {
|
||||||
|
"remote": row.get("remote", ""),
|
||||||
|
"owner": row.get("org", ""),
|
||||||
|
"repo_name": row.get("repo", ""),
|
||||||
|
}
|
||||||
|
break
|
||||||
|
return fallback
|
||||||
|
|
||||||
|
|
||||||
|
def api_get_json(api_url: str, token_value: str, user_name: str = "", basic: bool = False) -> object:
|
||||||
|
headers = {"Accept": "application/json"}
|
||||||
|
if basic:
|
||||||
|
raw_credentials = f"{user_name}:{token_value}".encode("utf-8")
|
||||||
|
headers["Authorization"] = "Basic " + base64.b64encode(raw_credentials).decode("ascii")
|
||||||
|
else:
|
||||||
|
headers["Authorization"] = f"token {token_value}"
|
||||||
|
request = urlrequest.Request(api_url, headers=headers)
|
||||||
|
with urlrequest.urlopen(request, timeout=5) as response:
|
||||||
|
return json.loads(response.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def scope_mask(scopes: list[str]) -> str:
|
||||||
|
scope_set = {scope.lower() for scope in scopes}
|
||||||
|
return "".join(
|
||||||
|
[
|
||||||
|
"+" if "all" in scope_set else "-",
|
||||||
|
"+" if "write:repository" in scope_set else "-",
|
||||||
|
"+" if "read:repository" in scope_set else "-",
|
||||||
|
"+" if {"read:organization", "write:organization"} & scope_set else "-",
|
||||||
|
"+" if "public-only" in scope_set else "-",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def org_permission_mask(org_permissions: dict) -> str:
|
||||||
|
return "".join(
|
||||||
|
[
|
||||||
|
"+" if org_permissions.get("is_owner") else "-",
|
||||||
|
"+" if org_permissions.get("is_admin") else "-",
|
||||||
|
"+" if org_permissions.get("can_write") else "-",
|
||||||
|
"+" if org_permissions.get("can_read") else "-",
|
||||||
|
"+" if org_permissions.get("can_create_repository") else "-",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def repo_permission_mask(repo_permissions: dict) -> str:
|
||||||
|
permission = str(repo_permissions.get("permission", "none")).lower()
|
||||||
|
levels = {
|
||||||
|
"owner": "++++",
|
||||||
|
"admin": "-+++",
|
||||||
|
"write": "--++",
|
||||||
|
"read": "---+",
|
||||||
|
}
|
||||||
|
return levels.get(permission, "----")
|
||||||
|
|
||||||
|
|
||||||
|
def load_gitea_authz(endpoint: str, user_name: str, token_name: str, token_value: str, owner: str, repo_name: str) -> dict:
|
||||||
|
api_base = endpoint.rstrip("/") + "/api/v1"
|
||||||
|
result = {
|
||||||
|
"ok": False,
|
||||||
|
"scope_mask": "!!!!!",
|
||||||
|
"org_mask": "!!!!!",
|
||||||
|
"repo_mask": "!!!!",
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
tokens_url = f"{api_base}/users/{quote(user_name, safe='')}/tokens"
|
||||||
|
token_list = api_get_json(tokens_url, token_value, user_name=user_name, basic=True)
|
||||||
|
token_last8 = token_value[-8:]
|
||||||
|
token_entry = next(
|
||||||
|
(
|
||||||
|
entry
|
||||||
|
for entry in token_list
|
||||||
|
if isinstance(entry, dict) and entry.get("token_last_eight") == token_last8
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
scopes = token_entry.get("scopes", []) if isinstance(token_entry, dict) else []
|
||||||
|
if not isinstance(scopes, list):
|
||||||
|
scopes = []
|
||||||
|
result["scope_mask"] = scope_mask([str(scope) for scope in scopes])
|
||||||
|
|
||||||
|
if owner:
|
||||||
|
org_url = f"{api_base}/users/{quote(user_name, safe='')}/orgs/{quote(owner, safe='')}/permissions"
|
||||||
|
org_permissions = api_get_json(org_url, token_value)
|
||||||
|
if isinstance(org_permissions, dict):
|
||||||
|
result["org_mask"] = org_permission_mask(org_permissions)
|
||||||
|
|
||||||
|
if owner and repo_name:
|
||||||
|
repo_url = (
|
||||||
|
f"{api_base}/repos/{quote(owner, safe='')}/{quote(repo_name, safe='')}"
|
||||||
|
f"/collaborators/{quote(user_name, safe='')}/permission"
|
||||||
|
)
|
||||||
|
repo_permissions = api_get_json(repo_url, token_value)
|
||||||
|
if isinstance(repo_permissions, dict):
|
||||||
|
result["repo_mask"] = repo_permission_mask(repo_permissions)
|
||||||
|
|
||||||
|
result["ok"] = True
|
||||||
|
return result
|
||||||
|
except (OSError, ValueError, urlerror.URLError, urlerror.HTTPError):
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
def print_token_item_rows(
|
def print_token_item_rows(
|
||||||
endpoint_names: list[str],
|
endpoint_names: list[str],
|
||||||
repo_servers: dict[str, dict],
|
repo_servers: dict[str, dict],
|
||||||
@@ -749,132 +979,100 @@ def print_token_item_rows(
|
|||||||
) -> dict[str, int]:
|
) -> dict[str, int]:
|
||||||
status_names = ["in_sync", "repo_only", "store_only", "repo_ahead", "store_ahead", "diverged"]
|
status_names = ["in_sync", "repo_only", "store_only", "repo_ahead", "store_ahead", "diverged"]
|
||||||
status_counts = {status_name: 0 for status_name in status_names}
|
status_counts = {status_name: 0 for status_name in status_names}
|
||||||
scan_rows_by_endpoint: dict[str, list[dict[str, str]]] = {}
|
for endpoint in endpoint_names:
|
||||||
for row in scan_rows or []:
|
|
||||||
endpoint = row.get("endpoint", "")
|
|
||||||
if endpoint:
|
|
||||||
scan_rows_by_endpoint.setdefault(endpoint, []).append(row)
|
|
||||||
|
|
||||||
item_rows: list[dict[str, str]] = []
|
|
||||||
for index, endpoint in enumerate(endpoint_names, start=1):
|
|
||||||
item_id = str(index)
|
|
||||||
repo_entry = repo_servers.get(endpoint)
|
repo_entry = repo_servers.get(endpoint)
|
||||||
store_entry = store_servers.get(endpoint)
|
store_entry = store_servers.get(endpoint)
|
||||||
status_name, _, _ = compare_server_entries(repo_entry, store_entry)
|
status_name, _, _ = compare_server_entries(repo_entry, store_entry)
|
||||||
if status_name in status_counts:
|
if status_name in status_counts:
|
||||||
status_counts[status_name] += 1
|
status_counts[status_name] += 1
|
||||||
endpoint_values = endpoint_column_values(endpoint, repo_entry, store_entry)
|
|
||||||
repo_users_label = users_label(repo_entry)
|
|
||||||
store_users_label = users_label(store_entry)
|
|
||||||
|
|
||||||
repo_rows = scan_rows_by_endpoint.get(endpoint, [])
|
store_rows = token_store_rows(store_servers)
|
||||||
if repo_rows:
|
store_by_value: dict[tuple[str, str, str], dict[str, str]] = {}
|
||||||
for row in repo_rows:
|
for row in store_rows:
|
||||||
token_label = "remote" if row["url_kind"] == "auth" else ""
|
store_by_value.setdefault((row["endpoint"], row["user"], row["token_value"]), row)
|
||||||
item_rows.append(
|
|
||||||
{
|
endpoint_values_by_name = {
|
||||||
"item": item_id,
|
endpoint: endpoint_column_values(endpoint, repo_servers.get(endpoint), store_servers.get(endpoint))
|
||||||
"source": "repo",
|
for endpoint in endpoint_names
|
||||||
"id": row["remote"],
|
|
||||||
**endpoint_values,
|
|
||||||
"credential": row["url_kind"],
|
|
||||||
"repo_user": row["user"],
|
|
||||||
"store_user": store_users_label,
|
|
||||||
"token": "yes" if token_label else "no",
|
|
||||||
"sync": sync_mark(status_name, "repo"),
|
|
||||||
}
|
}
|
||||||
)
|
project_fallback = fallback_project_by_endpoint(endpoint_names, repo_servers, scan_rows)
|
||||||
elif repo_entry is None:
|
authz_cache: dict[tuple[str, str, str, str, str], dict] = {}
|
||||||
item_rows.append(
|
used_store_keys: set[tuple[str, str, str, str]] = set()
|
||||||
|
token_rows: list[dict[str, str]] = []
|
||||||
|
|
||||||
|
def append_token_row(
|
||||||
|
endpoint: str,
|
||||||
|
user_name: str,
|
||||||
|
remote_name: str,
|
||||||
|
token_name: str,
|
||||||
|
token_value: str,
|
||||||
|
marker: str,
|
||||||
|
owner: str,
|
||||||
|
repo_name: str,
|
||||||
|
authz: dict | None = None,
|
||||||
|
) -> None:
|
||||||
|
endpoint_values = endpoint_values_by_name.get(endpoint, endpoint_column_values(endpoint, None, None))
|
||||||
|
token_rows.append(
|
||||||
{
|
{
|
||||||
"item": item_id,
|
"item": str(len(token_rows) + 1),
|
||||||
"source": "repo",
|
"server": endpoint_values["server"],
|
||||||
**endpoint_values,
|
"host": endpoint_values["host"],
|
||||||
"credential": "missing",
|
"owner": owner,
|
||||||
"store_user": store_users_label,
|
"repo_name": repo_name,
|
||||||
"token": "no",
|
"user": user_name,
|
||||||
"sync": sync_mark(status_name, "repo"),
|
"remote": remote_name,
|
||||||
}
|
"token_ref": format_token_ref(token_name, marker),
|
||||||
)
|
"token": short_secret(token_value),
|
||||||
else:
|
"scope_mask": authz.get("scope_mask", "?????") if authz else "?????",
|
||||||
repo_kind, repo_result = repo_result_for_entry(repo_entry)
|
"org_mask": authz.get("org_mask", "?????") if authz else "?????",
|
||||||
remote_names = ",".join(sorted(repo_entry.get("remotes", set())))
|
"repo_mask": authz.get("repo_mask", "????") if authz else "????",
|
||||||
repo_users = token_name_rows(repo_entry.get("users", {}))
|
|
||||||
if repo_users:
|
|
||||||
for user_name, _ in repo_users:
|
|
||||||
token_label = "remote" if repo_kind == "auth" else ""
|
|
||||||
item_rows.append(
|
|
||||||
{
|
|
||||||
"item": item_id,
|
|
||||||
"source": "repo",
|
|
||||||
"id": remote_names,
|
|
||||||
**endpoint_values,
|
|
||||||
"credential": repo_kind,
|
|
||||||
"repo_user": user_name,
|
|
||||||
"store_user": store_users_label,
|
|
||||||
"token": "yes" if token_label else "no",
|
|
||||||
"sync": sync_mark(status_name, "repo"),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
item_rows.append(
|
|
||||||
{
|
|
||||||
"item": item_id,
|
|
||||||
"source": "repo",
|
|
||||||
"id": remote_names,
|
|
||||||
**endpoint_values,
|
|
||||||
"credential": repo_kind,
|
|
||||||
"store_user": store_users_label,
|
|
||||||
"token": "yes" if repo_result == "present" else "no",
|
|
||||||
"sync": sync_mark(status_name, "repo"),
|
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
if store_entry is None:
|
for row in repo_token_rows(endpoint_names, repo_servers, scan_rows):
|
||||||
item_rows.append(
|
endpoint = row["endpoint"]
|
||||||
{
|
token_value = row["token_value"]
|
||||||
"item": item_id,
|
store_row = store_by_value.get((endpoint, row["user"], token_value))
|
||||||
"source": "tokens.json",
|
token_name = store_row["token_name"] if store_row else ""
|
||||||
**endpoint_values,
|
marker = "R"
|
||||||
"credential": "missing",
|
authz = None
|
||||||
"repo_user": repo_users_label,
|
if store_row:
|
||||||
"token": "no",
|
used_store_keys.add((endpoint, store_row["user"], store_row["token_name"], token_value))
|
||||||
"sync": sync_mark(status_name, "tokens.json"),
|
authz_key = (endpoint, row["user"], token_value, row["owner"], row["repo_name"])
|
||||||
}
|
authz = authz_cache.get(authz_key)
|
||||||
|
if authz is None:
|
||||||
|
authz = load_gitea_authz(endpoint, row["user"], token_name, token_value, row["owner"], row["repo_name"])
|
||||||
|
authz_cache[authz_key] = authz
|
||||||
|
marker = "*" if authz.get("ok") else "!"
|
||||||
|
append_token_row(
|
||||||
|
endpoint,
|
||||||
|
row["user"],
|
||||||
|
row["remote"],
|
||||||
|
token_name,
|
||||||
|
token_value,
|
||||||
|
marker,
|
||||||
|
row["owner"],
|
||||||
|
row["repo_name"],
|
||||||
|
authz if marker == "*" else None,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
for row in store_rows:
|
||||||
|
store_key = (row["endpoint"], row["user"], row["token_name"], row["token_value"])
|
||||||
|
if store_key in used_store_keys:
|
||||||
continue
|
continue
|
||||||
|
fallback = project_fallback.get(row["endpoint"], {})
|
||||||
store_users = token_name_rows(store_entry.get("users", {}))
|
append_token_row(
|
||||||
if not store_users:
|
row["endpoint"],
|
||||||
item_rows.append(
|
row["user"],
|
||||||
{
|
"",
|
||||||
"item": item_id,
|
row["token_name"],
|
||||||
"source": "tokens.json",
|
row["token_value"],
|
||||||
**endpoint_values,
|
"S",
|
||||||
"credential": "store",
|
fallback.get("owner", ""),
|
||||||
"repo_user": repo_users_label,
|
fallback.get("repo_name", ""),
|
||||||
"token": "no",
|
|
||||||
"sync": sync_mark(status_name, "tokens.json"),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
|
|
||||||
for user_name, token_name in store_users:
|
|
||||||
item_rows.append(
|
|
||||||
{
|
|
||||||
"item": item_id,
|
|
||||||
"source": "tokens.json",
|
|
||||||
"credential": "store",
|
|
||||||
"id": token_name,
|
|
||||||
**endpoint_values,
|
|
||||||
"repo_user": repo_users_label,
|
|
||||||
"store_user": user_name,
|
|
||||||
"token": "yes" if token_name else "no",
|
|
||||||
"sync": sync_mark(status_name, "tokens.json"),
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
|
|
||||||
print_token_item_table(item_rows)
|
print_fixed_table("tokens", TOKEN_COLUMNS, token_rows)
|
||||||
return status_counts
|
return status_counts
|
||||||
|
|
||||||
|
|
||||||
@@ -972,22 +1170,6 @@ def run_tokens_scan(config: WorkspaceConfig, args: argparse.Namespace) -> None:
|
|||||||
token_data = load_token_store(config)
|
token_data = load_token_store(config)
|
||||||
store_servers = token_data.get("servers", {})
|
store_servers = token_data.get("servers", {})
|
||||||
endpoint_names = sorted(set(repo_servers) | set(store_servers))
|
endpoint_names = sorted(set(repo_servers) | set(store_servers))
|
||||||
|
|
||||||
if args.verbose:
|
|
||||||
print_token_context(repo_root, config, repo_servers, store_servers, len(endpoint_names))
|
|
||||||
print()
|
|
||||||
print("scan")
|
|
||||||
print("item\tvalue")
|
|
||||||
print(f"remotes\t{report['remotes']}")
|
|
||||||
print(f"urls\t{report['urls']}")
|
|
||||||
print(f"scanned\t{report['scanned']}")
|
|
||||||
print(f"auth_urls\t{report['auth_urls']}")
|
|
||||||
print(f"plain_urls\t{report['plain_urls']}")
|
|
||||||
print(f"unsupported_urls\t{report['unsupported_urls']}")
|
|
||||||
print(f"found\t{report['found']}")
|
|
||||||
print(f"added\t{report['added']}")
|
|
||||||
print(f"servers\t{report['servers']}")
|
|
||||||
print()
|
|
||||||
print_token_item_rows(endpoint_names, repo_servers, store_servers, report["remote_rows"])
|
print_token_item_rows(endpoint_names, repo_servers, store_servers, report["remote_rows"])
|
||||||
|
|
||||||
|
|
||||||
@@ -1386,7 +1568,7 @@ def print_main_overview(config_path: Path) -> None:
|
|||||||
print_table(
|
print_table(
|
||||||
["step", "command", "result"],
|
["step", "command", "result"],
|
||||||
[
|
[
|
||||||
["1", "./rvctl tokens scan", "pair repo remotes with tokens.json entries"],
|
["1", "./rvctl tokens scan", "print token table from repo remotes and tokens.json"],
|
||||||
["2", "./rvctl list-series", "choose a series"],
|
["2", "./rvctl list-series", "choose a series"],
|
||||||
["3", "./rvctl list-cards inf", "choose a card"],
|
["3", "./rvctl list-cards inf", "choose a card"],
|
||||||
["4", "./rvctl tmux-container inf <card>", "start the working container"],
|
["4", "./rvctl tmux-container inf <card>", "start the working container"],
|
||||||
@@ -1413,7 +1595,7 @@ def print_tokens_overview() -> None:
|
|||||||
print_table(
|
print_table(
|
||||||
["command", "common options", "direction", "purpose"],
|
["command", "common options", "direction", "purpose"],
|
||||||
[
|
[
|
||||||
["scan", "[--repo PATH]", "repo + tokens.json", "pair endpoints and show where token refs are stored"],
|
["scan", "[--repo PATH]", "repo + tokens.json", "print token table with remote/store marker and auth masks"],
|
||||||
["read", "[--server ENDPOINT]", "tokens.json", "show servers, users and token names"],
|
["read", "[--server ENDPOINT]", "tokens.json", "show servers, users and token names"],
|
||||||
["stats", "[--repo PATH]", "repo + tokens.json", "compare remote URLs with local token store"],
|
["stats", "[--repo PATH]", "repo + tokens.json", "compare remote URLs with local token store"],
|
||||||
["write", "--remote R [--replace]", "tokens.json -> repo", "write selected token into a remote URL"],
|
["write", "--remote R [--replace]", "tokens.json -> repo", "write selected token into a remote URL"],
|
||||||
@@ -1524,7 +1706,6 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
help="Scan repo remotes and tokens.json, then print paired endpoint rows.",
|
help="Scan repo remotes and tokens.json, then print paired endpoint rows.",
|
||||||
)
|
)
|
||||||
add_repo_option(tokens_scan_parser)
|
add_repo_option(tokens_scan_parser)
|
||||||
tokens_scan_parser.add_argument("--verbose", action="store_true", help="Print context and scan counters before item rows.")
|
|
||||||
|
|
||||||
tokens_read_parser = tokens_subparsers.add_parser(
|
tokens_read_parser = tokens_subparsers.add_parser(
|
||||||
"read",
|
"read",
|
||||||
|
|||||||
Reference in New Issue
Block a user