Phase 3: scripts/kb-objects-map.py + audit/objects-map.json + projects/_index.md
Авто-генератор реестра: парсит netbird-inventory + frontmatter каждого проекта, выводит JSON для бота и человекочитаемый index с wiki-ссылками. Пока 16 проектов / 38 orphan-пиров без своих карточек — выявленные дыры станут input для Фазы 4 (stub-генератора). Скрипт идемпотентный, без deps (pure stdlib), запуск: cd ~/knowledge-base && python3 scripts/kb-objects-map.py Парсер обрабатывает offline-таблицу netbird (другой порядок колонок), normalize ye→e уравнивает Знаменское/Znamenskoe. Source of truth — frontmatter каждого проекта; добавление aliases/owner/region там сразу подхватится при следующем run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
dttb
1149
audit/objects-map.json
Normal file
1149
audit/objects-map.json
Normal file
File diff suppressed because it is too large
Load Diff
85
projects/_index.md
Normal file
85
projects/_index.md
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
---
|
||||||
|
date: 2026-05-06
|
||||||
|
type: index
|
||||||
|
source: scripts/kb-objects-map.py
|
||||||
|
tags: [index, registry, objects, netbird]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Реестр объектов и netbird-пиров
|
||||||
|
|
||||||
|
Авто-сгенерировано `2026-05-06T16:16` из [[dttb/netbird-inventory]] + frontmatter в `projects/`.
|
||||||
|
**Не править вручную** — перепишется. Источник правды — frontmatter в каждом README.
|
||||||
|
|
||||||
|
- Проектов: **16**, из них с netbird-привязкой: **4**
|
||||||
|
- NetBird-пиров без projects-страницы: **38** (TODO — создать стабы)
|
||||||
|
|
||||||
|
## Проекты с netbird-привязкой
|
||||||
|
|
||||||
|
| ID | Имена | NetBird IP | OS | Город | Файл | Статус |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| dttb | dttb | 100.70.12.3 | Windows Server 2025 | Istra | [[projects/dttb/README]] | unknown |
|
||||||
|
| lipki | Lipki, OpenWrt_Lipki, PSP_Network, lipki, Антон | 100.70.35.234 | OpenWrt 24.10.3 | Istra | [[projects/lipki/README]] | active |
|
||||||
|
| niikn | niikn | 100.70.117.21, 100.70.120.229 | Debian GNU/Linux 12, Ubuntu 24.04 | Istra | [[projects/niikn/README]] | unknown |
|
||||||
|
| znamenskoye | znamenskoye | 100.70.54.204, 100.70.100.155 | Debian GNU/Linux 11, OpenWrt 24.10.3 | Helsinki, Moscow | — | unknown |
|
||||||
|
|
||||||
|
## Проекты без netbird-привязки
|
||||||
|
|
||||||
|
| ID | Тип | Файл | Статус |
|
||||||
|
|---|---|---|---|
|
||||||
|
| all-projects-summary | project-note | [[projects/all-projects-summary]] | unknown |
|
||||||
|
| bitrix-sites | project-note | [[projects/bitrix-sites]] | unknown |
|
||||||
|
| clawdbot-bots | project-note | [[projects/clawdbot-bots]] | unknown |
|
||||||
|
| glavtorg | project | [[projects/glavtorg/README]] | unknown |
|
||||||
|
| homelab-proxmox | project-note | [[projects/homelab-proxmox]] | unknown |
|
||||||
|
| infrastructure-overview | project-note | [[projects/infrastructure-overview]] | unknown |
|
||||||
|
| krasnogorsk | project | [[projects/krasnogorsk/README]] | unknown |
|
||||||
|
| mmfb | project | — | unknown |
|
||||||
|
| nextcloud | project-note | [[projects/nextcloud]] | unknown |
|
||||||
|
| unresolved-issues | project-note | [[projects/unresolved-issues]] | unknown |
|
||||||
|
| video-surveillance | project-note | [[projects/video-surveillance]] | unknown |
|
||||||
|
| zelenograd | project | [[projects/zelenograd/README]] | unknown |
|
||||||
|
|
||||||
|
## NetBird-пиры без projects-страницы — TODO
|
||||||
|
|
||||||
|
Эти пиры есть в инвентаре, но у них нет своей карточки в `projects/`. Бот не сможет ответить на «найди мне X» — нет файла. Нужно создать стабы (Фаза 4 плана).
|
||||||
|
|
||||||
|
| Имя в NetBird | IP | OS | Город | Версия |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `89-111-140-86.swtest.ru` | 100.70.93.36 | Ubuntu 24.04 | Moscow | 0.65.1 |
|
||||||
|
| `Cups-Server` | 100.70.100.82 | Ubuntu 22.04 | | |
|
||||||
|
| `DESKTOP-2IOQS54` | 100.70.82.83 | Windows 10 | Saransk | 0.50.3 |
|
||||||
|
| `DESKTOP-5RGAUUG` | 100.70.221.26 | Windows 11 | Istra | |
|
||||||
|
| `DESKTOP-9VJ949T скоморохова` | 100.70.44.183 | Windows 10 | St Petersburg | |
|
||||||
|
| `DESKTOP-AGBMLPN` | 100.70.0.106 | Windows 11 | Helsinki | 0.66.2 |
|
||||||
|
| `DESKTOP-HL0BB05` | 100.70.235.80 | Windows 11 | Lipetsk | 0.59.7 |
|
||||||
|
| `DESKTOP-IC5A0K2 M.Maul` | 100.70.178.190 | Windows 10 | | |
|
||||||
|
| `DESKTOP-LBD73OR` | 100.70.78.170 | Windows 11 | Astana | |
|
||||||
|
| `KOMPUTER` | 100.70.83.120 | Windows 11 | St Petersburg | |
|
||||||
|
| `Kolyadenko` | 100.70.146.58 | Windows 10 | | |
|
||||||
|
| `Kripto-ARM` | 100.70.145.223 | Windows 11 | | 0.54.0 |
|
||||||
|
| `LAPTOP-3IR5EA9J` | 100.70.149.179 | Windows 11 | Mérida | |
|
||||||
|
| `MacBook-Pro` | 100.70.242.212 | Darwin 26.3.1 | Istra | 0.65.3 |
|
||||||
|
| `MacBook-Pro-Vera.local` | 100.70.252.228 | Darwin 26.3.1 | St Petersburg | |
|
||||||
|
| `MastaNotebook` | 100.70.116.166 | Windows 11 | Moscow | |
|
||||||
|
| `OpenWrt 1` | 100.70.239.211 | OpenWrt 24.10.3 | Moscow | 0.50.2 |
|
||||||
|
| `OpenWrt_4` | 100.70.235.2 | OpenWrt 24.10.3 | Moscow | 0.50.2 |
|
||||||
|
| `OpenWrt Benilux` | 100.70.207.97 | OpenWrt 24.10.3 | Istra | 0.59.13 |
|
||||||
|
| `OpenWrt_Sergey` | 100.70.110.164 | OpenWrt 24.10.3 | Odintsovo | 0.59.12 |
|
||||||
|
| `OpenWrt_ohothozyistvo` | 100.70.63.67 | OpenWrt 21.02.1 | Istra | 0.36.5 |
|
||||||
|
| `OpenWrt Вишневый сад ( Константин )` | 100.70.152.137 | OpenWrt 24.10.3 | Moscow | 0.59.12 |
|
||||||
|
| `WIN-BC0OTBOBBCH` | 100.70.181.152 | Windows Server 2025 | Moscow | |
|
||||||
|
| `clawdbot` | 100.70.219.93 | Debian GNU/Linux 12 | Istra | |
|
||||||
|
| `clawdbot-1` | 100.70.200.150 | Debian GNU/Linux 12 | Istra | 0.65.2 |
|
||||||
|
| `cloud` | 100.70.152.70 | Ubuntu 24.04 | | 0.66.3 |
|
||||||
|
| `code-server` | 100.70.92.138 | Ubuntu 24.04 | Istra | 0.66.0 |
|
||||||
|
| `finland5870.com` | 100.70.0.15 | Ubuntu 22.04 | Helsinki | 0.62.3 |
|
||||||
|
| `iPad-batlaew` | 100.70.211.159 | iPadOS 18.6.2 | Istra | |
|
||||||
|
| `iPhone-batlaew` | 100.70.57.167 | iOS 26.3.1 | Istra | |
|
||||||
|
| `iPhone-netbird` | 100.70.18.13 | iOS 26.3.1 | | |
|
||||||
|
| `kasm` | 100.70.121.49 | Ubuntu 24.04 | | |
|
||||||
|
| `netbird-29-HP` | 100.70.137.181 | Ubuntu 24.04 | Moscow | 0.64.5 |
|
||||||
|
| `pdm` | 100.70.128.10 | Debian GNU/Linux 13 | Istra | |
|
||||||
|
| `pve LionART` | 100.70.128.49 | Debian GNU/Linux 12 | Istra | 0.59.6 |
|
||||||
|
| `pve ded_mozay` | 100.70.121.235 | Debian GNU/Linux 13 | Istra | 0.28.6 |
|
||||||
|
| `rustdeskserver` | 100.70.191.161 | Debian GNU/Linux 12 | Istra | 0.66.0 |
|
||||||
|
| `Денис Тихая` | 100.70.155.107 | Windows 10 | | |
|
||||||
242
scripts/kb-objects-map.py
Normal file
242
scripts/kb-objects-map.py
Normal file
@@ -0,0 +1,242 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""kb-objects-map — собирает машиночитаемый реестр объектов и хостов.
|
||||||
|
|
||||||
|
Источники:
|
||||||
|
projects/dttb/netbird-inventory.md — netbird-пиры (источник правды по железу)
|
||||||
|
projects/<dir>/README.md — frontmatter каждого проекта (aliases, tags, status)
|
||||||
|
projects/<file>.md — singleton-проекты
|
||||||
|
|
||||||
|
Output:
|
||||||
|
audit/objects-map.json — структура для бота / structured-поиска
|
||||||
|
projects/_index.md — человеко-читаемый индекс с wiki-ссылками
|
||||||
|
|
||||||
|
Запускать вручную или cron:
|
||||||
|
cd ~/knowledge-base && python3 scripts/kb-objects-map.py
|
||||||
|
"""
|
||||||
|
from datetime import date, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
|
||||||
|
VAULT = Path(__file__).resolve().parent.parent
|
||||||
|
INV = VAULT / "projects/dttb/netbird-inventory.md"
|
||||||
|
JSON_OUT = VAULT / "audit/objects-map.json"
|
||||||
|
MD_OUT = VAULT / "projects/_index.md"
|
||||||
|
|
||||||
|
|
||||||
|
def parse_frontmatter(text: str) -> dict:
|
||||||
|
m = re.match(r"^---\n(.+?)\n---\n", text, re.S)
|
||||||
|
if not m:
|
||||||
|
return {}
|
||||||
|
fm = {}
|
||||||
|
for line in m.group(1).splitlines():
|
||||||
|
if ":" not in line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
k, _, v = line.partition(":")
|
||||||
|
k, v = k.strip(), v.strip()
|
||||||
|
if v.startswith("[") and v.endswith("]"):
|
||||||
|
v = [x.strip().strip("\"'") for x in v[1:-1].split(",") if x.strip()]
|
||||||
|
elif v.startswith('"') and v.endswith('"'):
|
||||||
|
v = v[1:-1]
|
||||||
|
fm[k] = v
|
||||||
|
return fm
|
||||||
|
|
||||||
|
|
||||||
|
ROW_RE = re.compile(
|
||||||
|
r"^\|\s*([^|]+?)\s*\|\s*(\d+\.\d+\.\d+\.\d+)\s*\|\s*([^|]*?)\s*\|\s*([^|]*?)\s*\|\s*([^|]*?)\s*\|"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_netbird(path: Path) -> list[dict]:
|
||||||
|
"""Парсит online + offline-таблицы netbird-inventory.md."""
|
||||||
|
if not path.exists():
|
||||||
|
return []
|
||||||
|
text = path.read_text()
|
||||||
|
parts = re.split(r"^##\s+(?:Оффлайн|Offline)\b.*$", text, maxsplit=1, flags=re.M)
|
||||||
|
online_text, offline_text = parts[0], parts[1] if len(parts) > 1 else ""
|
||||||
|
|
||||||
|
peers = []
|
||||||
|
for line in online_text.splitlines():
|
||||||
|
m = ROW_RE.match(line)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
name = m.group(1).strip()
|
||||||
|
if name in ("Имя", "---"):
|
||||||
|
continue
|
||||||
|
peers.append({
|
||||||
|
"name": name, "ip": m.group(2).strip(),
|
||||||
|
"os": m.group(3).strip(), "city": m.group(4).strip(),
|
||||||
|
"version": m.group(5).strip(), "online": True,
|
||||||
|
})
|
||||||
|
# offline-таблица: | Имя | IP | ОС | Последний раз онлайн | Город |
|
||||||
|
for line in offline_text.splitlines():
|
||||||
|
m = ROW_RE.match(line)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
name = m.group(1).strip()
|
||||||
|
if name in ("Имя", "---"):
|
||||||
|
continue
|
||||||
|
peers.append({
|
||||||
|
"name": name, "ip": m.group(2).strip(),
|
||||||
|
"os": m.group(3).strip(),
|
||||||
|
"last_seen": m.group(4).strip(), "city": m.group(5).strip(),
|
||||||
|
"version": "", "online": False,
|
||||||
|
})
|
||||||
|
return peers
|
||||||
|
|
||||||
|
|
||||||
|
def _norm(s: str) -> str:
|
||||||
|
"""Нормализация для нечёткого сравнения. ye→e уравнивает Знаменское/Znamenskoe."""
|
||||||
|
return (s or "").lower().replace("ye", "e").replace(" ", "").replace("_", "").replace("-", "")
|
||||||
|
|
||||||
|
|
||||||
|
def name_match(peer_name: str, candidates: list[str]) -> bool:
|
||||||
|
pn = _norm(peer_name)
|
||||||
|
if not pn:
|
||||||
|
return False
|
||||||
|
for c in candidates:
|
||||||
|
cn = _norm(c)
|
||||||
|
if cn and (cn in pn or pn in cn):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
peers = parse_netbird(INV)
|
||||||
|
objects = []
|
||||||
|
seen_files = set()
|
||||||
|
|
||||||
|
project_dirs = sorted(p for p in (VAULT / "projects").iterdir() if p.is_dir())
|
||||||
|
for d in project_dirs:
|
||||||
|
readme = d / "README.md"
|
||||||
|
fm = parse_frontmatter(readme.read_text()) if readme.exists() else {}
|
||||||
|
rel_file = f"projects/{d.name}/README.md" if readme.exists() else None
|
||||||
|
aliases = fm.get("aliases", [])
|
||||||
|
if isinstance(aliases, str):
|
||||||
|
aliases = [aliases]
|
||||||
|
names = sorted({d.name, *aliases})
|
||||||
|
nb = [p for p in peers if name_match(p["name"], names)]
|
||||||
|
objects.append({
|
||||||
|
"id": d.name,
|
||||||
|
"type": "project",
|
||||||
|
"names": names,
|
||||||
|
"netbird_peers": nb,
|
||||||
|
"tags": fm.get("tags", []),
|
||||||
|
"owner": fm.get("owner") or fm.get("client"),
|
||||||
|
"region": fm.get("region"),
|
||||||
|
"status": fm.get("status", "unknown"),
|
||||||
|
"file": rel_file,
|
||||||
|
})
|
||||||
|
if rel_file:
|
||||||
|
seen_files.add(rel_file)
|
||||||
|
|
||||||
|
for f in sorted((VAULT / "projects").glob("*.md")):
|
||||||
|
if f.name == "_index.md":
|
||||||
|
continue
|
||||||
|
rel = f"projects/{f.name}"
|
||||||
|
if rel in seen_files:
|
||||||
|
continue
|
||||||
|
fm = parse_frontmatter(f.read_text())
|
||||||
|
objects.append({
|
||||||
|
"id": f.stem,
|
||||||
|
"type": "project-note",
|
||||||
|
"names": [f.stem],
|
||||||
|
"netbird_peers": [],
|
||||||
|
"tags": fm.get("tags", []),
|
||||||
|
"owner": fm.get("owner"),
|
||||||
|
"region": fm.get("region"),
|
||||||
|
"status": fm.get("status", "unknown"),
|
||||||
|
"file": rel,
|
||||||
|
})
|
||||||
|
|
||||||
|
matched_peer_names = {p["name"] for o in objects for p in o["netbird_peers"]}
|
||||||
|
for p in peers:
|
||||||
|
if p["name"] in matched_peer_names:
|
||||||
|
continue
|
||||||
|
objects.append({
|
||||||
|
"id": p["name"].replace(" ", "_"),
|
||||||
|
"type": "netbird-only",
|
||||||
|
"names": [p["name"]],
|
||||||
|
"netbird_peers": [p],
|
||||||
|
"tags": [],
|
||||||
|
"owner": None,
|
||||||
|
"region": p["city"] or None,
|
||||||
|
"status": "no-project-page",
|
||||||
|
"file": None,
|
||||||
|
})
|
||||||
|
|
||||||
|
JSON_OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
JSON_OUT.write_text(json.dumps(objects, indent=2, ensure_ascii=False) + "\n")
|
||||||
|
|
||||||
|
today = date.today().isoformat()
|
||||||
|
now = datetime.now().isoformat(timespec="minutes")
|
||||||
|
n_proj = sum(1 for o in objects if o["type"] in ("project", "project-note"))
|
||||||
|
n_with_nb = sum(1 for o in objects if o["type"] == "project" and o["netbird_peers"])
|
||||||
|
n_orphan = sum(1 for o in objects if o["type"] == "netbird-only")
|
||||||
|
|
||||||
|
md = [
|
||||||
|
"---",
|
||||||
|
f"date: {today}",
|
||||||
|
"type: index",
|
||||||
|
"source: scripts/kb-objects-map.py",
|
||||||
|
"tags: [index, registry, objects, netbird]",
|
||||||
|
"---",
|
||||||
|
"",
|
||||||
|
"# Реестр объектов и netbird-пиров",
|
||||||
|
"",
|
||||||
|
f"Авто-сгенерировано `{now}` из [[dttb/netbird-inventory]] + frontmatter в `projects/`.",
|
||||||
|
"**Не править вручную** — перепишется. Источник правды — frontmatter в каждом README.",
|
||||||
|
"",
|
||||||
|
f"- Проектов: **{n_proj}**, из них с netbird-привязкой: **{n_with_nb}**",
|
||||||
|
f"- NetBird-пиров без projects-страницы: **{n_orphan}** (TODO — создать стабы)",
|
||||||
|
"",
|
||||||
|
"## Проекты с netbird-привязкой",
|
||||||
|
"",
|
||||||
|
"| ID | Имена | NetBird IP | OS | Город | Файл | Статус |",
|
||||||
|
"|---|---|---|---|---|---|---|",
|
||||||
|
]
|
||||||
|
for o in sorted(objects, key=lambda x: x["id"]):
|
||||||
|
if o["type"] == "project" and o["netbird_peers"]:
|
||||||
|
nb_ip = ", ".join(p["ip"] for p in o["netbird_peers"])
|
||||||
|
nb_os = ", ".join(sorted({p["os"] for p in o["netbird_peers"] if p["os"]}))
|
||||||
|
nb_city = ", ".join(sorted({p["city"] for p in o["netbird_peers"] if p["city"]}))
|
||||||
|
file_link = f"[[{o['file'][:-3]}]]" if o["file"] else "—"
|
||||||
|
names = ", ".join(o["names"][:5])
|
||||||
|
md.append(f"| {o['id']} | {names} | {nb_ip} | {nb_os} | {nb_city} | {file_link} | {o['status']} |")
|
||||||
|
|
||||||
|
md.extend([
|
||||||
|
"",
|
||||||
|
"## Проекты без netbird-привязки",
|
||||||
|
"",
|
||||||
|
"| ID | Тип | Файл | Статус |",
|
||||||
|
"|---|---|---|---|",
|
||||||
|
])
|
||||||
|
for o in sorted(objects, key=lambda x: x["id"]):
|
||||||
|
if o["type"] in ("project", "project-note") and not o["netbird_peers"]:
|
||||||
|
file_link = f"[[{o['file'][:-3]}]]" if o["file"] else "—"
|
||||||
|
md.append(f"| {o['id']} | {o['type']} | {file_link} | {o['status']} |")
|
||||||
|
|
||||||
|
md.extend([
|
||||||
|
"",
|
||||||
|
"## NetBird-пиры без projects-страницы — TODO",
|
||||||
|
"",
|
||||||
|
"Эти пиры есть в инвентаре, но у них нет своей карточки в `projects/`. Бот не сможет ответить на «найди мне X» — нет файла. Нужно создать стабы (Фаза 4 плана).",
|
||||||
|
"",
|
||||||
|
"| Имя в NetBird | IP | OS | Город | Версия |",
|
||||||
|
"|---|---|---|---|---|",
|
||||||
|
])
|
||||||
|
for o in sorted(objects, key=lambda x: x["id"]):
|
||||||
|
if o["type"] == "netbird-only":
|
||||||
|
p = o["netbird_peers"][0]
|
||||||
|
md.append(f"| `{p['name']}` | {p['ip']} | {p['os']} | {p['city']} | {p['version']} |")
|
||||||
|
|
||||||
|
md.append("")
|
||||||
|
MD_OUT.write_text("\n".join(md))
|
||||||
|
|
||||||
|
print(f"Wrote {JSON_OUT.relative_to(VAULT)} ({len(objects)} entries: "
|
||||||
|
f"{n_proj} projects, {n_orphan} netbird-only)")
|
||||||
|
print(f"Wrote {MD_OUT.relative_to(VAULT)}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user