modified: certbot_dns_netbox/dns_netbox.py

Add auto-create records as fallback. In case a token with more privileges is used (Bad idea in PROD).

.gitea/workflows/publish.yml

@@ -1,9 +1,12 @@
 name: Publish Python package to PyPI
 
+#on:
+#  push:
+#    tags:
+#      - "v*"
 on:
-  push:
+  release:
-    tags:
+    types: [published]
-      - "v*"
 
 jobs:
   publish:

README.md

@@ -1,10 +1,45 @@
-# Certbot DNS Netbox
+# Certbot DNS NetBox
 
 Certbot DNS-01 authenticator plugin that updates the challenge TXT records in
 [NetBox DNS](https://github.com/sys4/netbox-plugin-dns) (the `netbox-dns`
 NetBox plugin) via the bundled `nbapi.py` API helper.
 
-***DISCLAIMER: THIS PROJECT IS VIBE CODED USING AN LLM.***
+***DISCLAIMER: THIS PROJECT IS A 100% VIBE-CODED PRODUCT.***
+
+<a href="https://pypi.org/project/certbot-dns-netbox/">
+<img src="https://img.shields.io/pypi/v/certbot-dns-netbox" alt="PyPI">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/stargazers">
+<img src="https://img.shields.io/github/stars/Suraxius/certbot-dns-netbox?style=flat" alt="Stars Badge">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/network/members">
+<img src="https://img.shields.io/github/forks/Suraxius/certbot-dns-netbox?style=flat" alt="Forks Badge">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/issues">
+<img src="https://img.shields.io/github/issues/Suraxius/certbot-dns-netbox" alt="Issues Badge">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/pulls">
+<img src="https://img.shields.io/github/issues-pr/Suraxius/certbot-dns-netbox" alt="Pull Requests Badge">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/graphs/contributors">
+<img src="https://img.shields.io/github/contributors/Suraxius/certbot-dns-netbox?color=2b9348" alt="GitHub contributors">
+</a>
+<a href="https://github.com/Suraxius/certbot-dns-netbox/blob/master/LICENSE">
+<img src="https://img.shields.io/github/license/suraxius/certbot-dns-netbox?color=2b9348" alt="License Badge">
+</a>
+<a href="https://github.com/psf/black">
+<img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="Code Style Black">
+</a>
+<a href="https://pepy.tech/project/certbot-dns-netbox">
+<img src="https://static.pepy.tech/personalized-badge/certbot-dns-netbox?period=total&left_color=BLACK&right_color=BLUE&left_text=Downloads" alt="Downloads">
+</a>
+<a href="https://pepy.tech/project/certbot-dns-netbox">
+<img src="https://static.pepy.tech/personalized-badge/certbot-dns-netbox?period=monthly&left_color=BLACK&right_color=BLUE&left_text=Downloads%2fMonth" alt="Downloads/Month">
+</a>
+<a href="https://pepy.tech/project/certbot-dns-netbox">
+<img src="https://static.pepy.tech/personalized-badge/certbot-dns-netbox?period=weekly&left_color=BLACK&right_color=BLUE&left_text=Downloads%2fWeek" alt="Downloads/Week">
+</a>
+
 
 ## How it works
 
@@ -20,23 +55,6 @@ and the plugin **never creates or deletes records**:
 3. leaves the records in place after validation so they can be updated again
    on the next renewal.
 
-### Provisioning
-
-The challenge TXT records must be created beforehand in netbox-dns, in the
-view configured via `search_view` (`certbot_dns_netbox/dns_netbox.py`,
-default: `global`). Each challenge token needs its own record:
-
-| Certificate request                                    | TXT records needed |
-| ------------------------------------------------------ | ------------------ |
-| `-d example.com` or `-d '*.example.com'`               | 1                  |
-| `-d example.com -d '*.example.com'` (apex + wildcard)  | 2                  |
-
-All records for a challenge share the same name (e.g. name
-`_acme-challenge.unifi` in zone `ewcs.ch`); netbox-dns allows multiple TXT
-records with different values for the same name. If a certificate request
-needs more records than exist, certbot aborts with an error. Renewals simply
-rewrite the values, so the records never need to be touched again.
-
 Connection settings are taken from the environment, exactly as `nbapi.py`
 expects them:
 
@@ -45,19 +63,13 @@ expects them:
 | `NETBOX_URL`    | Base URL of the NetBox instance            |
 | `NETBOX_BEARER` | NetBox API token (`Authorization: Bearer`) |
 | `NETBOX_TOKEN`  | NetBox API token (`Authorization: Token`)  |
+| `SEARCH_VIEW`   | NetBox DNS view to search for records in.  |
 | `DEBUG`         | Optional: verbose nbapi.py logging         |
 
 One of `NETBOX_BEARER` / `NETBOX_TOKEN` is required. The token's user needs
 NetBox object permissions to view and change netbox-dns records only — no
 zone or view permissions are required.
 
-## Installation
-
-Install into the same Python environment as certbot:
-
-```
-pip install ./certbot-dns-netbox
-```
 
 ## Usage
 
@@ -98,6 +110,7 @@ When certbot runs unattended (cron or systemd timer), make sure the `NETBOX_*`
 environment variables are set, e.g. via a systemd `EnvironmentFile=` or a
 wrapper script.
 
+
 ## Verification
 
 ```
@@ -107,6 +120,135 @@ certbot certonly --dry-run \
     -d sub.example.com                # end-to-end against the staging ACME API
 ```
 
-Watch the `_acme-challenge` TXT records in the `global` view: their values
+Watch the `_acme-challenge` TXT records in the `_default_` view: their values
 change to the challenge tokens during the run and remain in place afterwards.
 The next renewal updates the same records again.
+
+
+## Tutorial: end-to-end setup
+
+This walkthrough sets up certificates for `example.com`, including the
+wildcard, from scratch. Adapt domains, names and paths to your environment.
+
+### 1. Create the DNS zone
+
+Create the `example.com` zone in netbox-dns, in the view the plugin will search
+(Set by `SEARCH_VIEW` environment variable, default: `_default_`). Note the
+numeric **zone ID** of the new zone — it is shown in the web UI URL and can be
+looked up with:
+
+```
+GET /api/plugins/netbox-dns/zones/?name=example.com
+```
+
+### 2. Create a NetBox permission
+
+In the NetBox admin interface, create a permission named
+`ACME Client - Zone example.com`:
+
+* **Object type**: `DNS › record` only
+* **Actions**: view and change — nothing else. The plugin never creates or
+  deletes records, and it does not need zone or view permissions.
+* **Constraint**:
+
+```json
+{"name": "_acme-challenge", "type": "TXT", "zone_id": 0000}
+```
+
+`0000` is the zone ID from step 1. The constraint limits the token to TXT
+records named `_acme-challenge` in exactly that zone. Note that certificates
+for sub-domains (`-d sub.example.com`) use a record named
+`_acme-challenge.sub`; in that case, relax the constraint to
+`"name__startswith": "_acme-challenge"` instead of the exact `name`.
+
+### 3. Create a service user
+
+* Create a user, e.g. `certbot`, without a usable login, and assign the
+  permission from step 2 directly to the **user**.
+* Optionally create a group, e.g. `Service Accounts`, purely for organizing
+  service accounts, and make the user a member of it.
+
+### 4. Pre-provision the TXT records
+
+The plugin only updates records, so they must exist before the first run.
+Create `_acme-challenge` TXT records in the `example.com` zone — one per
+challenge token:
+
+| Certificate request                                   | TXT records needed |
+| ----------------------------------------------------- | ------------------ |
+| `-d example.com` or `-d '*.example.com'`              | 1                  |
+| `-d example.com -d '*.example.com'` (apex + wildcard) | 2                  |
+
+All records for a challenge share the same name (`_acme-challenge`, relative
+to the zone); netbox-dns allows multiple TXT records with different values
+for the same name. Initial values are placeholders and get rewritten by the
+plugin. If a certificate request needs more records than exist, certbot
+aborts with `Not enough TXT records ...` — add another record and rerun.
+
+### 5. Create an API token and environment file
+
+On the `certbot` user, create an API token (leave write enabled) and expose
+it to whatever runs certbot — a cronjob in most cases — via environment
+variables. Keep them in a root-readable-only file, e.g.
+`/etc/default/certbot-netbox`:
+
+```sh
+NETBOX_URL=https://netbox.example.com
+NETBOX_TOKEN=<token>     # or: NETBOX_BEARER=<token>
+```
+
+```
+chmod 600 /etc/default/certbot-netbox
+```
+
+### 6. Install certbot and the plugin in a dedicated venv
+
+Do not install certbot from the system package manager — certbot and the
+plugin live together in a dedicated venv:
+
+```sh
+python3 -m venv /opt/certbot-netbox
+/opt/certbot-netbox/bin/pip install certbot certbot-dns-netbox
+```
+
+### 7. Test
+
+```sh
+/opt/certbot-netbox/bin/certbot certonly \
+    --authenticator dns-netbox \
+    -d example.com -d '*.example.com'
+```
+
+The first ever certbot run asks for an ACME account e-mail and the terms of
+service. During the run the plugin writes the challenge tokens into the
+`_acme-challenge` TXT records; the records remain in place afterwards. Tip:
+append `--dry-run` for a first try to validate the setup against the staging
+ACME API without touching rate limits.
+
+### 8. Automate renewal
+
+Create a wrapper that loads the environment and runs certbot from the venv,
+e.g. `/usr/local/bin/certbot-netbox-renew`:
+
+```sh
+#!/bin/sh
+set -a
+. /etc/default/certbot-netbox
+set +a
+exec /opt/certbot-netbox/bin/certbot renew --quiet
+```
+
+```
+chmod +x /usr/local/bin/certbot-netbox-renew
+```
+
+And a cron entry, e.g. `/etc/cron.d/certbot-netbox`:
+
+```
+0 3 * * * root /usr/local/bin/certbot-netbox-renew >> /var/log/certbot-netbox-renew.log 2>&1
+```
+
+`certbot renew` reads the renewal configuration written by step 7, so no
+authenticator flags are needed here. Add a `--deploy-hook` (e.g. restarting
+the web server) if services need to pick up the renewed certificate.
+

certbot_dns_netbox/dns_netbox.py

@@ -2,10 +2,11 @@
 
 The TXT records for the ACME dns-01 challenge are managed exclusively
 through the netbox-dns record API endpoint, via the bundled nbapi.py
-helper. The zone endpoint is never queried and the plugin never creates
+helper. The zone endpoint is never queried and the plugin never deletes
-or deletes records: the challenge TXT records (created beforehand in
+records: existing challenge TXT records are updated with the challenge
-NetBox, one per challenge token) are updated with the challenge tokens
+tokens, and a missing record is only created when no free record is left
-and left in place afterwards.
+and the API token is permitted to create it. Records are left in place
+afterwards so they can be updated again on the next renewal.
 
 Connection settings are taken from the environment, exactly as nbapi.py
 expects them:
@@ -26,7 +27,9 @@ from certbot.plugins import dns_common
 logger = logging.getLogger(__name__)
 
 # netbox-dns view that contains the zone with the challenge TXT record.
-search_view = "global"
+search_view = os.environ.get("SEARCH_VIEW") or "_default_"
+
+RECORD_DESCRIPTION = "certbot DNS-01 challenge"
 
 
 class Authenticator(dns_common.DNSAuthenticator):
@@ -61,7 +64,9 @@ class Authenticator(dns_common.DNSAuthenticator):
     def more_info(self) -> str:
         return (
             "This plugin updates pre-provisioned DNS TXT records in "
-            "netbox-dns with the tokens of the dns-01 challenges."
+            "netbox-dns with the tokens of the dns-01 challenges, creating "
+            "additional records only when none are left and the token "
+            "permits it."
         )
 
     def _setup_credentials(self) -> None:
@@ -89,15 +94,15 @@ class Authenticator(dns_common.DNSAuthenticator):
             raise
         except Exception as e:
             raise errors.PluginError(
-                f"Failed to update TXT record {validation_name}: {e}"
+                f"Failed to set TXT record {validation_name}: {e}"
             ) from e
 
     def _cleanup(self, domain: str, validation_name: str, validation: str) -> None:
-        # The TXT record is managed outside of this plugin and must remain in
+        # Records persist — including ones the plugin created — so they can
-        # place so that it can be updated again on the next renewal; deleting
+        # be updated again on the next renewal; deleting them would leave
-        # it would leave nothing to update.
+        # nothing to update.
         logger.debug(
-            "Leaving TXT record %s in place (this plugin only updates records).",
+            "Leaving TXT record %s in place (records are reused, not deleted).",
             validation_name,
         )
 
@@ -106,9 +111,9 @@ class _NetboxDnsClient:
     """netbox-dns client built on the bundled nbapi helper.
 
     Only the record API endpoint is used, so the API token needs no
-    permissions on zones. Records are never created or deleted: the
+    permissions on zones. Records are never deleted: existing challenge
-    challenge TXT records must be provisioned beforehand in NetBox, one
+    TXT records are updated in place, and a record is only created when
-    per challenge token, and are updated in place.
+    no free record is left and the token is permitted to create it.
     """
 
     def __init__(self, ttl: Optional[int] = None, insecure: bool = False):
@@ -148,9 +153,13 @@ class _NetboxDnsClient:
     def update_txt_record(self, record_fqdn: str, validation: str) -> None:
         """Write the challenge token into one of the pre-provisioned records.
 
-        Each challenge token claims its own record so that several tokens
+        Records claimed earlier in this certbot run are skipped so that
-        for the same name (e.g. apex and wildcard on one certificate) stay
+        several tokens for the same name (e.g. apex and wildcard on one
-        live simultaneously.
+        certificate) stay live simultaneously. Among the remaining records,
+        the least recently updated one is chosen, according to the
+        timestamps NetBox keeps on the records themselves — that selection
+        needs no shared state, so separate certbot invocations for domains
+        sharing a record name still claim different records.
         """
         fqdn = self._fqdn(record_fqdn)
         records = self._get_txt_records(fqdn)
@@ -164,20 +173,23 @@ class _NetboxDnsClient:
                 )
                 return
 
-        existing = sorted(
+        free = [
-            (record for record in records if not record.get("managed")),
+            record
-            key=lambda record: record["id"],
+            for record in records
-        )
+            if not record.get("managed") and record["id"] not in self._claimed
-        free = [record for record in existing if record["id"] not in self._claimed]
+        ]
         if not free:
-            raise errors.PluginError(
+            self._create_txt_record(record_fqdn, validation, records)
-                f"Not enough TXT records for {record_fqdn} in view "
+            return
-                f"{search_view!r}: {len(existing)} record(s) exist, but all of "
-                "them are already in use by this certificate request. Add "
-                "another TXT record with the same name and rerun certbot."
-            )
 
-        target = free[0]
+        # least recently updated record first; ties break on the record id.
+        target = min(
+            free,
+            key=lambda record: (
+                record.get("last_updated") or "",
+                record["id"],
+            ),
+        )
         patch = {"id": target["id"], "value": validation}
         if self._ttl:
             patch["ttl"] = self._ttl
@@ -186,3 +198,93 @@ class _NetboxDnsClient:
         logger.info(
             "Updated TXT record %s (id %s).", target["fqdn"], target["id"]
         )
+
+    def _resolve_zone(self, fqdn: str, existing_records: list) -> dict:
+        """Resolve the zone covering fqdn using the record endpoint only.
+
+        Any existing record exposes the zone it belongs to, so the zone id
+        can simply be harvested from it. Without records, candidate zone
+        names are probed against the record list filter; a zone without a
+        single visible record cannot provide an id and is referenced by
+        name instead, which requires view permission on the zone.
+        """
+        if existing_records:
+            zone = existing_records[0]["zone"]
+            return {"id": zone["id"], "name": zone["name"]}
+
+        bare = fqdn.rstrip(".").lower()
+        for candidate in dns_common.base_domain_name_guesses(bare):
+            try:
+                results = (
+                    self._nbapi.get_dns_record(
+                        {"zone": candidate, "view": search_view, "limit": 1},
+                        multi_exp=True,
+                    )
+                    or []
+                )
+            except requests.exceptions.HTTPError:
+                # the record filter rejects zone names that do not exist
+                continue
+            if results:
+                zone = results[0]["zone"]
+                return {"id": zone["id"], "name": zone["name"]}
+            # the zone exists, but none of its records are visible to this token
+            return {"name": candidate}
+        raise errors.PluginError(
+            f"No zone found for {fqdn!r} in view {search_view!r}."
+        )
+
+    def _create_txt_record(
+        self, record_fqdn: str, validation: str, existing_records: list
+    ) -> None:
+        """Create the challenge TXT record via the record endpoint.
+
+        Only called when no free record is left. The zone is resolved
+        through the record endpoint (see _resolve_zone); the created
+        record joins the pool of pre-provisioned records.
+        """
+        zone = self._resolve_zone(record_fqdn, existing_records)
+        zone_name = zone["name"].rstrip(".").lower()
+        bare = record_fqdn.rstrip(".").lower()
+        data = {
+            "zone": zone.get("id", {"name": zone["name"]}),
+            "name": "@" if bare == zone_name else bare[: -(len(zone_name) + 1)],
+            "type": "TXT",
+            "value": validation,
+            "description": RECORD_DESCRIPTION,
+        }
+        if self._ttl:
+            data["ttl"] = self._ttl
+
+        try:
+            created = self._nbapi.create_dns_record(data)
+        except Exception as create_error:
+            message = str(create_error)
+            hint = ""
+            if "Related object not found" in message:
+                hint = (
+                    " (referencing the zone by name requires view permission "
+                    "on the zone; alternatively pre-provision any record in "
+                    "the zone so that its id can be harvested via the record "
+                    "endpoint)"
+                )
+            elif "permission" in message.lower():
+                hint = (
+                    " (the API token likely lacks the add action for "
+                    "netbox-dns records; grant it, or pre-provision the "
+                    "record)"
+                )
+            raise errors.PluginError(
+                f"Failed to create TXT record for {record_fqdn}: "
+                f"{create_error}{hint}"
+            ) from create_error
+
+        record_id = created.get("id") if isinstance(created, dict) else None
+        if record_id:
+            self._claimed.add(record_id)
+        logger.info(
+            "Created TXT record %s in zone %s (view %s).",
+            data["name"],
+            zone_name,
+            search_view,
+        )

certbot_dns_netbox/nbapi.py

@@ -274,7 +274,7 @@ RESOURCES = {
     "vlan": "/ipam/vlans",
     "dns_view": "/plugins/netbox-dns/views/",
     "dns_zone": "/plugins/netbox-dns/zones/",
-    "dns_record": "/plugins/netbox-dns/records/"
+    "dns_record": "/plugins/netbox-dns/records/",
 }
 
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "certbot-dns-netbox"
-version = "1.0.0"
+version = "1.0.3"
 description = "NetBox DNS (netbox-plugin-dns) DNS-01 authenticator plugin for Certbot"
 readme = "README.md"
 requires-python = ">=3.9"