fix(web): plugin settings endpoints - a refused save no longer leaks into config.json; GET masks secrets (#742)

* fix(config): load_config hands each caller a private copy

ConfigManager.load_config() returned its cached self.config itself (the
mtime fast path from #410 kept the full path's aliasing). Web handlers
edit what they load and then validate: the plugin form save applies the
posted fields to the loaded section (a shallow .copy(), so nested dicts
were the cache's own), and save_main_config sets its checkboxes before
it checks auto_update_channel. When the save was refused, the edit
stayed in the cache the fast path serves, and the next save of any
other setting wrote it to config.json: the refused value, and a nested
secret typed into the same form (mqtt.password, league.espn_s2,
flightaware.api_key) in plain text, since it never reached
config_secrets.json to be stripped. The form also reloaded showing the
refused values.

load_config() now returns a private copy on both paths, and
save_config/save_config_atomic keep a copy of what they were given, so
nothing a caller edits reaches the cache unless it is saved. Fixing it
here rather than in each handler covers every route that edits before it
validates. No caller relies on editing the cache without saving: every
src/ and web_interface/ caller either reads, or saves the dict it
edited. get_config() still returns the live dict for the display
process's readers.

The copy is a pickle round trip: on a Pi 4 with its real 64 KiB config,
2.0 ms against 6.9 ms for copy.deepcopy (json round trip 3.4 ms). Two
tests asserted the aliasing itself and now assert a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): GET /plugins/config masks secrets and refuses core sections

The route returned the plugin's section as load_config() has it, with
config_secrets.json merged in: API keys and tokens went out in plain
text. #276 masked them here; #330's rewrite of the route dropped it,
while the settings page and GET /config/secrets kept masking. It also
took any plugin_id, so ?plugin_id=web_auth returned the login's
cookie-signing key and password hash, and ?plugin_id=github the Plugin
Store token, which GET /config/main strips and redacts.

The route now refuses what _non_plugin_id_error refuses for reset and
uninstall (core sections, malformed ids) with a 400, and blanks x-secret
fields with mask_secret_fields after the defaults merge, as the page
does. A plugin with no schema has its credential-named fields blanked by
_redact_credentials, as GET /config/main does. Blank rather than the
bullets of GET /config/secrets: the save drops a blank secret as
"unchanged" (remove_empty_secrets) but would store the bullets, so the
response must post back as it came. Tested: GET, then POST the response
unchanged, keeps every stored secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): parse a table row's cells against the list's item schema

An array of objects drawn as a table posts each cell as
"cities.0.timezone". _get_schema_property stopped at "cities" (an array,
not an object with properties), so _parse_form_value_with_schema got no
schema for the cell and guessed: a blank optional text cell became None
and a text cell holding digits became an int. Validation refused both,
so every save of the page failed for as long as such a row existed --
geochron's city without a timezone, a countdown named "2027". A secret
cell is always drawn blank, so a plugin with secrets in its rows could
not be saved from the form at all.

The lookup now steps from an index segment into the array's items: to
the item schema itself for "color.2", into its properties for a row
cell. Number, boolean and required cells convert as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a blank secret field saves as "unchanged", required or not

The settings page draws a stored secret blank (mask_secret_fields) and
posts the blank back. _parse_form_value_with_schema turned a blank
optional string into "" -- which the save drops as unchanged
(remove_empty_secrets) -- but a blank required one into None. For a
secret that is required with no default (youtube-stats' api_key) that
None failed validation, so every save of the page was refused until the
key was typed in again.

A blank text secret (x-secret, type string) now parses to "", whatever
its required list says; a list or object secret keeps getting [] or {},
which the save drops the same way. Not _SKIP_FIELD: skipping keeps the
value load_config() merged in, and the save would then write it back to
config_secrets.json -- after a secret change the cached section can
still hold the old one, so that write reverted it. A test covers that
sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): POST /plugins/config refuses core sections and malformed ids

Reset and uninstall check the plugin id with _non_plugin_id_error; the
save did not. {"plugin_id": "display", "config": {...}} found no schema,
so nothing was validated or filtered, and the body was merged into the
core display section along with "enabled": true -- rows: "banana"
included. A plugin_id that was not a string (a list, an object, a number)
reached config.get() or the schema lookup, raised TypeError, and came
back as a 500.

Both the JSON and the form path now call _non_plugin_id_error first and
answer its 400.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a text field keeps "true", "[1, 2]" and "{}" as typed

_parse_form_value_with_schema guessed before it consulted the schema:
"true"/"false" became booleans, and a value starting with "[" or "{"
that parsed as JSON became a list or object, whatever the field's type.
A text setting holding "true", "False", "[1, 2]" or "{}" was then
refused by validation ("Expected type string, got bool"), and the save
with it.

A field whose schema type is string, or string-or-null, now returns the
posted text as it came. Every other type goes through the conversions as
before; numbers in text fields were already left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(config): copy the cached config without pickle

_private_copy was a pickle round trip. It only ever unpickled bytes it had
just made from our own dict, so nothing untrusted reached it, but it put
pickle in the config path and Codacy failed the PR for it (B301/B403).
The config is JSON data, so copying its dicts and lists is a full copy;
every other value is immutable. Measured on ledpi (Pi 4) with its real
60 KiB config: 2.11 ms, against 1.92 ms for pickle and 6.75 ms for
copy.deepcopy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): describe the config copy without pickle

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

web_interface/blueprints/api_v3/__init__.py

@@ -957,6 +957,19 @@ def _get_schema_property(schema, key_path):
                     i = j
                     matched = True
                     break
+                # Through an array to its items: a table row posts its cells
+                # as "cities.0.timezone", where the index names no property.
+                # Stopping here left each cell parsed with no schema at all,
+                # so a blank text cell became null and "2027" a number.
+                items = prop.get('items') if _schema_type_is(prop, 'array') else None
+                if isinstance(items, dict) and parts[j].isdigit():
+                    if j + 1 == len(parts):
+                        return items
+                    if 'properties' in items:
+                        current = items['properties']
+                        i = j + 1
+                        matched = True
+                        break
                 # Matched a non-object before consuming the path — can't go deeper.
                 return None
         if not matched:
@@ -1040,6 +1053,16 @@ def _parse_form_value_with_schema(value, key_path, schema):
 
     # Handle None/empty values
     if value is None or (isinstance(value, str) and value.strip() == ''):
+        # The form draws a stored secret blank, so a blank secret means
+        # "unchanged", and "" is what the save drops as unchanged
+        # (remove_empty_secrets). A required one with no default fell
+        # through to None below, failed validation, and blocked every save
+        # of the page until the secret was typed in again. Not _SKIP_FIELD:
+        # that keeps the merged value from load_config(), which the save
+        # would then write back to config_secrets.json. Text secrets only:
+        # a list or object one gets its empty value below, dropped the same.
+        if prop and prop.get('x-secret') and prop.get('type', 'string') == 'string':
+            return ""
         # A nullable field left blank means null, not an empty container.
         # This is the inherit sentinel for per-mode style overrides: an
         # empty list there would read as "the user chose no colour" rather
@@ -1074,6 +1097,14 @@ def _parse_form_value_with_schema(value, key_path, schema):
     if isinstance(value, str):
         stripped = value.strip()
 
+        # A text field keeps what was typed. The guesses below ran first, so
+        # "true", "False", "[1, 2]" or "{}" in a text field became a boolean,
+        # list or object, and the save failed validation for a good string.
+        declared = prop.get('type') if isinstance(prop, dict) else None
+        if declared == 'string' or (isinstance(declared, list) and
+                                    [t for t in declared if t != 'null'] == ['string']):
+            return value
+
         # Check for boolean strings
         if stripped.lower() == 'true':
             return True

web_interface/blueprints/api_v3/plugin_config.py

@@ -9,13 +9,15 @@ from web_interface.blueprints.api_v3 import (
     _enhance_schema_with_core_properties, _non_plugin_id_error,
     _filter_config_by_schema, _get_schema_property,
     _hidden_array_item_property, _plugin_directory,
-    _parse_form_value_with_schema, _schema_allows_null, _schema_type_is,
-    _set_missing_booleans_to_false, _set_nested_value, api_v3, datetime,
-    deep_merge, error_response, exception_error_response, find_secret_fields,
-    json, jsonify, logger, merge_secrets, os, remove_empty_secrets, request,
-    separate_secrets, success_response, validate_request_json,
+    _parse_form_value_with_schema, _redact_credentials, _schema_allows_null,
+    _schema_type_is, _set_missing_booleans_to_false, _set_nested_value, api_v3,
+    datetime, deep_merge, error_response, exception_error_response,
+    find_secret_fields, json, jsonify, logger, merge_secrets, os,
+    remove_empty_secrets, request, separate_secrets, success_response,
+    validate_request_json,
 )
 from src.web_interface.config_arrays import coerce_array_shapes
+from src.web_interface.secret_helpers import mask_secret_fields
 from src.web_interface.validators import dedup_unique_arrays
 import web_interface.blueprints.api_v3 as _pkg
 # Read through the module rather than bound by value: tests patch these
@@ -43,6 +45,12 @@ def get_plugin_config():
                 context={'missing_params': ['plugin_id']},
                 status_code=400
             )
+        # load_config() merges config_secrets.json in, core sections
+        # included: ?plugin_id=web_auth returned the login's cookie key and
+        # password hash, and ?plugin_id=github the Plugin Store token.
+        id_error = _non_plugin_id_error(plugin_id)
+        if id_error:
+            return id_error
 
         # Get plugin configuration from config manager
         main_config = api_v3.config_manager.load_config()
@@ -52,12 +60,13 @@ def get_plugin_config():
         # missing fields, reading legacy booleans as objects first: what the
         # plugin runs with, and what posts back through the JSON save
         schema_mgr = api_v3.schema_manager
+        schema = None
         if schema_mgr:
             try:
                 from src.plugin_system.schema_manager import prepare_plugin_config
+                schema = schema_mgr.load_schema(plugin_id, use_cache=True)
                 defaults = schema_mgr.generate_default_config(plugin_id, use_cache=True)
-                plugin_config = prepare_plugin_config(
-                    plugin_config, schema_mgr.load_schema(plugin_id, use_cache=True), defaults)
+                plugin_config = prepare_plugin_config(plugin_config, schema, defaults)
             except Exception as e:
                 # Log but don't fail - defaults merge is best effort
                 logger.warning("Could not merge defaults for %s: %s", plugin_id, e)
@@ -158,6 +167,17 @@ def get_plugin_config():
                 'display_duration': 30
             }
 
+        # Secrets go out blank, as the settings page renders them (#276 added
+        # this; #330 dropped it). Blank, not the bullets GET /config/secrets
+        # uses: the save reads a blank secret as "unchanged", so this
+        # response posts back without erasing one.
+        properties = schema.get('properties') if isinstance(schema, dict) else None
+        if isinstance(properties, dict):
+            plugin_config = mask_secret_fields(plugin_config, properties)
+        else:
+            # No schema to mark them: blank whatever is named like one
+            plugin_config = _redact_credentials(plugin_config)
+
         return success_response(data=plugin_config)
     except Exception as e:
         return exception_error_response(e, ErrorCode.CONFIG_LOAD_FAILED)
@@ -183,6 +203,12 @@ def save_plugin_config():
             if error:
                 return error
             plugin_id = data['plugin_id']
+            # As reset and uninstall do: {"plugin_id": "display"} merged
+            # unvalidated values into the core display section, and an id
+            # that was not a string raised a TypeError, answered as a 500.
+            id_error = _non_plugin_id_error(plugin_id)
+            if id_error:
+                return id_error
             submitted_config = data.get('config', {})
             if not isinstance(submitted_config, dict):
                 return error_response(
@@ -201,6 +227,9 @@ def save_plugin_config():
                     'plugin_id required in query string',
                     status_code=400
                 )
+            id_error = _non_plugin_id_error(plugin_id)
+            if id_error:
+                return id_error
 
             # Load existing config as base (partial form updates should merge, not replace)
             existing_config = {}