Chuck 7f230f625d Feature/one shot installer (#175) ...

* fix(plugins): Remove compatible_versions requirement from single plugin install

Remove compatible_versions from required fields in install_from_url method
to match install_plugin behavior. This allows installing plugins from URLs
without manifest version requirements, consistent with store plugin installation.

* fix(7-segment-clock): Update submodule with separator and spacing fixes

* fix(plugins): Add onchange handlers to existing custom feed inputs

- Add onchange handlers to key and value inputs for existing patternProperties fields
- Fixes bug where editing existing custom RSS feeds didn't save changes
- Ensures hidden JSON input field is updated when users edit feed entries
- Affects all plugins using patternProperties (custom_feeds, feed_logo_map, etc.)

* Add array-of-objects widget support to web UI

- Add support for rendering arrays of objects in web UI (for custom_feeds)
- Implement add/remove/update functions for array-of-objects widgets
- Support file-upload widgets within array items
- Update form data handling to support array JSON data fields

* Update plugins_manager.js cache-busting version

Update version parameter to force browser to load new JavaScript with array-of-objects widget support.

* Fix: Move array-of-objects detection before file-upload/checkbox checks

Move the array-of-objects widget detection to the top of the array handler so it's checked before file-upload and checkbox-group widgets. This ensures custom_feeds is properly detected as an array of objects.

* Update cache-busting version for array-of-objects fix

* Remove duplicate array-of-objects check

* Update cache version again

* Add array-of-objects widget support to server-side template

Add detection and rendering for array-of-objects in the Jinja2 template (plugin_config.html).
This enables the custom_feeds widget to display properly with name, URL, enabled checkbox, and logo upload fields.

The widget is detected by checking if prop.items.type == 'object' && prop.items.properties,
and is rendered before the file-upload widget check.

* Use window. prefix for array-of-objects JavaScript functions

Explicitly use window.addArrayObjectItem, window.removeArrayObjectItem, etc.
in the template to ensure the functions are accessible from inline event handlers.
Also add safety checks to prevent errors if functions aren't loaded yet.

* Fix syntax error: Missing indentation for html += in array else block

The html += statement was outside the else block, causing a syntax error.
Fixed by properly indenting it inside the else block.

* Update cache version for syntax fix

* Add debug logging to diagnose addArrayObjectItem availability

* Fix: Wrap array-of-objects functions in window check and move outside IIFE

Ensure functions are available globally by wrapping them in a window check
and ensuring they're defined outside any IIFE scope. Also fix internal
function calls to use window.updateArrayObjectData for consistency.

* Update cache version for array-of-objects fix

* Move array-of-objects functions outside IIFE to make them globally available

The functions were inside the IIFE scope, making them inaccessible from
inline event handlers. Moving them outside the IIFE ensures they're
available on window when the script loads.

* Update cache version for IIFE fix

* Fix: Add array-of-objects functions after IIFE ends

The functions were removed from inside the IIFE but never added after it.
Also removed orphaned code that was causing syntax errors.

* Update cache version for array-of-objects fix

* Fix: Remove all orphaned code and properly add array-of-objects functions after IIFE

* Add array-of-objects functions after IIFE ends

These functions must be outside the IIFE to be accessible from inline
event handlers in the server-rendered template.

* Update cache version for syntax fix

* Fix syntax error: Add missing closing brace for else block

* Update cache version for syntax fix

* Replace complex array-of-objects widget with simple table interface

- Replace nested array-of-objects widget with clean table interface
- Table shows: Name, URL, Logo (with upload), Enabled checkbox, Delete button
- Fix file-upload widget detection order to prevent breaking static-image plugin
- Add simple JavaScript functions for add/remove rows and logo upload
- Much more intuitive and easier to use

* Add simple table interface for custom feeds

- Replace complex array-of-objects widget with clean table
- Table columns: Name, URL, Logo (upload), Enabled checkbox, Delete
- Use dot notation for form field names (feeds.custom_feeds.0.name)
- Add JavaScript functions for add/remove rows and logo upload
- Fix file-upload detection order to prevent breaking static-image plugin

* Fix custom feeds table issues

- Fix JavaScript error in removeCustomFeedRow (get tbody before removing row)
- Improve array conversion logic to handle nested paths like feeds.custom_feeds
- Add better error handling and debug logging for array conversion
- Ensure dicts with numeric keys are properly converted to arrays before validation

* Add fallback fix for feeds.custom_feeds dict-to-array conversion

- Add explicit fallback conversion for feeds.custom_feeds if fix_array_structures misses it
- This ensures the dict with numeric keys is converted to an array before validation
- Logo field is already optional in schema (not in required array)

* feat(web): Add checkbox-group widget support for plugin config arrays

Add server-side rendering support for checkbox-group widget in plugin
configuration forms. This allows plugins to use checkboxes for multi-select
array fields instead of comma-separated text inputs.

The implementation:
- Checks for x-widget: 'checkbox-group' in schema
- Renders checkboxes for each enum item in items.enum
- Supports custom labels via x-options.labels
- Works with any plugin that follows the pattern

Already used by:
- ledmatrix-news plugin (enabled_feeds)
- odds-ticker plugin (enabled_leagues)

* feat(install): Add one-shot installation script

- Create comprehensive one-shot installer with robust error handling
- Includes network checks, disk space validation, and retry logic
- Handles existing installations gracefully (idempotent)
- Updates README with quick install command prominently featured
- Manual installation instructions moved to collapsible section

The script provides explicit error messages and never fails silently.
All prerequisites are validated before starting installation.

* fix: Remove accidental plugins/7-segment-clock submodule entry

Remove uninitialized submodule 'plugins/7-segment-clock' that was
accidentally included. This submodule is not related to the one-shot
installer feature and should not be part of this PR.

- Remove submodule entry from .gitmodules
- Remove submodule from git index
- Clean up submodule configuration

* fix(array-objects): Fix schema lookup, reindexing, and disable file upload

Address PR review feedback for array-of-objects helpers:

1. Schema resolution: Use getSchemaProperty() instead of manual traversal
   - Fixes nested array-of-objects schema lookup (e.g., news.custom_feeds)
   - Now properly descends through .properties for nested objects

2. Reindexing: Replace brittle regex with targeted patterns
   - Only replace index in bracket notation [0], [1], etc. for names
   - Only replace _item_<digits> pattern for IDs (not arbitrary digits)
   - Use specific function parameter patterns for onclick handlers
   - Prevents corruption of fieldId, pluginId, or other numeric values

3. File upload: Disable widget until properly implemented
   - Hide/disable upload button with clear message
   - Show existing logos if present but disable upload functionality
   - Prevents silent failures when users attempt to upload files
   - Added TODO comments for future implementation

Also fixes exit code handling in one-shot-install.sh to properly capture
first_time_install.sh exit status before error trap fires.

* fix(security): Fix XSS vulnerability in handleCustomFeedLogoUpload

Replace innerHTML usage with safe DOM manipulation using createElement
and setAttribute to prevent XSS when injecting uploadedFile.path and
uploadedFile.id values.

- Clear logoCell using textContent instead of innerHTML
- Create all DOM elements using document.createElement
- Set uploadedFile.path and uploadedFile.id via setAttribute (automatically escaped)
- Properly structure DOM tree by appending elements in order
- Prevents malicious HTML/script injection through file path or ID values

* fix: Update upload button onclick when reindexing custom feed rows

Fix removeCustomFeedRow to update button onclick handlers that reference
file input IDs with _logo_<index> when rows are reindexed after deletion.

Previously, after deleting a row, the upload button's onclick still referenced
the old file input ID, causing the upload functionality to fail.

Now properly updates:
- getElementById('..._logo_<num>') patterns in onclick handlers
- Other _logo_<num> patterns in button onclick strings
- Function parameter indices in onclick handlers

This ensures upload buttons continue to work correctly after row deletion.

* fix: Make custom feeds table widget-specific instead of generic fallback

Replace generic array-of-objects check with widget-specific check for
'custom-feeds' widget to prevent hardcoded schema from breaking other
plugins with different array-of-objects structures.

Changes:
- Check for x-widget == 'custom-feeds' before rendering custom feeds table
- Add schema validation to ensure required fields (name, url) exist
- Show warning message if schema doesn't match expected structure
- Fall back to generic array input for other array-of-objects schemas
- Add comments for future generic array-of-objects support

This ensures the hardcoded custom feeds table (name, url, logo, enabled)
only renders when explicitly requested via widget type, preventing
breakage for other plugins with different array-of-objects schemas.

* fix: Add image/gif to custom feed logo upload accept attribute

Update file input accept attributes for custom feed logo uploads to include
image/gif, making it consistent with the file-upload widget which also
allows GIF images.

Updated in three places:
- Template file input (plugin_config.html)
- JavaScript addCustomFeedRow function (base.html)
- Dynamic file input creation in handleCustomFeedLogoUpload (base.html)

All custom feed logo upload inputs now accept: image/png, image/jpeg,
image/bmp, image/gif

* fix: Add hidden input for enabled checkbox to ensure false is submitted

Add hidden input with value='false' before enabled checkbox in custom feeds
table to ensure an explicit false value is sent when checkbox is unchecked.

Pattern implemented:
- Hidden input: name='enabled', value='false' (always submitted)
- Checkbox: name='enabled', value='true' (only submitted when checked)
- When unchecked: only hidden input submits (false)
- When checked: both submit, checkbox value (true) overwrites hidden

Updated in two places:
- Template checkbox in plugin_config.html (existing rows)
- JavaScript addCustomFeedRow function in base.html (new rows)

Backend verification:
- Backend (api_v3.py) handles string boolean values and converts properly
- JavaScript form processing explicitly checks element.checked, independent of this pattern
- Standard form submission uses last value when multiple values share same name

* fix: Expose renderArrayObjectItem to window for addArrayObjectItem

Fix scope issue where renderArrayObjectItem is defined inside IIFE but
window.addArrayObjectItem is defined outside, causing the function check
to always fail and fallback to degraded HTML rendering.

Problem:
- renderArrayObjectItem (line 2469) is inside IIFE (lines 796-6417)
- window.addArrayObjectItem (line 6422) is outside IIFE
- Check 'typeof renderArrayObjectItem === function' at line 6454 always fails
- Fallback code lacks file upload widgets, URL input types, descriptions, styling

Solution:
- Expose renderArrayObjectItem to window object before IIFE closes
- Function maintains closure access to escapeHtml and other IIFE-scoped functions
- Newly added items now have full functionality matching initially rendered items

* fix: Reorder array type checks to match template order

Fix inconsistent rendering where JavaScript and Jinja template had opposite
ordering for array type checks, causing schemas with both x-widget: file-upload
AND items.type: object (like static-image) to render differently.

Problem:
- Template checks file-upload FIRST (to avoid breaking static-image plugin)
- JavaScript checked array-of-objects FIRST
- Server-rendered forms showed file-upload widget correctly
- JS-rendered forms incorrectly displayed array-of-objects table widget

Solution:
- Reorder JavaScript checks to match template order:
  1. Check file-upload widget FIRST
  2. Check checkbox-group widget
  3. Check custom-feeds widget
  4. Check array-of-objects as fallback
  5. Regular array input (comma-separated)

This ensures consistent rendering between server-rendered and JS-rendered forms
for schemas that have both x-widget: file-upload AND items.type: object.

* fix: Handle None value for feeds config to prevent TypeError

Fix crash when plugin_config['feeds'] exists but is None, causing
TypeError when checking 'custom_feeds' in feeds_config.

Problem:
- When plugin_config['feeds'] exists but is None, dict.get('feeds', {})
  returns None (not the default {}) because dict.get() only uses default
  when key doesn't exist, not when value is None
- Line 3642's 'custom_feeds' in feeds_config raises TypeError because
  None is not iterable
- This can crash the API endpoint if a plugin config has feeds: null

Solution:
- Change plugin_config.get('feeds', {}) to plugin_config.get('feeds') or {}
  to ensure feeds_config is always a dict (never None)
- Add feeds_config check before 'in' operator for extra safety

This ensures the code gracefully handles feeds: null in plugin configuration.

* fix: Add default value for AVAILABLE_SPACE to prevent TypeError

Fix crash when df produces unexpected output that results in empty
AVAILABLE_SPACE variable, causing 'integer expression expected' error.

Problem:
- df may produce unexpected output format (different locale, unusual
  filesystem name spanning lines, or non-standard df implementation)
- While '|| echo "0"' handles pipeline failures, it doesn't trigger if
  awk succeeds but produces no output (empty string)
- When AVAILABLE_SPACE is empty, comparison [ "$AVAILABLE_SPACE" -lt 500 ]
  fails with 'integer expression expected' error
- With set -e, this causes script to exit unexpectedly

Solution:
- Add AVAILABLE_SPACE=${AVAILABLE_SPACE:-0} before comparison to ensure
  variable always has a numeric value (defaults to 0 if empty)
- This gracefully handles edge cases where df/awk produces unexpected output

* fix: Wrap debug console.log in debug flag check

Fix unconditional debug logging that outputs internal implementation
details to browser console for all users.

Problem:
- console.log('[ARRAY-OBJECTS] Functions defined on window:', ...)
  executes unconditionally when page loads
- Outputs debug information about function availability to all users
- Appears to be development/debugging code inadvertently included
- Noisy console output in production

Solution:
- Wrap console.log statement in _PLUGIN_DEBUG_EARLY check to only
  output when pluginDebug localStorage flag is enabled
- Matches pattern used elsewhere in the file for debug logging
- Debug info now only visible when explicitly enabled via
  localStorage.setItem('pluginDebug', 'true')

* fix: Expose getSchemaProperty, disable upload widget, handle bracket notation arrays

Multiple fixes for array-of-objects and form processing:

1. Expose getSchemaProperty to window (plugins_manager.js):
   - getSchemaProperty was defined inside IIFE but needed by global functions
   - Added window.getSchemaProperty = getSchemaProperty before IIFE closes
   - Updated window.addArrayObjectItem to use window.getSchemaProperty
   - Fixes ReferenceError when dynamically adding array items

2. Disable upload widget for custom feeds (plugin_config.html):
   - File input and Upload button were still active but should be disabled
   - Removed onchange/onclick handlers, added disabled and aria-disabled
   - Added visible disabled styling and tooltip
   - Existing logos continue to display but uploads are prevented
   - Matches PR objectives to disable upload until fully implemented

3. Handle bracket notation array fields (api_v3.py):
   - checkbox-group uses name="field_name[]" which sends multiple values
   - request.form.to_dict() collapses duplicate keys (only keeps last value)
   - Added handling to detect fields ending with "[]" before to_dict()
   - Use request.form.getlist() to get all values, combine as comma-separated
   - Processed before existing array index field handling
   - Fixes checkbox-group losing all but last selected value

* fix: Remove duplicate submit handler to prevent double POSTs

Remove document-level submit listener that conflicts with handlePluginConfigSubmit,
causing duplicate form submissions with divergent payloads.

Problem:
- handlePluginConfigSubmit correctly parses JSON from _data fields and maps to
  flatConfig[baseKey] for patternProperties and array-of-objects
- Document-level listener (line 5368) builds its own config without understanding
  _data convention and posts independently via savePluginConfiguration
- Every submit now sends two POSTs with divergent payloads:
  - First POST: Correct structure with parsed _data fields
  - Second POST: Incorrect structure with raw _data fields, missing structure
- Arrays-of-objects and patternProperties saved incorrectly in second request

Solution:
- Remove document-level submit listener for #plugin-config-form
- Rely solely on handlePluginConfigSubmit which is already attached to the form
- handlePluginConfigSubmit properly handles all form-to-config conversion including:
  - _data field parsing (JSON from hidden fields)
  - Type-aware conversion using schema
  - Dot notation to nested object conversion
  - PatternProperties and array-of-objects support

Note: savePluginConfiguration function remains for use by JSON editor saves

* fix: Use indexed names for checkbox-group to work with existing parser

Change checkbox-group widget to use indexed field names instead of bracket
notation, so the existing indexed field parser correctly handles multiple
selected values.

Problem:
- checkbox-group uses name="{{ full_key }}[]" which requires bracket
  notation handling in backend
- While bracket notation handler exists, using indexed names is more robust
  and leverages existing well-tested indexed field parser
- Indexed field parser already handles fields like "field_name.0",
  "field_name.1" correctly

Solution:
- Template: Change name="{{ full_key }}[]" to name="{{ full_key }}.{{
  loop.index0 }}"
- JavaScript: Update checkbox-group rendering to use name="."
- Backend indexed field parser (lines 3364-3388) already handles this pattern:
  - Detects fields ending with numeric indices (e.g., ".0", ".1")
  - Groups them by base_path and sorts by index
  - Combines into array correctly

This ensures checkbox-group values are properly preserved when multiple
options are selected, working with the existing schema-based parsing system.

* fix: Set values from item data in fallback array-of-objects rendering

Fix fallback code path for rendering array-of-objects items to properly
set input values from existing item data, matching behavior of proper
renderArrayObjectItem function.

Problem:
- Fallback code at lines 3078-3091 and 6471-6486 creates input elements
  without setting values from existing item data
- Text inputs have no value attribute set
- Checkboxes have no checked attribute computed from item properties
- Users would see empty form fields instead of existing configuration data
- Proper renderArrayObjectItem function correctly sets values (line 2556)

Solution:
- Extract propValue from item data: item[propKey] with schema default fallback
- For text inputs: Set value attribute with HTML-escaped propValue
- For checkboxes: Set checked attribute based on propValue truthiness
- Add inline HTML escaping for XSS prevention (since fallback code may
  run outside IIFE scope where escapeHtml function may not be available)

This ensures fallback rendering displays existing data correctly when
window.renderArrayObjectItem is not available.

* fix: Remove extra closing brace breaking if/else chain

Remove stray closing brace at line 3127 that was breaking the if/else chain
before the 'else if (prop.enum)' branch, causing 'Unexpected token else'
syntax error.

Problem:
- Extra '}' at line 3127 closed the prop.type === 'array' block prematurely
- This broke the if/else chain, causing syntax error when parser reached
  'else if (prop.enum)' at line 3128
- Structure was: } else if (array) { ... } } } else if (enum) - extra brace

Solution:
- Removed the extra closing brace at line 3127
- Structure now correctly: } else if (array) { ... } } else if (enum)
- Verified with Node.js syntax checker - no errors

* fix: Remove local logger assignments to prevent UnboundLocalError

Remove all local logger assignments inside save_plugin_config function that
were shadowing the module-level logger, causing UnboundLocalError when nested
helpers like normalize_config_values() or debug checks reference logger before
those assignments run.

Problem:
- Module-level logger exists at line 13: logger = logging.getLogger(__name__)
- Multiple local assignments inside save_plugin_config (lines 3361, 3401, 3421,
  3540, 3660, 3977, 4093, 4118) make logger a local variable for entire function
- Python treats logger as local for entire function scope when any assignment
  exists, causing UnboundLocalError if logger is used before assignments
- Nested helpers like normalize_config_values() or debug checks that reference
  logger before local assignments would fail

Solution:
- Removed all local logger = logging.getLogger(__name__) assignments in
  save_plugin_config function
- Use module-level logger directly throughout the function
- Removed redundant import logging statements that were only used for logger
- This ensures logger is always available and references the module-level logger

All logger references now use the module-level logger without shadowing.

* fix: Fix checkbox-group serialization and array-of-objects key leakage

Multiple fixes for array-of-objects and checkbox-group widgets:

1. Fix checkbox-group serialization (JS and template):
   - Changed from indexed names (categories.0, categories.1) to _data pattern
   - Added updateCheckboxGroupData() function to sync selected values
   - Hidden input stores JSON array of selected enum values
   - Checkboxes use data-checkbox-group and data-option-value attributes
   - Fixes issue where config.categories became {0: true, 1: true} instead of ['nfl', 'nba']
   - Now correctly serializes to array using existing _data handling logic

2. Prevent array-of-objects per-item key leakage:
   - Added skip pattern in handlePluginConfigSubmit for _item_<n>_ names
   - Removed name attributes from per-item inputs in renderArrayObjectItem
   - Per-item inputs now rely solely on hidden _data field
   - Prevents feeds_item_0_name from leaking into flatConfig

3. Add type coercion to updateArrayObjectData:
   - Consults itemsSchema.properties[propKey].type for coercion
   - Handles integer and number types correctly
   - Preserves string values as-is
   - Ensures numeric fields in array items are stored as numbers

4. Ensure currentPluginConfig is always available:
   - Updated addArrayObjectItem to check window.currentPluginConfig first
   - Added error logging if schema not available
   - Prevents ReferenceError when global helpers need schema

This ensures checkbox-group arrays serialize correctly and array-of-objects
per-item fields don't leak extra keys into the configuration.

* fix: Make _data field matching more specific to prevent false positives

Fix overly broad condition that matched any field containing '_data',
causing false positives and inconsistent key transformation.

Problem:
- Condition 'key.endsWith('_data') || key.includes('_data')' matches any
  field containing '_data' anywhere (e.g., 'meta_data_field', 'custom_data_config')
- key.replace(/_data$/, '') only removes '_data' from end, making logic inconsistent
- Fields with '_data' in middle get matched but key isn't transformed
- If their value happens to be valid JSON, it gets incorrectly parsed

Solution:
- Remove 'key.includes('_data')' clause
- Only check 'key.endsWith('_data')' to match actual _data suffix pattern
- Ensures consistent matching: only fields ending with '_data' are treated
  as JSON data fields, and only those get the suffix removed
- Prevents false positives on fields like 'meta_data_field' that happen to
  contain '_data' in their name

* fix: Add HTML escaping to prevent XSS in fallback code and checkbox-group

Add proper HTML escaping for schema-derived values to prevent XSS vulnerabilities
in fallback rendering code and checkbox-group widget.

Problem:
- Fallback code in generateFieldHtml (line 3094) doesn't escape propLabel
  when building HTML strings, while main renderArrayObjectItem uses escapeHtml()
- Checkbox-group widget (lines 3012-3025) doesn't escape option or label values
- While risk is limited (values come from plugin schemas), malicious plugin
  schemas or untrusted schema sources could inject XSS
- Inconsistent with main renderArrayObjectItem which properly escapes

Solution:
- Added escapeHtml() calls for propLabel in fallback array-of-objects rendering
  (both locations: generateFieldHtml and addArrayObjectItem fallback)
- Added escapeHtml() calls for option values in checkbox-group widget:
  - checkboxId (contains option)
  - data-option-value attribute
  - value attribute
  - label text in span
- Ensures consistent XSS protection across all rendering paths

This prevents potential XSS if plugin schemas contain malicious HTML/script
content in enum values or property titles.

---------

Co-authored-by: Chuck <chuck@example.com>

2026-01-08 15:38:08 -05:00

12 KiB

Executable File

Raw Blame History

View Raw