Message Processing
==================

Thread Architecture
-------------------

.. only:: drawio

   .. drawio-image:: architecture.drawio
      :page-name: thread-architecture

Main Thread
-----------

1. Receives message from RabbitMQ via ``dashboard.notifications.common.run``
2. Calls ``should_create_ticket(message)`` to validate
3. If valid: adds ``retry_count=0`` and enqueues to ``queue.Queue``
4. If invalid: discards message

Processing Thread
-----------------

For each message:

1. **DB Check**: Query ``DashboardAlarm`` by ``id``

   - If not found → raise ``TicketError`` (non-transient) → discard

2. **TTS Check**: Find overlapping maintenance tickets

   - If found → update existing ticket
   - If not found → create new ticket

3. **DB Update**: Set ``DashboardAlarm.ticket_ref`` to ticket number

4. **Error Handling**:

   - ``TicketError`` → log error, discard immediately
   - Other exceptions → retry with exponential backoff


Retry Mechanism
---------------

.. only:: drawio

   .. drawio-image:: architecture.drawio
      :page-name: retry-mechanism

- **Exponential Backoff**: ``delay = min(BASE_DELAY * (2^retry_count), MAX_DELAY)``

  - ``BASE_DELAY = 1.0`` second
  - ``MAX_DELAY = 60.0`` seconds

- **Max Retries**: ``MAX_RETRIES = 5``

- **Implementation**: Uses ``threading.Timer`` to re-queue after delay

- **After Max Retries**: Log error and discard message

Retry Schedule:

.. list-table::
   :header-rows: 1

   * - Retry
     - Delay
   * - 1
     - 1s
   * - 2
     - 2s
   * - 3
     - 4s
   * - 4
     - 8s
   * - 5
     - 16s
   * - After 5
     - Discard


Thread Safety
-------------

- **Queue**: ``queue.Queue`` is thread-safe
- **Timers**: Tracked in a lock-protected list
- **DB Session**: Each message gets its own session via ``session_scope()``


Signal Handling
---------------

The service handles graceful shutdown on:

- ``SIGTERM`` (standard termination)
- ``SIGINT`` (Ctrl+C)

**Shutdown Behavior**:

1. Set ``stop_event``
2. Main thread exits (RabbitMQ consumer stops)
3. Processing thread terminates (daemon thread)


Logging
-------

Standard Python logging with format:

.. code-block:: text

   %(asctime)s - %(name)s - %(levelname)s - %(message)s

Log Levels
^^^^^^^^^^

- **INFO**: Service startup/shutdown, ticket creation/updates
- **WARNING**: Transient errors with retry information
- **ERROR**: TicketError, max retries exceeded
- **DEBUG**: Message enqueue/dequeue, validation results

Example Logs
^^^^^^^^^^^^

.. code-block:: text

   2026-04-02 10:15:30 - tts_api_notifier - INFO - Starting TTS API notifier service
   2026-04-02 10:15:31 - tts_api_notifier - INFO - Processing thread started
   2026-04-02 10:15:32 - tts_api_notifier - INFO - Created ticket TTS-12345 for alarm 67890
   2026-04-02 10:16:05 - tts_api_notifier - WARNING - Error processing alarm 67891 (retry 1): Connection timeout
   2026-04-02 10:16:05 - tts_api_notifier - WARNING - Scheduling retry 2 for alarm 67891 in 2.0s


Dependencies
------------

- ``dashboard.notifications.common`` - RabbitMQ integration
- ``dashboard.correlation.db`` - Database access
- ``click`` - CLI framework
- Python standard library: ``threading``, ``queue``, ``signal``, ``logging``, ``json``