✈️Executive Product Overview
The Problem
Airfare decisions are time-sensitive, fragmented and repetitive. Prices change, different providers surface different inventory or timing, and manually re-running the same searches creates friction exactly when a traveler needs fast confidence. A single snapshot answers “what is the fare now?” but not “how has it moved, which provider is lowest, when was that low first observed, and should I act?”
The Solution
MyFlightWatcher turns those repeated searches into a monitored process. It checks configured providers automatically, applies trip rules such as route and nonstop requirements, records normalized fares, maintains provider and historical analytics, governs provider cadence and quotas, and pushes alerts when target conditions are met. Large raw provider payloads are archived outside the production fare database so operational history stays compact and usable.
What MyFlightWatcher Does
⚙️System Architecture
The application is intentionally split into operator, acquisition, data and archival responsibilities. This prevents any single provider response, scheduled task or user action from becoming the entire system.
Separation of normalized and raw data
The production database is used for queryable operational facts: fare, provider, airport, time observed, airline, status and related metadata. Large raw provider responses are not stored in each fare row; they are queued for export and collected into the Windows archive. This keeps analytics fast and prevents raw payload volume from dominating database storage.
🌐Provider Layer
MyFlightWatcher uses a provider-adapter model so each source can be queried and normalized independently while the rest of the application works with a common fare representation.
Provider Request Lifecycle
- Confirm the database is writable before spending an external provider request.
- Resolve the provider definition, credentials, endpoint and request parameters.
- Call the provider through the external API layer.
- Classify transport, rate-limit, quota and provider-specific errors.
- Normalize valid results into the common fare model.
- Apply trip qualification rules before writing fares.
- Record provider-run evidence and expose status to the control panel.
⏱️Scheduler, Cadence & Quota Governance
The scheduler is designed to prevent two opposite failures: checking too infrequently to catch a meaningful fare movement, and over-calling a provider until an API quota or rate limit is exhausted. Each active provider operates on its own cadence, with quota-aware request governance and provider-specific health monitoring.
🗄️Data, Storage & Archive Model
| Layer | Purpose | Representative Data |
|---|---|---|
| Trips | Defines the watched itinerary and alert target | Origins, destination, dates, target fare, active settings |
| fare_checks | Normalized historical fare ledger | Observed time, provider, origin, fare, airline, flight metadata |
| provider_runs | Operational evidence for provider execution | Run status, request count, fares found, failures and timing |
| alerts | Alert event history | Fare check linkage and notification evidence |
| app_settings | Runtime configuration | Provider schedules, credentials/settings and application controls |
| Windows JSON Archive | Durable raw-response retention outside MySQL | Full provider payload envelopes organized by date/provider |
📊Analytics, History & Alerting
The analytics layer turns repeated observations into decision support. Rather than showing only the latest fare, it can expose the historical range, provider-specific lows, averages, daily trends, the first time a provider's current low was observed, and the detailed fare ledger behind those summaries.
🧪90-Day Production Audit
After sustained production operation, MyFlightWatcher accumulated enough real provider traffic and fare history to evaluate reliability at system scale rather than from isolated test runs. The audit focused on execution stability, provider usefulness, alert correctness, analytics integrity and security controls. The result is a bounded remediation plan: preserve the stable acquisition architecture, retire low-value provider calls, correct one alert-state defect, and improve the presentation of data already being collected.
Confirmed defect The “$586 bug” was a stale historical-low alert, not evidence of a continuously available Expedia fare.
Code review found that the alert evaluator selected the lowest qualifying fare across the entire fare history and then re-evaluated that same historical row after the alert cooldown. A JetBlue/Expedia fare first observed on August 22 could therefore continue generating fresh buy alerts even when it was no longer present in a current provider response. This explains the repeated $586 notification and the inability to reproduce the fare on Expedia.
| Audit Item | Finding | Disposition |
|---|---|---|
| Alert freshness | Historical all-time minimum could be treated as the current actionable fare. | Surgical fix required Alert only from a fare observed in the current monitoring cycle / freshness window. |
| Historical analytics | Provider ranking intentionally uses minimum fare over the selected range. | Preserve Keep as historical evidence, but label it explicitly as historical low rather than current fare. |
| Provider estate | Google Flights and Expedia produced the most useful continuing price intelligence. | Simplify Retain only active Google Flights and Expedia calls; keep retired adapters in source for rollback/history. |
| Execution stability | No failed provider runs were reported in the supplied audit summary. | Strong Preserve architecture; no rewrite justified. |
| Run reconciliation | 20,596 total − 19,610 successful − 774 no-fare = 212 unclassified runs. | Reconcile Classify skipped, quota-paused, not-due or other statuses before publishing a closed audit total. |
Root Cause & Regression Boundary
Root cause: the alert evaluator queried fare_checks ordered by price ascending across all history, rather than evaluating a fare created by the current provider run. The cooldown prevented constant duplication, but once the cooldown expired the same historical minimum became eligible again.
Required correction: preserve historical lows for analytics, but gate notification eligibility on current-cycle provenance. A historical row may remain visible in history forever; it must not create a new “buy” alert unless a current provider response reproduces that fare.
Regression boundary: acquisition, normalization, persistence, history, dashboard intelligence, CSV export and archival remain unchanged. Only alert candidate selection/freshness is altered.
Analytics UX — Actual Production View & Repair
The 90-day audit made one UX problem impossible to ignore: the application had accumulated tens of thousands of fare observations, but the analytics presentation still behaved like an early prototype. The issue was not lack of data. It was that the visualization was failing to turn that data into useful decision support.
Production Analytics — Before
Native HTML reconstruction of the actual Analytics & Reporting screen. This preserves the visual logic of the running application without relying on a screenshot.
Fare Trend Over Time
Lowest Fare by Provider
Why this was not good enough
| Visualization defect | Observed behavior | User impact |
|---|---|---|
| No price scale | The chart has no visible Y-axis or dollar ticks, so the vertical distance between lines has no interpretable meaning. | The user cannot tell whether a movement is $20, $200 or $800. |
| Only two dates | The X-axis exposes only the beginning and end of the selected range. | Price events cannot be tied to a meaningful day or week. |
| High series omitted | The data query already calculates daily low, average and high, but the legacy chart renders only low and average. | Most of the available range information is discarded. |
| Historical low looks live | The provider panel presents the $586 Expedia minimum prominently without a freshness distinction. | A month-old historical observation appears actionable and contributed directly to the $586 bug confusion. |
| No current-fare marker | The visualization does not visibly separate the latest verified observation from the historical minimum. | The chart answers “what once happened” better than “what should I do now?” |
| No observation density | 36,130 saved fares are reduced to two thin lines with no signal of sample volume. | The user cannot see where confidence is supported by dense observation history. |
Production Analytics — Repaired
Same stored data, but redesigned to separate current truth from history and to expose the price range, trend, target and observation volume already available in the application.
Fare Trend
Fare Trend Over Time — Repaired
Provider Signal Quality
What was repaired in “Fare Trend Over Time”
| Repair | Implementation | Result |
|---|---|---|
| Meaningful dollar scale | Add a visible Y-axis with dollar ticks derived from the selected fare range, with modest padding rather than an empty canvas. | Movement becomes quantitatively readable rather than merely decorative. |
| Use all queried series | Render daily high in addition to daily low and average; optionally shade the low-to-high band. | The user can see volatility and the true daily market range. |
| Better time context | Render multiple date ticks across the selected period instead of only the first and last timestamp. | Spikes and lows can be tied to actual periods in the monitoring window. |
| Current vs historical | Add a current verified marker and visually separate it from the historical low marker. | The user immediately understands what is bookable now versus what was once observed. |
| Freshness semantics | Historical $586 remains on the chart as evidence, but is labeled history-only until a current provider response reproduces it. | The analytics UI reinforces the same correctness rule as the alert-engine fix. |
| Observation volume | Add a compact observation-density strip from counts already present in the data. | The chart communicates the scale and confidence of the production dataset. |
| Active-provider focus | Show Google Flights and Expedia as active signal sources; keep retired-provider history out of current-provider ranking. | The reporting surface reflects the simplified production provider estate. |
🛡️Security Status & Operational Controls
The production implementation includes a practical defense-in-depth model suited to a privately operated monitoring application. Security is part of the operating architecture, not an afterthought.
Audit conclusion
The evidence supports a precision-maintenance posture. The production architecture is stable enough that the correct engineering action is to isolate and repair the stale-alert defect, simplify the active provider estate, improve observability and reporting, and leave the rest of the working system alone.
🛡️Stability & Operating Controls
- Database write preflight: external provider calls can be stopped when the database cannot accept the resulting fare data.
- Non-destructive history behavior: normal monitoring does not require clearing fare history.
- Compact normalized storage: raw provider payloads are excluded from fare rows.
- External archival: a Windows collector polls the export endpoint and retains raw JSON independently from the hosted database.
- Provider isolation: one provider's error or rate limit does not have to invalidate the entire monitoring system.
- Operator visibility: control-panel tests and provider status make failures observable instead of silent.
