Flavio Espinoza
eScanner is a real-time cryptocurrency market scanner I designed, built, and deployed end to end for a private trading client. A trading terminal is where you place orders; eScanner is the layer in front of that — it watches an entire exchange at once, streams every market live, and surfaces the handful of moves actually worth acting on, filtered the way a trader thinks.
The hard problem was throughput. Scanning a whole exchange means ingesting a constant firehose of price and candle data, aggregating it into something queryable in near real time, and pushing it out to a browser that has to chart and re-rank it continuously without locking up. A normal database cannot aggregate that fast, and a normal request/response API cannot keep a browser live. So the system is built around two ideas: Elasticsearch as a time-series aggregation engine, and Socket.IO as a bi-directional live transport, with the whole backend clustered eight ways so the socket load has somewhere to land.
I was the sole developer — architecture, frontend, backend, the real-time layer, auth, billing, and the Nginx-and-PM2 production deployment.
The Architecture
eScanner is a two-tier real-time application — a React single-page app talking to a Koa.js API over both HTTP and a persistent Socket.IO connection — sitting on top of two stores: Elasticsearch for the market firehose and MongoDB for everything about the user. Market data flows in from the exchange, is aggregated in Elasticsearch, and is pushed to the browser as live events; the browser charts it, filters it, and lets a user pin the markets they care about. The sections below follow that path.
A. The Real-Time Market Data Pipeline
The core of the product: get an entire exchange's data in, make it queryable fast, and get it to the browser live.
- Market data is pulled from the HitBTC exchange through the CCXT library, normalized into a consistent OHLCV (open / high / low / close / volume) candle shape rather than a vendor-specific format.
- Candles are aggregated and stored in Elasticsearch, which is built for exactly this — time-series range queries and moving-average aggregations across hundreds of market pairs that a relational database would choke on.
- CryptoCompare supplies historical OHLCV to backfill the series, so a market has depth behind it the moment it appears, not just from the point the scanner started watching.
- Aggregated results are pushed to the client over Socket.IO the instant they are computed, so the scan reflects the market in near real time instead of on a refresh.
B. The Scanner and Filtering Interface
The front door, and the thing a trader actually stares at. A React SPA that turns the live feed into a sortable, filterable board with drillable candlestick charts.
- Built as a React single-page app with Redux for state — redux-thunk for async, redux-persist so a user's view survives a reload, and redux-socket.io binding live socket events straight into the store.
- Interactive candlestick charts rendered with React-Stockcharts over D3.js, so a user can open any market and see its candles without leaving the scanner.
- Customizable filters are the whole point of a scanner: percent-change thresholds, volume ranges, quote-currency toggles, and time-interval selectors, so a trader narrows hundreds of markets down to the few that match what they are hunting for.
- Watchlists and ignore-lists let a user pin the markets they care about and silence the ones they do not, stored per user so the board is personal.
- 50-plus components across sortable / filterable data grids, real-time price lists, and chart modals, using Microsoft's Office UI Fabric (Fluent UI) for consistent enterprise UI patterns.
C. The Event-Driven Real-Time Layer
What makes it feel live rather than polled. The browser and server hold an open Socket.IO connection and talk in events.
- A bi-directional Socket.IO event system (roughly fifty event types) carries market-data requests, filter changes, watchlist operations, and preference updates in both directions.
- User preferences sync through those events, so a change made in one place is reflected live rather than on the next page load.
- redux-socket.io wires the socket directly into the Redux store, so an inbound market event updates the UI through the same state path as everything else, with no special-casing.
D. Backend, Auth, and Data
The Koa.js server tier and the stores behind it.
- A Koa.js API — chosen over Express for clean async / await middleware — with routes split by domain (auth, user, billing, chat) behind a controller layer.
- Authentication on Passport.js with a local strategy and JWT session tokens (bcrypt-hashed passwords), with a higher-order component guarding protected routes on the client.
- MongoDB through Mongoose holds users, preferences, and the messaging data (conversations and messages), with connection pooling and auto-reconnect for a long-running server.
- A secure password-reset workflow: a cryptographic reset token emailed as a time-limited link.
E. Billing and Email
eScanner was a subscription product, so it had to take money and talk to its users.
- Stripe handles subscription billing — customer lifecycle, webhooks, and invoice tracking — so access is gated on an active subscription.
- SendGrid sends the transactional email: registration confirmation, password-reset links, and billing notifications.
F. Scaling and Deployment
How it actually ran in production, on a single well-tuned server rather than a cloud orchestration stack.
- The Koa API runs under PM2 in cluster mode across eight instances, so the Node process scales across every core on the box instead of pinning one.
json{ "name": "Api", "script": "index.js", "instances": 8, "exec_mode": "cluster" }
- Nginx sits in front as the SSL terminator (Let's Encrypt) and reverse proxy, redirecting HTTP to HTTPS and upgrading WebSocket connections.
- Socket.IO is load-balanced across five upstream servers using Nginx ip_hash, which pins each client to the same backend for the life of its connection — the sticky-session requirement that stateful WebSocket connections demand.
- Configuration and credentials are externalized through dotenv across development, staging, and production, so secrets live in the environment rather than in the codebase.
What This Project Shows
eScanner is the project where the whole challenge was real-time throughput — taking a firehose of exchange data and making it queryable, filterable, and live in a browser, as one person owning every tier. I chose Elasticsearch because the aggregations a scanner needs are exactly what it is built for, Socket.IO because a scanner has to be pushed and not polled, and an eight-way PM2 cluster behind ip_hash Nginx because stateful sockets need somewhere to land and a way to stay pinned. Frontend, backend, real-time, auth, billing, and deployment — I built and shipped all of it.