Stream processing at scale
for

The challenge

Business

Verizon needed a cloud backend to deliver a Smart Home solution (consumer IoT platform) to their subscriber base.

Verizon needed a system where the specific applications & services can rapidly evolve over time and new services & applications can be deployed on the same platform, without knowledge of their specifications ahead of time.

Technical

The system has to scale to tens of millions of homes per metro, with minimal latency (example: when a user presses a button in the mobile app, the home's lights need to come on).

Each device in each home sends continuous streams of data for energy consumption and monitoring updates, resulting in unique scale, fault-tolerance, and legal compliance requirements.

Why is Verizon using the SiliconMint platform?

1. Performance

Stream Processor: handles 1.5–2 million events/second/node, on commodity hardware, with typical topologies. 99th percentile latency of stream processor is ~8ms.

To handle 20 billion events/day, ~231k events/second must be processed. Even with a complex topology, this can be done on a small cluster of SiliconMint runtimes.

The stream processor's performance means that in typical scenarios, we see that external I/O subsystems (network, queues, & others) are the primary bottleneck.

2. Flexibility

Visual Designer: it is typically > 10x faster to craft applications with the Visual Designer than by hand with an engineering team.