Council Post: ​Reliability Ends Where The Signal Does

Philip Ellis is a founding frontend engineer at chrt, where he builds software for time-critical logistics operations.

getty

​A driver walks into the basement of a hospital with a time-critical shipment. The elevator doors close, the signal disappears and the smartphone app becomes useless. Back outside, the connection is perfect. Every API responds, and the status page is green. From the perspective of your team back in the office, the system is completely available. From the driver’s perspective, it’s useless. The bottom line is: Is your software available right now?​

The industry adopted its definition of reliability from web software, where an outage is visible to everyone. When the web app goes down, you know. But when a driver loses a signal, your dashboard stays green. During that time, things outside your control can happen to your software’s users. That’s when the always-on assumption is the point of failure that ruins the experience—without you knowing.​

What This Costs​

I’ve set up both software applications on dedicated Android hardware as well as a mobile application used by drivers worldwide. Both applications started out being always online, with millions of sessions per month. Engineering, however, was spending hours every week debugging issues stemming from user reports about losing connection.

The reports traced back to the same conditions:​

• ISP blips and outages meant users couldn’t complete their tasks, costing revenue and goodwill.

• Users lost their work and session mid-task.

• Low signal areas caused logistics teams to lose access to their drivers, leaving no trace of their previous tracked locations.

• Drivers struggled with completing proof-of-pickup and delivery tasks, such as signatures and pictures.​

These weren’t special cases. Rather, they represented the conditions under which the software was supposed to be able to perform.​

Offline-First Has Two Halves—Don’t Build Just One​

For your company’s software to perform, two sides of the equation must be satisfied. The first is local processing on the device, keeping the state consistent with the actions the user took. This isn’t as simple as just keeping something in device memory and continuing on when the internet reconnects. There needs to be ordering of actions, reconciliation of offline actions, failure states and, ultimately, a smooth user experience. If internet recovery causes errors or puts your users into the wrong state, you might end up with a worse situation than not working in offline mode. The second is backend visibility and consistency, which is what lets you run your business. You’ll want to know when a user goes offline and that recovery has landed in the correct order.​

Most teams build the first half and ship. The buttons respond, the screen updates and the work looks done. Weeks later, a reconnect replays the driver’s actions in the wrong order and the dispatcher doesn’t get the real picture. The backend team then spends time debugging and struggling to find what went wrong. You also have an incorrect business record to update and fix.​

What Changes​

Building for both halves changes three things about how you work.:

1. Design for the worst-case condition, not for the average. Always test the best-case scenario, but your main focus should be on low or zero connectivity situations. This is where your application breaking can cause the most friction with your users and customers.​

2. Provide an honest representation of what’s confirmed and what remains local. A status of “saved to this device” must differ visually from being reconnected and having data synced to the backend. Ambiguous feedback produces false confidence, and false confidence is worse than just being offline.​

3. Test under real-world conditions. Field experience can’t be reproduced from the desktop with simulated conditions. It’s best to get your hands dirty: Run a live setup in the field and test it end-to-end just like your users, either in the basement, rural areas or during an internet outage.​

That said, there are scenarios where it isn’t worth it to go down this avenue, depending on the business case. If you’re building a proof of concept or an early-stage product, this isn’t the feature you want to worry about and spend time building. First learn your use case and failure modes, then create a plan if the payoff is there or is deal-breaking for your customers.​

The True Measure​

Intermittent or absent network connectivity can equally affect home health care, construction, utilities, inspection, field service and agriculture. For that reason, all software used away from the desk should be subject to the same scrutiny and needs to be able to work under real-world conditions.

The driver in the basement gets the shipment signed for, photographed and timestamped. The elevator rises, the bars come back and the sync completes without anyone noticing it was ever pending. Nobody files a ticket. Nobody calls dispatch.​

That’s what reliability looks like in the field: not a green status page, but people using your system with full confidence and ease.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?