Agentic Care at the Edge: Should the Phone Decide, or the Cloud?

Agentic AI is arriving in healthcare from two directions at once. From above, large multimodal models in the cloud are getting good enough to reason over a patient’s record, recent vitals, and clinical guidelines and propose a next action. From below, the phones and wearables that patients already carry are shipping with dedicated neural accelerators capable of running surprisingly capable models without ever touching the network. The question facing anyone building IoMT software in the next few years is no longer whether to use AI at the point of care. It is where, exactly, the agent should run.

What “agentic” means at the bedside

For the purposes of this article, an agent is a piece of software that does more than classify a single signal. It maintains context, decides what to do next, and can take or recommend an action. At the point of care that might mean watching a continuous SpO2 stream, noticing a drift that matches the patient’s known pattern of overnight desaturation, deciding whether to escalate, prompting the patient to reposition, and only then alerting the on-call clinician if the pattern does not resolve. None of those steps are individually hard. Stringing them together with the right judgement, in real time, in someone’s bedroom, is where the architecture starts to matter.

The case for running the agent on the phone

Modern phones are not the constrained devices they were even three years ago. Recent generations of mobile silicon ship with dedicated neural processing units that can run multi-billion-parameter models locally, often in under a second per turn. For a monitoring use case the advantages are real.

• Latency is bounded. The signal does not have to leave the device. There is no round trip, no DNS, no TLS handshake, no queue at the inference endpoint. When the patient is deteriorating, the difference between two seconds and twenty is not academic.
• It works offline. Patients are not always on Wi-Fi. They drive, they sleep through router reboots, they go to rural clinics with bad cellular coverage. An agent that needs the cloud to think is an agent that goes dark exactly when the environment is most uncertain.
• Privacy is structural, not promised. A reading that never leaves the phone is a reading that cannot be intercepted in transit, logged at the wrong layer, or pulled into a training set by accident. For ambient sensing in the home, that matters.
• Cost scales with the user, not with usage. The inference is already paid for by the device. A monitoring program that triples in usage does not triple its cloud bill.

These advantages are not hypothetical. They are the reason that on- device speech recognition, photo understanding, and translation have almost entirely moved off the cloud in consumer apps. The same forces apply to clinical monitoring, perhaps more strongly.

The case for keeping the agent in the cloud

The phone is not, however, a small datacentre. There are real things a cloud-hosted agent can do that an on-device one cannot, and pretending otherwise leads to thin clinical products.

• Bigger models reason better. The frontier models that can integrate a patient’s history, current vitals, lab trends, and clinical guidelines into a coherent recommendation are not yet runnable on a phone. They probably will not be for some time.
• Cross-patient context lives there. Population baselines, cohort comparisons, fleet-wide anomaly detection, and updates from clinical evidence all naturally live in shared infrastructure. An on-device agent cannot see the other ten thousand patients.
• Auditability and governance are easier. A central inference service can log every prompt and response, version every model, and roll back instantly. Doing the same across a million phones is possible but considerably harder.
• Updates are immediate. When a new guideline lands, a cloud agent reflects it the same day. A device agent reflects it when the user next opens the app and accepts an update.

The cost of these advantages is the round trip. Even a healthy connection introduces hundreds of milliseconds of latency, and an unhealthy one introduces seconds or minutes. For some clinical decisions that is fine. For others it is the difference between useful and dangerous.

The honest answer: tier the agent

The framing of edge versus cloud is misleading because it implies a single choice. In practice the right architecture for an agentic IoMT product is almost always tiered, with different decisions made at different layers and with explicit rules about which layer owns which call.

• Sensor tier. Tiny on-device models running on the wearable or patch itself, doing signal cleanup, artefact rejection, and simple thresholding. Latency in milliseconds, no network required, no clinical judgement attempted.
• Phone tier. A capable on-device agent that holds the patient’s recent context, runs the moment-to-moment monitoring loop, generates prompts and reminders, and makes time-critical escalation decisions. Latency in the low seconds, fully offline capable, privacy-preserving by default.
• Cloud tier. A larger reasoning agent that the phone consults for non-urgent questions: weekly summaries, medication interaction checks, comparison against guideline updates, and anything that needs cross-patient context. Latency budget measured in seconds, with graceful degradation when the network is poor.
• Clinician tier. A human, eventually, for anything that crosses a defined threshold or that the cloud agent flags as beyond its competence.

The discipline this requires is unglamorous. Every capability the product offers has to be assigned to a tier, with an explicit story for what happens when the next tier up is unreachable. A well- designed system fails downward gracefully: if the cloud is gone, the phone still runs the monitoring loop; if the phone is gone, the wearable still raises a basic alarm.

How to choose the tier for a given decision

A few questions tend to settle the placement of any specific capability in practice.

• What is the latency budget? If a delay of more than a couple of seconds changes the clinical outcome, the decision belongs on the device. If the user will not notice a ten-second delay, the cloud is fine.
• How much context does it need? If the decision can be made from the last few minutes of one patient’s data, the phone has enough. If it needs the EHR, guidelines, or other patients, it needs the cloud.
• What is the cost of being wrong while offline? If a wrong answer when disconnected is worse than no answer, the feature should either run on device or refuse to run at all.
• What does the regulator expect to see? Some decisions need an auditable, versioned model with a paper trail that is easier to maintain centrally. That is a real argument for the cloud, even when latency would prefer the edge.

Designing for the next few years

The capability gap between on-device and cloud models is narrowing every release cycle, but it is not closing. The phones will keep getting more powerful, and so will the cloud, and the interesting agents will sit on top of both. Teams that build IoMT software now should assume two things. First, that whatever they ship today as a cloud-only feature will eventually move, at least in part, to the device, and the architecture should not make that migration painful. Second, that whatever they ship today as an on-device feature will sometimes need to call out to a larger model, and the protocol for doing so should be designed deliberately rather than bolted on.

The point of care is not a single place. It is a stack: a sensor on the body, a phone in the hand, a service in the cloud, a clinician on the other end of an alert. Agentic AI is going to live on every step of that stack within the next few years. The teams that do this well will not be the ones that picked the edge or picked the cloud. They will be the ones that decided, for every clinical decision, exactly where it belongs and what happens when that place is unreachable.

Related articles

Hardware

Biocompatible Implantable Sensors and the SafetyPatch: A Future Vision

Biodegradable SpO2, temperature, and blood pressure implants paired with a skin-worn SafetyPatch for calibration and alerting.

Security

Interoperability vs. Security: Balancing the IoMT Paradox

Open standards make connected devices useful and widen the attack surface. How mature programs hold both.