On-Prem to Cloud Migration: Running the Systems That Can't Move Yet

If your on-prem to cloud migration plan assumes one clean cutover, expect that plan to change. Most migrations run hybrid for months or years. The real work is deciding how the systems left behind connect to the ones that moved, how people sign in to both, and who answers when something breaks in between.

You probably know the situation. The customer portal and the new APIs already run in AWS or Azure. The ERP, the warehouse database, a licence server tied to a physical dongle and a nightly mainframe batch job are still in the server room. The migration plan has a slide that says "phase 2", and nobody has a date for it.

This piece covers the four decisions that make that period stable or painful: network connectivity, shared identity, latency, and ownership of failures that cross the boundary.

Why do so many systems get left behind?

Some systems stay behind because of contracts, not code. Common examples are a vendor licence tied to specific hardware, a regulator that wants data kept in a particular facility, or a support agreement that ends if you change where the software is hosted.

Others stay because nobody fully understands them. Think of a 15-year-old application with hard-coded IP addresses, a shared database that six other tools read from directly, or a nightly file drop to a partner over SFTP. Moving those safely means untangling them first, and that takes longer than the migration schedule allowed.

A hybrid state does not mean the planning failed. It is the normal shape of a migration. The real mistake is treating it as temporary and building it with temporary decisions.

What does it cost to get the hybrid period wrong?

The cost rarely arrives as one big outage. It shows up as slow, confusing incidents that eat weeks of your team's time.

• A cloud service calls an on-prem database over a VPN that was sized for admin traffic. Pages that used to load instantly now take several seconds, and the team spends weeks blaming the new code.
• Staff have two logins, one for the old domain and one for the cloud tenant. When someone leaves, only one gets switched off. Months later, that former employee can still reach the ERP.
• An order fails somewhere between the cloud checkout and the on-prem inventory system. The cloud team sees a timeout. The infrastructure team sees a healthy server. The customer gets no confirmation because nobody owns the gap.

In each case, nobody made the decision on purpose. You can make it now, before an incident forces it.

When can you handle this without outside help?

You may not need outside help if what's left on-prem is a few internal tools with light traffic and you already use one identity provider for single sign-on. Set up redundant VPN tunnels, name an owner for each flow, add link monitoring and look at it again in six months.

Outside help is worth it when customer-facing traffic crosses the boundary, when user accounts live in two separate directories, or when the setup has been "temporary" for over a year and incidents keep landing in the gap.

How should the old and new environments connect?

Choose the connection based on how much traffic crosses it and how much an outage would hurt. A site-to-site VPN over the internet is quick to set up and fine for admin access, light API traffic and batch jobs. A dedicated link such as AWS Direct Connect, Azure ExpressRoute or Google Cloud Interconnect gives steadier speed and capacity, but the carrier can take weeks to set it up.

Decision rules:

• If customer-facing requests cross the boundary on every page load, plan for a dedicated link with a VPN as backup.
• If only nightly jobs and admin tools cross it, a pair of redundant VPN tunnels is usually enough.
• If you can't tell yet, measure first. Log every cross-boundary call for two weeks, then choose.

Two details get missed often. The first is overlapping IP address ranges between the office network and the cloud network, so plan addresses before you build because renumbering later is miserable. The second is DNS, the system that turns server names into addresses. Cloud services need to find on-prem servers by name, and on-prem servers need to find private cloud services. Set up forwarding in both directions on day one.

Should you run one identity system or two?

Run one. Staff should sign in once and reach both the old and new systems. When someone leaves, a single switch-off should remove their access everywhere.

In practice, that usually means keeping Active Directory on-prem as the master list of users for now, syncing it to Microsoft Entra ID or connecting it to Okta, and putting cloud apps behind single sign-on using SAML or OIDC. Older on-prem apps that only understand LDAP or Kerberos keep pointing at Active Directory. Once most apps are in the cloud, you make the cloud directory the master list.

System-to-system logins need the same care. If a cloud function reads the on-prem database, give it its own named account with only the access it needs. Keep its credentials in a secrets manager, never as a shared admin password copied into a config file.

If the answer to "how many places do I disable a leaver?" is anything other than one, fix identity before you migrate anything else.

How do you handle latency between old and new systems?

Assume every call across the boundary is slower and less reliable than a call inside one data centre. An app that made 200 small database queries per request was fine on a local network. With 20 to 40 milliseconds per round trip, those queries add up to seconds.

Try these fixes in this order:

1. Keep chatty systems together. If an app and its database talk constantly, migrate them together or leave both on-prem.
2. Batch and cache. Replace one-row-at-a-time calls with bulk requests, and keep a copy of slow-changing reference data on the cloud side with a clear refresh rule.
3. Use a queue. Put a queue between the two sides for anything that doesn't need an instant answer, such as order sync, invoice posting or stock updates. If the link drops, messages wait in line instead of failing.

Give every cross-boundary call a deliberate timeout and retry rule. A 60 second default timeout on a request a customer is waiting for is a bug.

Who owns a failure that crosses the boundary?

Name one person before go-live. Hybrid incidents drag on when two teams each prove their side is healthy while the fault sits between them.

What closes the gap:

• One owner per flow, not per system. For example, someone owns "checkout to inventory" from end to end, including the alert, the runbook and the fix.
• One trace across both sides. Give each cloud request a tracking ID, pass it into the on-prem logs, and send both sets of logs to one tool such as Datadog, Grafana or Elastic. Without that, someone ends up matching timestamps by hand.
• Monitoring for the link itself. Track latency, dropped traffic and tunnel status, and give the link its own alerts like any other service.
• A runbook for when the link goes down. Write down what slows down, what waits in a queue and what you tell customers.

What order should you do this in?

1. Map every flow that crosses the boundary, with its traffic volume and the teams that depend on it.
2. Fix identity so one account and one offboarding step cover both sides.
3. Choose connectivity from that map, and plan IP ranges and DNS.
4. Redesign the chattiest flows by moving pairs together, batching, caching or queueing.
5. Give every cross-boundary flow an owner, tracing and a runbook.
6. Give each system that stayed behind an exit date and an exit condition, even if the date moves.

Without step 6, a hybrid setup can become permanent by accident.

A next step if you're in the middle of this

Geminate Solutions has shipped 50+ products, including an exam platform handling 10M+ requests per minute in production. Much of that work comes down to making old and new systems run as one. If you'd like a second opinion on your hybrid setup, the cloud and DevOps team offers a free review of your migration plan under NDA, with a reply within 24 hours.