The traffic was never the problem. About 42,000 people a month landed on this lender’s loan pages, and only 3.12 percent of them started an application on the site. The regional benchmark was 5.80 percent. Of the ones who started, fewer than one in four finished.
The company is a regional financial services firm headquartered in Orlando, with decades in consumer lending, auto financing and commercial real estate across Orange, Seminole and Osceola counties. Paid search and organic were doing their job. The website’s mortgage and commercial credit line application funnels were not doing theirs. Leadership knew it. What they did not have was anyone in-house who had built the kind of testing and personalization system that fixes it.
I was brought in to lead that. It turned out to be less about writing better headlines and more about rebuilding the plumbing so that testing was even possible, then earning the right, one experiment at a time, to show different borrowers different pages. What follows is how that went, in the order it happened.
Every borrower saw the same page, and the test tool made it flicker
A hospitality operator looking to refinance a property near the International Drive corridor and a young family shopping for a first mortgage in Lake Nona landed on the same layout, same message, same button. That is not personalization failing. That is personalization not existing.
The tooling underneath was worse. The site was running an old, unmanaged copy of Adobe Target’s at.js 1.8, dropped in through an outdated tag manager. Every time the internal team tried a simple A/B test, the page loaded, then rearranged itself. Cumulative layout shift was regularly above 0.28. So the tests they had run were not just inconclusive, they were actively hurting the page they were trying to improve. And the session identifier broke when a visitor moved from the marketing site into the secure loan portal, which meant nobody could connect a landing page to a submitted application.
Fixing the plumbing before touching a headline
We moved delivery to the Adobe Experience Platform Web SDK on the Edge Network, in two parts. On the public marketing site, the Web SDK went in with pre-hiding CSS scoped only to the hero containers that would ever change, capped at 250 milliseconds. No more white flash, no more rearranging. On the secure application, a React single-page app, we did the decisioning server-side through the Target Delivery API in their Node.js middle tier, so the page arrived already decided.
The analytics engineers and I then built an Experience Data Model schema that captured what we actually needed: customer state, geography, calculator interactions, engagement depth. No personally identifiable information. We unified the Experience Cloud ID across the marketing and application domains and turned on Analytics for Target, with Adobe Analytics as the single source of truth. For the first time, a click on the marketing site could be followed to a funded loan milestone coming back from the core banking system.
What the compliance officers would let us target, and what they would not
This is a bank. Gramm-Leach-Bliley and federal fair-lending rules are not a checklist you run at the end. I set up a weekly review with the enterprise risk and compliance officers before a single audience rule existed, and we wrote the boundaries down together.
No personalization model could use protected demographic attributes or anything that could be read as a proxy for creditworthiness. Targeting was limited to behavior and context the visitor supplied themselves: the loan category they picked, the down payment range they typed into a calculator, where they were coming from geographically, and the campaign that referred them. Every rule we later shipped went through that review first. It slowed us down for two weeks and saved us the entire program.
A multivariate test on the mortgage page: headline, first step, and proof
Before building audiences, I wanted to know which page structure worked for everyone. So we ran a full-factorial multivariate test on the primary mortgage landing page, two options for each of three elements, which gives you every combination and lets you see whether elements help or hurt each other.
| Element | Control | Challenger |
|---|---|---|
| Hero message | “Competitive Mortgage Rates for Central Florida Homeowners.” | “Orlando’s Trusted Mortgage Partner. Pre-Approved in as Little as 15 Minutes.” |
| First step | Static form: name, phone, loan amount | Three-card wizard: Buy a Home, Refinance, Cash-Out Equity |
| Proof | Equal housing lender and FDIC badges | Star rating from verified regional reviews, plus “Helping 1,420+ Central Florida families secure homes this year” |
Variations went out as JSON payloads through the Web SDK, not as scripts rewriting the page after it loaded. That matters more than it sounds. It is the difference between a test that measures the page and a test that measures the flicker.
The winner: a local speed message, a three-card wizard, and real local numbers
All three challengers won, and they won together. The combination of the Orlando speed headline, the three-card wizard and the localized proof beat the control on application starts by 53 percent, 3.12 to 4.78 percent, at a p-value of 0.002. Field engagement on the first step went up 38.6 percent. Bounce dropped from 52.4 to 41.2 percent. And the completion rate on applications that were started went up too, 22.4 to 26.9 percent, which told us the extra starts were not junk.
The wizard is the piece I would point to. A static form asks a stranger for their phone number. Three cards ask which of three things they want to do. Nobody hesitates over that, and once they have clicked one, they are in.
Why the payment calculator lost
The internal team’s favorite idea was an interactive payment calculator on the landing page, sliders pre-set to a $385,000 median home value for Orange and Osceola counties. We tested it as a follow-up against the winning wizard. It held people on the page 44 seconds longer and produced fewer application starts. People were playing with sliders instead of starting. A calculator is a great tool two steps later. On the landing page, it was a very engaging way to not apply.
Four audiences, four experiences, and a permanent 10 percent holdout
With the layout settled, we started showing different borrowers what they came for.
- Commercial and hospitality borrowers. Visitors from IP ranges mapped to commercial zones like Downtown Orlando and Lake Buena Vista who had viewed commercial real estate or business lending content at least twice in 14 days. Consumer mortgage imagery came out. Commercial financing, local underwriting speed and a direct booking link to an Orlando-based relationship manager went in.
- In-state first-time buyers. Central Florida visitors engaging with first-time buyer content, down payment calculators and FHA criteria. The page led with Florida down payment assistance, specifically Hometown Heroes terms, and pointed to advisors at the Downtown Orlando, Winter Park and Lake Nona branches.
- Out-of-state relocators. Out-of-state geolocation, mostly Northeast and Midwest, arriving on Florida relocation campaign keywords. “Moving to the Sunshine State? Lock your rate from anywhere. Completely digital closings,” with guides on Florida property taxes and insurance.
- Returning incomplete applicants. Anyone who had reached step 2 or beyond and dropped off within the previous 72 hours. The hero became a recovery card: “Pick up right where you left off. Your application is 60% complete.” One click passed a secure state token, skipped the fields they had already filled, and dropped them at document upload.
The week desktop recovery flatlined, and what it turned out to be
To keep this running after launch, I set up an Optimization Review Board: product managers, UX, engineering leads and compliance, every two weeks. Every new rule or test had the same checklist before it went live: compliance sign-off, cross-device validation with the Experience Platform Debugger and Charles Proxy to check the actual payloads, edge latency under 80 milliseconds (we ran a median of 42), and a sample ratio mismatch check in the first 72 hours of traffic.
The checklist is what caught the problem. Three weeks into the recovery campaign, mobile recovery conversions were climbing and desktop completions had gone flat. I sat with the client’s front-end engineers and session recordings. A session timeout on the legacy core banking system was forcing desktop users returning from corporate networks to clear their cookies, which killed the pre-filled application token. They were clicking “pick up where you left off” and landing on a dead session.
The fix was a patch that handled an expired token gracefully: instead of ending the session, route the user through a quick two-factor SMS pin and restore their application. Five days after it shipped, desktop completions in that segment were up 14.2 percent.
What changed over six months
These are the six months after full deployment, compared with the baseline before the project. The multivariate winner shipped first and produced the first jump. The audience experiences and the recovery campaign came after and produced the rest.
The recovery campaign alone brought back 1,840 abandoned applications and turned 712 of them into submissions. Over the two quarters after rollout, the funnels were traced to an additional $68.4 million in residential mortgage volume and $24.2 million in commercial originations. Largest Contentful Paint went from 3.8 to 1.9 seconds, layout shift from 0.28 to 0.02, and two independent compliance audits came back with zero privacy or fair-lending findings.
What I would not claim from these numbers
The page speed work, the multivariate winner and the personalization all went out within a couple of months of each other, and rates were moving during that window too. The holdout groups are the honest part: they tell us the personalized experiences beat the default page for those audiences. They do not tell us how much of the six-month total came from the layout versus the audiences versus the faster page. My read is that the layout did the most, because it applied to everyone, and the recovery segment did the most per visitor. That is a read, not a measurement.
And 742 applications a month was possible partly because the old site was leaking so badly. A lender that already starts at the 5.8 percent benchmark should expect a smaller number from the same work.
The order that made it work
- Fix delivery before testing anything. If the page flickers, every test you run is measuring the flicker. Scope the pre-hide, cap the timeout, decide server-side where you can.
- Connect the front door to the back office. One visitor ID from landing page to funded loan. Without it you are optimizing for form starts and hoping.
- Write the targeting rules with compliance, not for them. Behavior the visitor gave you, never attributes about them. Weekly, in the room, before the first rule ships.
- Find the layout that works for everyone first. A coarse multivariate test with big cells beats a fine one you cannot fill. Build audiences on a winner, not a guess.
- Kill the favorite idea with data. The calculator was engaging and lost. Time on page is not a goal.
- Keep a holdout forever. Ten percent of every segment sees the default. That is the only way to know personalization is earning its keep.
- Check the sample ratio in the first 72 hours, and watch each device separately. The desktop token problem would have hidden inside a healthy blended number for a month.