Before your page can show anything, your server has to answer the browser's first request. That's response time — and if it's over 2–3 seconds, a measurable share of visitors give up while staring at a blank tab. They never see your design, your offer, or your prices.
Load time is how long the full page takes to appear — images, scripts, everything. Response time is just the server's first answer, and it gates everything else. A fast server with heavy images beats a slow server with light pages, because visitors at least see progress.
1) Enable page caching (a plugin on WordPress, one setting on most hosts). 2) Put a free CDN like Cloudflare in front of your site. 3) Deactivate plugins you don't use. 4) If still slow after that, the host itself is the bottleneck — upgrade or move.
Test from a normal connection, not your office where the site is cached. And measure the server response specifically — a tool that reports only total load time can't tell you whether the problem is your server or your images.