What will affect performance are the applications and associated software services that get installed and steal available memory and CPU resources from your system. Many common applications, such as iTunes, Microsoft Office, Dropbox, Skype, and antivirus programs, install startup items that run when you boot the computer, additional services that run all the time in the background, and scheduled tasks that run periodically to perform maintenance tasks. Each item by itself is not significant, but add them up over several years, and suddenly your fast computer is more like a turtle than a Ferrari.
Installing updates to your operating system and applications is crucial to the reliability and security of these software components, but typically each new version is bigger and more demanding than the previous one. This is the paradox of computer ownership: you keep your computer up-to-date as you should, but with each update, you might be slowing it down.
When you buy a computer, the three typical performance-related categories are how much memory, how fast the processor, and the type of internal hard drive (e.g., HDD vs SSD). Memory is inexpensive, so buy as much as you can afford. Faster processors are more expensive and for average users, don’t make that much of a performance difference. HDD vs SSD does make a big difference, and the price of SSDs (solid state drives) is dropping dramatically, so if you can afford a SSD instead of a HDD, you won’t regret it. But, think about how much space you will need, since SSD storage is still more expensive than traditional HDD storage.
In terms of upgrading existing computers, upgrading the processor is typically not an option, but upgrading the memory and/or upgrading to a SSD often is very viable and cost effective. If you know what to look for and how to do it, you could seek out the services, scheduled tasks, and performance hogging applications, and disable or remove them. But this is often a trial-and-error activity, taking much time.
Another option is to reinstall the operating system and the applications you require today (of course with first backing up your data), thereby getting a fresh start with your existing hardware. But this can take many hours of work, so you might just prefer buying a new computer or upgrading the hardware of the existing computer.