On one hand, expectations change with time. Most people outside my bubble look at interfaces I like using and say they look "ugly" and that they're "weird" (their words); they wouldn't have said that when I was younger.
On the other hand, some "annoyances" are actually removable _barriers_. Accessibility comes to mind. If you take software that does not work with assistive technologies (ATs) and fix it, AT-users might move on to the next accessibility issue. But they'll be markedly happier than before, when they just couldn't use it.