While walking, you will sometimes feel a stone in your shoe which is so small you're able to walk on it. It might be bothersome, but it's innocuous. When is the right time to remove it, as soon as it bothers, or at your next stop?
Removing it involves pausing at a suitable location where you can sit (which usually involves taking off your backpack), untie your shoe, shake the stone out, and put your shoe back on. Seems simple enough, but momentum is difficult to regain, in walking as in all disciplines.
The parallel with work is that we often have small things that don't completely impede activity, but are a drag which makes work less pleasant. I'll volunteer my own problem: I came back with my clothes of the camino, and my wardrobe is extremely limited. I could go to where my things are stored, but then I'd lose half a day in this task. My work momentum is good and I don't want to lose it. Should I resolve this issue when I need a break from work, or should I address it immediately?
The wise person would answer "it depends", and may consider having a limited wardrobe to be a vain impediment to work. Let the wise person know that my entire wardrobe consists of exactly one shirt, one underwear, three pairs of socks, one pair of pants, and two bathing suits (it's a long story). This partially binds me home instead of being able to work at the library, where I accomplish better concentration.
The wise person would also question the validity of some assumptions: do you really need to take your backpack to take off your shoe? do you really lose momentum or you're just prone to procrastination and can't help but loiter once you sit down? Do you really have to untie your shoe or can you pull it out and slip it back on?
And the wise person would whisper to your ear: you're overthinking this. Just get down to work, and when you need a break, go fetch your clothes. Godspeed.