The things you cite specifically there aren't important, what are important are overrate calculations, time for the scheduled breaks, television crews being in place. Off the top of my head. I'm sure a more experienced high level umpire can cite a few more things that are time-crucial.
You tell the captains, team managers, the match referee, third and fourth umpires, broadcasters, groundsmen a starting time. They all get ready by then. You have all your calculations in place by then. And there are a few minutes to go and we're all set. Is it worth going around and telling all those people, redoing the calculations, changing up the times written down, etc just because we have 3 minutes to go? That alone will burn up 5 minutes. It's ridiculous. No one's going to ****ing do that. You just wait the 3 minutes and start at the time you announced you would start. Done.