I'd say anytime there are too many players on the field the ref didn't follow the proper subbing procedure and isn't game smart.
Unless a coach maliciously slides on another player behind a refs back, all the subbing is in the refs view.
Players at centerline, let field player off, let sub on, never causes problems.
IMO there is no need to rush this.
Also, if you get a line up card before the game, you'll see when the game starts how many subs are on the bench/sideline.
After each subbing I count the bench/sideline players and if I can't account for them all, I count the field players.
Just this Sunday I had to warn a coach who would yell "sub" and send his troops without waiting for my acknowledgement. (That after at first being nice and telling him to wait for my signal)
I may be patient with U 10's and explain, but if a coach does it at the older levels and "forgets" it's rounds of yellow for everybody , so they learn quickly.
Even then after so many years, I keep on counting the players after each subbing, regardless of correct procedure.
It's good to be paranoid:-)