Nope not at all, if you're attractive which I think it sounds like you are, 99% of the time it means he's shitting himself and can't make a move (or is in a committed relationship of course, but if you look at a guy more than once and he doesn't look back, this is probably the case).
But it's interesting you bring that up, from my mentality as I wrote, it means that women's attitude is "I've checked him out 3 times now, he doesn't have the balls, let's move our drinks elsewhere" but as you say, it's possible she simply doesn't think I'd be interested for whatever reason. That's not the case, it's simply difficult to walk up and approach a girl, even if she does show interest there's still often a long road to go before she'll really openly let you know she likes you.
As for basically "cold calling", going up to a woman who hasn't shown interest, that's a whole new level of difficult. I've done it plenty of times but it's not often down the years, I tip my hat towards every guy who regularly does it (even if I do watch cringe videos making fun of them).