If we're taking bow and arrow technology and putting it in a head to head competition with modern firearm technology, there's no contest. The modern firearm will win. The stipulation that we're going to limit the modern firearms to handguns is somewhat silly.
1.) Handguns weren't intended for long range accuracy. That's what Rifles are for. The slug, barrell length and casing are designed to purposely give up range in exchange for a decreased size.
2.) The archer with a bow has to use both hands. We're not giving the handgunner the option of full use of both of his hands. If we were, he'd have an assault rifle (or a sniper gun for the longer ranges of course.)
But even opperating under the limitation that we're restricted to "handguns", the limitations are still easily overcome. For one thing, I modify down a relatively powerful rifle into a handgun form. Below is a picture of the Smith and Wesson .50 Magnum handgun. This is NOT the longest barrelled verion of it, but I had trouble finding a picture of that variant. Since it is about 14 inches long and is still classified as a "handgun" then that could be a working limit for how big we can make ahand rifle.
From there, we simply rid ourselves of these bullets:
... and replace them with something more like this:
With 14" to work with, I think we can easily come up with a handgun that will outrange or at least equal any bow. And since many handguns are fully automatic, we can throw that in as well, if we want.
Lets remember, what was the biggest advantage of firearms historically? Ease of training. A good archer takes years to train. Teaching someone how to shoot a gun of whatever type takes only a few days. Frankly, its much easier to become very, very good at firing a gun than it is to get very very good at firing a bow and arrow.