read sword of the immortal and veganbond then we can get into it further.
don't get me wrong guts epic but he doesn't do well against fast opponents, especially one thats fast and has the ability to cut a human in half with one handle swing.
http://i55.tinypic.com/2drgwgj.jpg
heres a picture that can help you Imagen it. the dragon slayer would be close to impossible(even tho we seen him done it before) to stop midway giving musashi the chance to step into his range and cut guts head off clean ( like how his about to do in the picture with his wabiki or however you spell that ) and im comparing guts with out the berserk armor ability
First of all it's incredible hard to value "speed" in a drawn anime which contains of single images. However, there were allusions that Gutt's speed is absolutely beyond human capabilities at some scenes:
http://manga.animea.net/berserk-chapter-2-page-52.html
Looks fast, doesn't it? He also managed to get himself into position behind that guy afterwards and to kill him.
Against any kind of Samurai or other japanese sword fighters: No matter how skillful the fighter, with a "natural" Katana he would not be able to hit Gutts anywhere beside at his head or his unarmored arm, and Gutts will very well know so.
It's also not like Gutts would be a random spear carrier #19515 who lets his life in battle because he didn't use the range advantage of his weapon to his advantage. Gutts, with his skill and knowledge, would definitly know that he has a range advantage and use it to it's best. Apart from that he also has a crossbow and a cannon, just in case there might be a possibility to use them at one point, and an iron hand with which he crushed quite some skulls already when it seemed like he couldn't defend himself anymore.
He also managed to catch Serpico's quick sword with his bare hand and bend it so his hand was safe. Just as an example of how quick and skillful he is.
Apart from this:
-He's in a perfect age for fighting probably
-Even so, he has incredible experience: After all he's the boy who hugged his sword at the age of six. His sword was pretty much the only constant in his life and some kind of friend to him. He's also fighting since very early childhood and never stopped doing so ever since - real fighting, not just some practice matches mostly.
-And something which isn't very useful for a duell, but makes him even more complete as a warrior: He knows every kind of war and fighting: He can fight in any kind of armor, on horseback, knows about huge scale battles, besiegement, probably some about military tactics and acted as a leader for the raiders. Probably he can use many other weapons besides swords, such as polearms, bows and crossbows.