Rabbits are jumpy. It's not just how they get around, it's how they keep themselves safe. While some rabbits don't mind if you just reach over and pet them, most require a calm slow approach. It takes patience to make the experience of petting enjoyable for both you and your rabbit, but it's worth it.
Your rabbit is a prey animal and probably sees you as a predator, so a fast direct approach is not best unless your rabbit knows you well and already trusts you. Approach slowly and give him time to check you out and get relaxed around you before trying to pet him. If he stomps his foot, runs and hides in the corner or freezes in place, it means he thinks the situation is dangerous. Just sit down and hang out near him until he relaxes before approaching closer.
A gentle touch will go a long way with your rabbit. Don't make any fast movements, pat or scratch him very hard. Most rabbits enjoy very gentle scratching rather than long petting strokes. Try scratching around the base of his ears, between his eyes and on his rump. Some rabbits like to be scratched under the chin, while others hate it. Very few rabbits like to be touched on their belly, feet or ears, so avoid those areas until you know your bunny well.
Establishing trust with your rabbit will make him seek out attention from you and enjoy being petted. Spend time sitting near him but not touching him. Eventually he will get curious and come over to you. Pet him when he approaches and let him leave when he is ready. Don't try to restrain him or force him to be petted. Over time he will learn that approaching you and being petted is a positive experience and he'll seek out more and more attention.
If your bunny is especially shy or unsure about being petted, a few treats can help convince him that it's fun. A small piece of apple, melon, carrot or pear should entice him to spend time near you. Set the treat on the ground near you instead of holding it out to him. He'll be more likely to approach and won't accidentally nibble your hand while trying to get the treat. Wait until he is either done or nearly done eating the treat before petting him, and make sure he notices your hand approaching so that he isn't startled.