Prepare your grill by filling the drip pan or foil pan with water and turn grill controls on high. Grill is ready when you start seeing steam (approximately 20 minutes). Even though grill is on high your goal is to have the temperature be between 275˚ F and 300˚ F. If your grill gets too hot, add more water.
In a bowl, combine olive oil, vinegar and lime juice. Add salt and castor sugar, to taste, and set dressing aside.
Place oysters on cooking grid and grill over fire until shells open (approximately 8 minutes). Remove from fire and let cool.
Remove top half of shell and discard. With an oyster-shucking knife or blunt back of a small knife, disconnect oyster meat from the bottom shell. Place spoonful of prepared dressing over each oyster. Sprinkle herbs on top of oyster before serving.