You can attempt to remove the petals if you like, but it is a very
labor intensive process. I made two batches two years ago, one batch
with just the petals and the other made with the whole flower (stem
removed). I thoroughly enjoyed both, but the difference between the
two was slight at best. The wine made with the whole flower was a
little nicer golden color, it had a bit more body and I personally
preferred it over the wine made with just the petals. BTW, both were
Jack Keller recipes.

Paul