Find a couple of photos of your prototype, and that will give you a good guide. Pay attention to how big the ballast pieces are compared to the tie cross section.
I used "Coarse" ballast on my layout, and while it looked good initially, I'm starting to feel that it looks too large. It's about 6-8 scale inches, which is about 2-3 times the size of what is used on the prototype, except underneath on long fills. Next time I do it, I will use "Fine" sized ballast. This is probably a better guide than "HO" or "N" scales.
As far as performance, there is no difference. It's for appearances only, and as long as you really lock it down with glue, it's fine.
Connecticut Valley Railroad A Branch of the New York, New Haven, and Hartford
"If you think you can do a thing or think you can't do a thing, you're right." -- Henry Ford
Craig North Carolina