Do you have an Intel chipset by any chance? You can easily boot to Windows (provided you have a windows install CD) with Boot Camp. Another possibility is Parallels Workstation, which emulates the Windows enviornment on a Mac. I *beleive* this works on both PowerPC chipsets and Intel chipsets.
Another less cost effective solution is to buy a cheaper older computer (MSTS does not require a ton of RAM or Video RAM or processor).
I am in the same boat you are in, booting to and from Windows is the better of the two solutions as Paralells is quite the RAM hog and you may (i.e will) run into some graphical issues as both Windows AND MSTS are being emulated.
The only train simulation software that I know of for Mac platforms is the TrainPlayer program, which is just a simulation of a model railroad, I bet if you do a search here you can find more information on it, but it is nowhere near BVE or MTST or Trainz.
Go here for my rail shots! http://www.railpictures.net/showphotos.php?userid=9296
Building the CPR Kootenay division in N scale, blog here: http://kootenaymodelrailway.wordpress.com/
I have an older PowerPC mac, so it's kind of hard to use boot camp or anything like that. I do have a Window$ computer with Trainz and M$T$, but it's got a weird sort of 4 year old graphics card in it despite the fact that it's brand new, so I want to use the Mac because it's better in the graphical area in this case.