I can sympathize with your situation.
My mother and grandmother abosulutely hated trains, mainly because of having to wait at crossings. My grandfather loved trains and that was one of the things that rubbed off on me. But, for about 4 years or so, my mother would literally beat me for mentioning trains. She destroyed the only train set that I had, and would tell me to "grow up" when she noticed me watching trains when we were out and about.
Grandpa was extremely henpecked about his Lionel train set (mid 70's Silver Star set). He would often call me over to do some "painting" or work on his car, and give grandma some money to go shopping, and we wold play with that train for hours, or we would dissapear to the local railyard to watch trains, or even on occasion, get a tour.
Like everthing else, my mother finally gave up, and "allowed" me to indulge myself in trains. She even bought me a HO scale train set with the hopes that I would "grow out of it".
Grandpa left me his train set in his will, but grandma still wanted me to "grow up", so she gave it to my bratty nephew, who prompty destroyed it.
I since then have gotten myself a Silver Star set, and the same train that I had for many years until my mother, in one of her drunken rages, destroyed. (A 0-6-0 battery powered train with a tender, B+M Gondola, and a ATSF caboose. I was misty eyed when I found that at a train show.)
Even when you get older, people, especially the stupid ones, will give you a hard time about being a railfan. I had postcards of trains taped on the wall of my rack on my old ship, the USS Annapolis (SSN-760). I was made fun of by quite a few people for my being content to spend time in France just watching the TGVs go by.
I even brought the shell to my Lionel 221 with me underway one time, and I spent a fair chunk of my spare time sitting in the torpedo room with a wire brush and a dental pick cleaning out the salt and rust off of it.
Giving up on something that you love, just because people want to be immature, is wrong. If the taunting is bad enough, seek help. I am pretty sure that not all of the people that you encounter think that playing with trains is a bad thing, you may find some like minded people and start a club.
Quite a few sailors have asked to come over to my house to see all of the trains that I have. My train collection is rumored to be the largest in the area.
When I was in the 8th grade, my Literature Teacher had each of us do a report on our favourite hobby, and I did my report on model railroading. I brought in a few of my HO scale engines, and cars, and the gondola from my grandfather's set, and brought photos of my first layout.
Some idiots made fun of me for that, but at least the teacher paid attenton to that and failed them on their projects and sent them to the princpal. (Most had done their projects on videogames and such, some brought in pets, but out of the entire lot, I was the only one with a train.) (My future wife Nicholle did her report on her cat, Thread, who was happy to be part of my report as well. I met her in 8th grade.)
Just keep your head up, and remember, do what makes you happy (within reason of, course), not because it's the popular thing.