I can only speak to my own between-the-ears experience. I recall having a plastic, simple, toy train set that I would push around our wooden floors when we lived nearely Sudbury back in the mid-fifties. I was three at the time. I really loved that four item train set. I don't even think it had wheels!
At the same time (this is important in my case), when we drove into Sudbury from Lively, where we lived, to do shopping, we would pass by the INCO switching yards. There I would see steamers, and I am sure I drove my parents nuts with my carrying on.
Two years later, we were at 14,000 feet in the Peruvian Andes where steam still ruled. And I saw them just across the road and down a 50' embankment from our house. The wheel shop was there. 
I became fascinated yet again when I understood the nature of flanged wheels and saw them being rolled into place.
Then school, a move back to Canada for more school and a career change for my father, growing up, college, marriage and Army life for 30 years followed. Both CN and CP occupied tracks on either side of the Thompson River where we lived for about 15 years. Many trains daily, but long-since diesels. I never paid much attention.
Then came retirement and I wondered what I would do for fun. Wife bought me a Model Railroader magazine for my Christmas stocking that first year, and I was in an LHS inside of 10 days. The rest is just a fast-paced dream. 
I think kids may benefit from playing with anything resembling a train early on, but they should be able to relate it to the prototype concurrently.
My 
-Crandell