Here is link of FRA research on ATO. Note that a lot of different cameras were tested.
Automated Train Operations (ATO) Safety and Sensor Development (dot.gov)
I didn't get very far into this article until I was outraged right in the first paragraph where they began to discuss SP (sensory platform).
To mean, I will always equate "SP" with the late, great Southern Pacific.
May she rest in peace! Gone but not forgotten.
Regards,
FMC
blue streak 1 Here is link of FRA research on ATO. Note that a lot of different cameras were tested. Automated Train Operations (ATO) Safety and Sensor Development (dot.gov)
Thanks for a more definitive post link. I hope all interested in this topic will read carefully and with an open mind.
The article seems to suggest that several different sensors will need to be employed to cover a range of possible conditions. Layering the images together to provide a composite will require a pretty powerful computer, as well as calibration of the sensors so they see the same thing in the same place.
I would imagine a dedicated pod of sensors would be involved. Front of the loco? On the roof?
That said, there's no reason such integration can be done.
The challenge will be identifying and programming the actionable objects of interest so appropriate actions can be taken when they are detected.
That would include errant autos and herds of pronghorns.
Larry Resident Microferroequinologist (at least at my house) Everyone goes home; Safety begins with you My Opinion. Standard Disclaimers Apply. No Expiration Date Come ride the rails with me! There's one thing about humility - the moment you think you've got it, you've lost it...
First thing that came to mind was redundancy in sensors. For example, having two or more separate forward facing cameras, so a bug splat on one camera doesn't blind the computer. Multiple detection modalities should be mandatory, mm wave radar data is a lot easier to process to get a 3D image of what's up ahead. Integration with lineside sensors would help, especially in populated areas.
Note that this only barely scratches the surface -- it is limited to the raw sensor modalities. The real 'interest' in this space is in the discipline we called 'sensor fusion' -- this has taken a leaf from successful user-interface technology to include a relatively enormous number of parallel devices and sources for the desired redundancy.
I assume that many of the "SPs" being developed for autonomous road vehicles will be quickly costed-down for production and the 'technology', if not indeed available as both modules and cores OTS, will then be adaptable to the special conditions necessary for railroad service.
I scanned through the PDF, but did not see much explaining why various sensors are needed to see such a large area. Trains can’t stop for emergencies anyway. Even attempting to stop is too risky for the train. I did notice the point that sensors will help prevent trains from running over herds of animals. I think the railroads will just want to use what Rio Tinto uses to watch the track ahead.
What I would like to see is a full diagram of what will be included in the sensor “platform” and what it will cost to equip one locomotive with it.
Why the Need for Such Elaborate Sensing Ability?
With self-driving trains, there is no need for the train to “see” where it is going. That need is for self-driving road vehicles because they need to see in order to steer. On the contrary, trains do not need the aid of human or artificial intelligence in order to steer because they are self-guiding. Also, unlike road vehicles, trains do not need to see in order to avoid collisions because the collision protection is already provided PTC.
So, why the emphasis on ultra-high ability for automatic trains to sense and assimilate all elements of the unfolding scene ahead when it is not necessary? Where it is necessary is with self-driving road vehicles that are not self-guiding and do have to detect and react to other vehicles. This need by self-driving road vehicles to see and assimilate everything in their view is their biggest technological challenge, and may never be adequately resolved to permit them to be practical. With road vehicles, drivers must be ready to take many different evasive actions to avoid collisions and keep the vehicle on the road.
So why is the FRA (in the link of the OP) advocating the elaborate platform array carrying 8 different modes of detection sensors? Why, for instance, does the automatic train need to know whether it is foggy, snowing, or if animals are on the right of way? It seems like massive overkill to address a problem that does not exist for trains, but does exist for road vehicles.
I believe the explanation for wanting this intense forward vision for driverless trains is this:
It enables industry proponents to argue that automatic crewless trains will be FAR safer than trains relying on humans watching the track ahead. It is the same argument used by proponents of self-driving cars and trucks, but at least they need the most perfect detection possible in order to keep the vehicle on the road and responding to signs, signals, and other warnings.
So, even though this ultra-high detection is not technically necessary for driverless trains, this move to ultra-detection will shut down the resistance of the unions who say that crews are needed on trains for public safety, and removing them with automation will endanger the public.
That augment is already weak because the engineer is relatively powerless to prevent collisions once they are observed developing. Trains cannot stop in time, and even attempting to slow down is overruled by the danger it imposes on the train.
I also think it is possible that the FRA simply does not understand that self-driving trains do not need to thoroughly sense the view and react to it. Perhaps they are just assuming that issue because it is the natural and justified mission in the development of self-driving road vehicles.
I wonder, when trains become driver-less, if railfans will want to photograph them as much. It's true that in a lot of photos (including good ones) currently one can't actually see any crew. However, you know that they're there!
I had never thought about this until this moment; but for whatever reason, I don't think I'd shoot trains as much if they were robots.
Euclid: Agree in part but would point out two things. 1. Hazards such as stalled autos or trucks on crossings occur and stopping or at least slowing can help avoicld disastrous derailments. 2. Automated freight trains will probably be much shorter** so stopping distance is reduced.
** Shorter trains on tighter headways should mean better service to more end customers.
The short answer for elaborate sensor fusion and interlocking AI/ES is, in a word, lawyers. There are a raft of additional reasons, but that one alone is sufficient.
That the railroad applications will benefit from costed-down or OTS 'vehicle technology' should go without saying, but there, I've said it again. Many enabling technologies that would be ridiculously expensive and brittle if developed, say, like NAJPTC could be remarkably good if systems-integrated from areas where 'early adoption' is subsidized in some way.
Lithonia OperatorI had never thought about this until this moment; but for whatever reason, I don't think I'd shoot trains as much if they were robots.
tree68 Lithonia Operator I had never thought about this until this moment; but for whatever reason, I don't think I'd shoot trains as much if they were robots. At first, it will be the novelty. After that, I have my doubts that the railroads will go to any special lengths (ie, tribute locos, etc) to attract the fans.
Lithonia Operator I had never thought about this until this moment; but for whatever reason, I don't think I'd shoot trains as much if they were robots.
At first, it will be the novelty. After that, I have my doubts that the railroads will go to any special lengths (ie, tribute locos, etc) to attract the fans.
You may be right. I don't remember his name, but a rail writer back in the late 50's said something to the effect of "When all the steam locomotives are gone trains will be no more interesting than a conveyor belt."
Obviously that didn't happen, but he may have been just a bit premature. Robot trains? Can't say that idea excites me too much.
The only positive thing I can think of in favor of robot trains is there won't be any more head-end crew traumas due to tresspasser strikes. A robot won't care.
OvermodThe short answer for elaborate sensor fusion and interlocking AI/ES is, in a word, lawyers.
For that to worry the railroads, they would have to believe that lawyers can make the case that an engineer could have stopped a train in time to avoid a collision whereas the automatic system is incapable of that. It would easy for the defense to prove that neither an engineer nor the automatic system could stop a train in time to avoid a collision in many cases.
I am not saying that the automatic system does not need any scanning detection for danger ahead. I am saying that what is being proposed by the FRA is sensor overkill.
The main issue, as Charlie mentioned, is spotting stalled vehicles on grade crossings. That is the one contingency that needs the intelligence and quick reaction, no matter whether artificial or of a human engineer.
I recall reading that Rio Tinto automatic trains are equipped with a grade crossing sensor to spot obstructions on the crossing and apply braking if necessary. So that would also be needed for U.S. automatic trains.
But instead, the FRA is proposing a sensor “platform” featuring the following equipment:
VISUAL CAMERAS
THERMAL CAMERAS
INFRARED CAMERAS
SPECTRAL CAMERAS
LIGHT DETECTION AND RANGING
TIME OF FLIGHT CAMERAS
RADIO DETECTION AND RANGING
TEMPERATURE SENSORS
I wonder if this sensing platform might cost more than the locomotive. Originally, I speculated that this overkill may be a means of promoting automatic trains as being far safer than trains with human operators. Mr. Creel did promote automatic trains a few months ago on the basis of their safety advantage over trains with human onboard operators. So that would fit with this extravagant detection system watching the track ahead. Jumping on this supposed safety advantage certainly mimics the marketing of automatic cars and trucks. But then those vehicles are far more subject to human error that trains. So it is easy to claim that Artificial Intelligence can outperform human intelligence behind driving cars and trucks on the roadways.
But now, the more I think about it, the railroads will have to pay for this extravagant sensor platform installed on every locomotive. I think they would, like me, see the sensor platform as being overkill and too costly. So maybe instead of this sensor platform being promoted by the railroads to thwart the union opposition to automatic trains, it is the other way around.
After all, it is the unions who are saying that automatic trains will be too dangerous compared to manned trains. So maybe this extravagant sensor array is an idea being promoted by the unions as a necessity to make automatic trains viable. Then if that became the accepted premise, the railroads would back off on their push to automate the trains because the hardware is too expensive compared to the savings from reducing crews on trains through automation.
I got the impression that they were testing a variety of sensors to see which best suited the need. Not that they intended for all of them to be used.
Several of the sensors are similar in function.
From the FRA report linked in the original post:
“To satisfy SP [sensor platform] requirements, TTCI concluded that a suite of various sensor types, working in unison, will be necessary.”
“Field testing must also explore the use of multiple types of sensors and the fusion of data from those sensors to enhance SP functionality beyond what can be done with a single sensor type.”
While they do not say how many sensor types they will use, or how many of each type, the thrust of the document is the assumption that sensing must be able to thoroughly see and assimilate as much detail of the forward field as possible. The obvious reason for this assumption is that the content of the forward field will need to be thoroughly assimilated by the automatic operating system in order for it to drive the train.
In other words, it is the same as the human engineer sitting in the seat and watching every detail ahead and factoring them as needed to make the decisions about operating the manual controls. Therefore the so-called, Sensor Platform is providing the same sense data for the automatic operator as an engineer’s eyes provide to the human engineer operator.
That is the way it is for road vehicles where there are many decisions to be made by a driver personally interacting with the decisions of other drivers. That is a non-stop process for drivers of road vehicles. That process also includes constantly steering to safely follow the roadway. To automate that task of driver perception does indeed require the most exhaustively thorough set of sensors possible. And even with that, there is still a risk that something outside of the sensor capability will occur and cause death.
Automatic trains do not need a stream of sensed information to guide them like automatic road vehicles need for steering. Automatic trains also do not need sensed information it to tell them when to accelerate and decelerate for speed limits, where to blow the horn for grade crossings, when to apply and release braking, etc. All of that is in the computer program for the track route.
The one most critical train need for forward scrutiny is to watch grade crossings to assure the tracks are not blocked by stalled vehicles. This is a relatively easy detection task in plain view. Sensors for this detection could be on the locomotive, but also might be fixed in place near the crossing.
The overall point is that this requirement for infallible sensor ability for automated road vehicles does not exist for railroad trains. The FRA document seems to miss that point.
The visual sensing capabilities (IR, heat, UV and glare filter, telescopic) and reaction times would be far better than a human operator.
I find it a little sad that a fifth of the way into the 21st Century there are still people who think a train can be always, or even primarily, brought to a stop short of a vehicle stopped on a crossing by use of sensors only on that train's locomotive, filling in for the 'eagle eye' of an engine crewman.
Where sensor fusion and intelligent sensing is most valuable is in the modern version of restricted speed, where the higher net resolution and lower latency will vastly improve safety.
One concept I've been fond of is the idea of extended 'haptic space' -- for a train this would include determination and if necessary filling-in and confirmation from things like wayside cameras, scanners or loops, or from strategic orbiting or launched drone coverage. In this context any vehicle or other intrusion that is tracked by a 'third party' for enforcement can be relayed (ideally, autonomically with high reliability and low latency) to trains well in advance of their brake limits; even false positives result in not stopping, but the 'right' counterpart of restricted speed on approach.
EuclidThe one most critical train need for forward scrutiny is to watch grade crossings to assure the tracks are not blocked by stalled vehicles.
Can't forget rock slides, large trees, MOW vehicles, etc.
Sensors fixed along railroad corridor versus sensors on locomotives:
Declaring that autonomous running must include 100% of line advanced hazard detection places an unnecessary burden on the challenge of autonomous running.
I think slide protection will just continue as electro-mechanical slide fences linked to PTC. Generally, there should be a distinction between locomotive-borne sensors and fixed sensors along the track. Both make sense but their purpose differs. Fixed sensors could be maximized to include any possible track damage or obstruction such as broken rails, collapsed bridges, landslides, rock falls, washouts, and all forms of encroaching vehicles either on the rails or ground. And most importantly, they would give warning as soon as the obstruction occurred rather than when it comes into the line of sight from the locomotive. That would allow time to get the train stopped in many cases.
But again, I mention that there is a distinction between sensors detecting random dangers versus sensors to know the train location and operate it accordingly. The automation does indeed need the latter, including sensors and programing to replace the capability of the human engineer. But even with that function, sensors are mostly needed for reacting to grade crossings if there is time to stop or slow down enough to make a collision non-fatal, or possibly stop in time to avoid a collision. The rest of the engineer's function would be addressed with the program and PTC.
So all of this worry about wayside hazards does not need to be addressed by locomotive-borne, super sensors that can “see” beyond the engineer’s line of sight view. It is much more logical to address wayside hazards with fixed sensors along the track as another improvement outside of automation for autonomous train operation, and neither one requires the other. For one thing, placing all the wayside sensors on the locomotive means that each locomotive must be equipped with them.
The sensors needed only for autonomous running are only intended to replace the engineer, and not to do so under the assumption that the engineer has super human powers to see around curves. Perhaps this piling on of all these extra features in the name of unlimited safety is just a ploy to make automatic, crewless running into an unachievable goal in the interest of those who oppose it.
It is a little telling that automatic running should include these new features to do things that the crew was never able to do. We have been told endlessly how trains cannot be stopped in time for hazards ahead. We are even told that just trying to stop poses great dangers to the crews as well as the general public. We are told that engineers must choose between saving the day with an emergency application and the risk that the application itself will put the train in the ditch.
I am not a big fan of automated vehicles of any kind, whether it is trains, cars, trucks. I know planes have had auto pilot for years, long before there were computers, not sure how it works. But computers can fail or be hacked and I have seen TV shows where someone can take over your car and drive it wherever they want, cause a wreck and you as the driver are powerless to take it back. Have talked to a tech friend and he said it can happen with any kind of computer. Tells me to always keep my gas tank full, because if terrorists use EMP =electro magnetic pulse everything with a computer will fail. I have a classic 1966 Chevy that belonged to my Dad, so I have nothing under the hood for anyone to hack. Does not even have 50,000 miles on it, Dad always walked to work at Frisco railyard, Mom never learned to drive and I always took bus downtown to work. I only drive around my area, no highways any longer, they move way too fast for me, and lanes look narrow, I have no passenger side mirror was not an option, no power steering, brakes or a/c. But it is a good car for what I need. Get many offers to buy it when I am out, but I do not want a newer car. This is simple for mechanic to fix and me to drive. No bells and tweets telling me to put on seat belt, which is only a lap one anyway. And automation has done away with many jobs, I have nothing against computers just not under the hood of a vehicle or running a train.
SunnylandI am not a big fan of automated vehicles of any kind, whether it is trains, cars, trucks. I know planes have had auto pilot for years, long before there were computers, not sure how it works. But computers can fail or be hacked and I have seen TV shows where someone can take over your car and drive it wherever they want, cause a wreck and you as the driver are powerless to take it back. Have talked to a tech friend and he said it can happen with any kind of computer. Tells me to always keep my gas tank full, because if terrorists use EMP =electro magnetic pulse everything with a computer will fail. I have a classic 1966 Chevy that belonged to my Dad, so I have nothing under the hood for anyone to hack. Does not even have 50,000 miles on it, Dad always walked to work at Frisco railyard, Mom never learned to drive and I always took bus downtown to work. I only drive around my area, no highways any longer, they move way too fast for me, and lanes look narrow, I have no passenger side mirror was not an option, no power steering, brakes or a/c. But it is a good car for what I need. Get many offers to buy it when I am out, but I do not want a newer car. This is simple for mechanic to fix and me to drive. No bells and tweets telling me to put on seat belt, which is only a lap one anyway. And automation has done away with many jobs, I have nothing against computers just not under the hood of a vehicle or running a train.
Hot cars of the 60's had performance (in a straight line) in spades (with a cost to the atmosphere). They were easy for technicians of the era to keep running as their technology, when viewed from today, was very simple. Technologies were put on vehicles during that era that 'worked' but didn't have reliability - infact the total vehicles of the era did not have lasting reliability.
A technician raised on today's cars with EFI, individual coils per cylinder, and with myriad of sensors for everything you never even thought of being required for the successful operation of a vehicle all reporting to the 'computer' which manages every aspect of the combustion event on each cylinder of the engine as well as controling the workings of the transmission to most effectively put the power to the road. That technician would be lost with points and condenser ignitions of the 1960's and earlier.
Today's vehicles are very complex - example of diagnostics required to 'troubleshoot' a problem. https://www.youtube.com/watch?v=uu-L2PKr9LM (South Main Auto Channel - has numerous videos of numerous types of repairs on many different makes and models of vehicles)
Today's cars do last if given reasonable maintenance, my 1990 Jeep Cherokee lasted 300K miles and my 2003 Dodge Durango lasted 360K miles (drivetrain was still good - drivers side seatbelt wouldn't extend).
Never too old to have a happy childhood!
Sensors either fixed at, or on approach to grade crossings could detect vehicles stalled on a grade crossing prior to a train entering the crossing circuit. Prior to that circuit activation point, vehicles approaching the crossing are free to cross legally even if a train is within the driver’s range of vision.
This raises an interesting question: If a train is not yet in the crossing activation circuit, and the engineer observes a vehicle on the crossing that is stationary with no traffic standing ahead of it, he should realize that the vehicle is stalled on the crossing and quite possibly will not clear the crossing before the train gets to it. So what should the engineer do in such a case?
Do not begin any braking because the train is not yet in the crossing circuit and thus vehicles have a right to be fouling the crossing.
Initiate some braking based on the observation that a vehicle appears to be stalled on the crossing and may be unable to move even after the train enters the crossing circuit.
This dilemma is probably the most critical challenge to public safety that can be resolved by a train engineer to prevent injury or death. Yet, it may be unclear what the responsibility is for an engineer to begin braking before reaching the crossing circuit.
However with sensors spotting the impending emergency, an automatic driverless program can assimilate the problem and be programed to begin braking immediately rather than waiting for the train to enter the crossing circuit. Such a system would clarify the situation and the call for train response to it without worry about recriminations from the company for possibly delaying the train for a vehicle that clears before impact.
FRA Sensor testing
https://www.youtube.com/watch?v=fn3KWM1kuAw
That is scary and amazing at the same time..
Atlas, Spot & Co. -- what a good way to celebrate the new year!
I had not realized that Hyundai had bought Boston Dynamics. Interesting things should come of that.
I see no indication that Rio Tinto mining railroad uses a wide variety of different sensors to model every facet of the headway view of their trains. It may be that a lot of this senor frenzy is being pushed by companies that make sensors and look for new uses for them. Railroads are a perfect customer target for these sensor marketers. So for that motive, the sensor makers are jumping on the automation bandwagon by offering to cloak trains with sensors that can detect all that is.
Rio Tinto reportedly only uses forward headway cameras from which managers can watch anything of interest in front of the train. They also use fixed cameras at all grade crossings. That seems like all that is actually needed once you take away all of this sensor excitement.
EuclidI see no indication that Rio Tinto mining railroad uses a wide variety of different sensors to model every facet of the headway view of their trains.
Little more than the old QNS&L proximity radio system would give effective PTC for autonomous operation ... and you'd still need combinations of sensors for kink, emergent track breakage, and other various things outside LOS braking effectiveness.
I still consider a major driver to be actual 'ruggedizable' OTS cost, not B52-hammer economics, so more complex sensor suites and operating modalities appropriate for road vehicles might as well be considered,and their "excess-to-requirements" capabilities and potential used to advantage.
Overmod Euclid I see no indication that Rio Tinto mining railroad uses a wide variety of different sensors to model every facet of the headway view of their trains. Does it occur to you that operating private unit trains over a private railroad through completely undeveloped country similar to the surface of Mercury part of the year is a leetle bit different from most North American practice?
Euclid I see no indication that Rio Tinto mining railroad uses a wide variety of different sensors to model every facet of the headway view of their trains.
Does it occur to you that operating private unit trains over a private railroad through completely undeveloped country similar to the surface of Mercury part of the year is a leetle bit different from most North American practice?
Yes of course I realize that Rio Tinto operation differs from U.S. operations, but I don’t see that to mean that automatic running is impossible with the U.S. operations as is so desperately proclaimed by people with a strong interest in preventing it here. I have heard that argument 10,000 times.
I understand your point about using sensors to spot track defects, and I have carefully addressed that in several preceding posts here. My point is that the objective of spotting track defects in time to stop for them is completely different than the objective of using sensors to know where the train is during autonomous running in the way they are needed for self-driving road vehicles.
Rio Tinto is in wide open desolate country running unit trains that do no setout and pickup work requiring manual labor. I get that. But, even so, I do detect that U.S. railroads are considering autonomous freight trains despite the fact that some of their trains have to do work enroute. They will focus automation for trains that do not have to do work along the way.
But even though Rio Tinto is in wide open country, they do have the same issue with track defects that you describe. But autonomous operation does not require the ability of detecting those track defects, in the same way it is not required here for automatic running. It is a separate objective.
All automatic running needs is to do is replace the human engineer without any loss of safety. And that appears to be what Rio Tinto has done. They capture video of the track ahead and they have fixed cameras watching all grade crossings. I do not know whether those fixed cameras offer any ability to warn of a stalled vehicle, but that is the one critical need at grade crossings because there is often time to stop the train in stalled vehicle cases. If the grade crossing camera/sensor cannot do that, I see no point. If crash documentation is desired, it is already in the video taken from the locomotive.
Our community is FREE to join. To participate you must either login or register for an account.