ANDREW S KIRKOn Jan. 11 an FAA computer outage had grounded over 11,000 flights. Between last Thursday and Sunday, back office server issues with Amtrak I-ETMS (PTC) resulted in over 50 Amtrak train cancelations out of Chicago alone along with numerous spot cancelations. The FRA had a preliminary report the next day - we haven't seen anything from the FRA or Amtrak even offering to investigate. At least 15,000 and perhaps over 20,000 passengers suddenly had to find alternate means of reaching their destinations. It appears that neither Amtrak nor the company that offers the server service had backup facilities. Continuous tranportation can be provided if the carrier is willing to accommodate some redundancy in its facilities. Recently when CSX had a situation that affected the computers used by train dispatchers in the systemwide facility in Jacksonville, the dispatchers were moved into a back-up facilitiy until the main office was up and running. When I worked for Conrail, their busy Selkirk Yard kept running without a hitch after an October heavy/wet snow felled so many branches that the greater Albany, NY area lost commerial power for several days. Why? Because Selkirk had backup generators. I firmy believe that everyone who had a ticket who was denied transportation during this period needs to contact his/her elected officials. And
When traffic control systems malfunction or crash, operations cannot continue safely - it makes little difference if the mode of transportation is air, marine or railroad - all of which ARE NOT line of sight operations when operated at their customary maximum speeds.
I have been an airline passenger inroute and ON my connecting flight when the ATC lost all flight plans that the carriers had filed for flights to leave Atlanta. Instead of a before Noon arrival at destination it was a after 8 PM arrival. Doo Doo Happens.
In the early 2000's the CSX Mainframe computer system contracted a 'virus' or some other bug that 'bussied up' the comm lines between the Mainframe and the CADS for the dispatching system to the point the CADS would crash everytime it was rebooted and brought back on line. Problem started about 0100 and by 0600 CSX was shut down - Dispatchers could not safely identify or move trains. About Noon the busied comm lines was diagnosed as the problem, but the cause of that was still not diagnosed. The comm lines were 'cut' and CADS was restored to operation about 1300. With CADS being 'UP' the consist and crew information that was supplied from the Mainframe had to be sourced and entered manually by the Train Dispatchers. The Train Bulletins for each crew going On Duty could not be sent over printers as that information had to pass from the CADS to the mainframe to go to the printers in the field, thus all Train Bulletins had to be 'hand faxed' to the appropriate locations - account CSX Rules, upon receipt hand faxed bulletins had to be 'read back' to confirm proper receipt of all data. All available Extra Train Dispatchers were called in to work the read backs. After 48 hours the real cause was determined and fixed and operations returned to normal.
I recently heard of a maritime collision in a port area because the ports traffic control system had a failure.
Traffic Control Systems in commercial transportation are not toys.
Never too old to have a happy childhood!
"We are currently experiencing a communication issue impacting the nationwide network managed by a third party. This is causing train cancellations and delays in some areas," Amtrak said in a statement. "Some trains may also be running at reduced speeds. Updates will be provided as available."
Didn't we have a thread about out-sourcing?
ANDREW S KIRK I firmy believe that everyone who had a ticket who was denied transportation during this period needs to contact his/her elected officials. And
What's that going to do? Really?
It's because of the law that the trains couldn't deaprt without having PTC initiated. They helped caused this mess.
It's been fun. But it isn't much fun anymore. Signing off for now.
The opinions expressed here represent my own and not those of my employer, any other railroad, company, or person.t fun any
On Jan. 11 an FAA computer outage had grounded over 11,000 flights. Between last Thursday and Sunday, back office server issues with Amtrak I-ETMS (PTC) resulted in over 50 Amtrak train cancelations out of Chicago alone along with numerous spot cancelations. The FRA had a preliminary report the next day - we haven't seen anything from the FRA or Amtrak even offering to investigate. At least 15,000 and perhaps over 20,000 passengers suddenly had to find alternate means of reaching their destinations. It appears that neither Amtrak nor the company that offers the server service had backup facilities. Continuous tranportation can be provided if the carrier is willing to accommodate some redundancy in its facilities. Recently when CSX had a situation that affected the computers used by train dispatchers in the systemwide facility in Jacksonville, the dispatchers were moved into a back-up facilitiy until the main office was up and running. When I worked for Conrail, their busy Selkirk Yard kept running without a hitch after an October heavy/wet snow felled so many branches that the greater Albany, NY area lost commerial power for several days. Why? Because Selkirk had backup generators. I firmy believe that everyone who had a ticket who was denied transportation during this period needs to contact his/her elected officials. And
BaltACD PJS1 BaltACD Why would Amtrak not use the servers of the Freight Carriers who are actually operating their trains? Why do we have Amtrak? Creating it was a mistake. If passenger trains are critical for the well being of the United States, the government should have subsidized those trains that were being operated by the investor owned railroads. What have is another Washington, D.C. bureaucracy with more than 20,000 employees guided by executives and managers that seem to have little idea what they are doing. Politics of half a century ago and today are different - just as life from half a century ago is different from today. Remember, when Amtrak was created it was expected to be dead in five years.
PJS1 BaltACD Why would Amtrak not use the servers of the Freight Carriers who are actually operating their trains? Why do we have Amtrak? Creating it was a mistake. If passenger trains are critical for the well being of the United States, the government should have subsidized those trains that were being operated by the investor owned railroads. What have is another Washington, D.C. bureaucracy with more than 20,000 employees guided by executives and managers that seem to have little idea what they are doing.
BaltACD Why would Amtrak not use the servers of the Freight Carriers who are actually operating their trains?
Politics of half a century ago and today are different - just as life from half a century ago is different from today.
Remember, when Amtrak was created it was expected to be dead in five years.
30 and 59 On time out of CHI. However the planned late departure for 59 of 18:35 was met but appears that it has stopped at the service yard for over 90 minutes so far. Now only 48 to go.
zugmannIs it even possible to use another road's servers?
And when we are saying 'Servers' what data is actually involved?
When I was still working (Pre-PTC) on CSX we had continual issues getting timely and accurate train data from Amtrak - appropriate data was Train ID, Engine(s) number of cars, Conductor & Engineer names and their ID numbers as well as the On Duty times that apply to each as well as their HOS time. Pre-PTC we could inquire of the crew and apply the information manually. What additional data is now required with the implementation of PTC, I don't know.
How how long will it take to get everything straightened out? Note: Nothing on any Amtrak system publications had notice saying that boh Auto Train cancellations happened SAT. Only checking reservation's Status check showed them cancelled. How were the riders notified? What the heck?
Is it even possible to use another road's servers?
As of 1800 EDT 3, 5, 7, 21 All out CHI on time. s well 339, 393 also on time
Eagle #21 & CZ #5 got out of CHI on time. 335 HIA also out both after 1300 CDT. Maybe problem fixed? However, with all the equipment out of position probably 6 days to get all midwest LD trains running normally.? Amtrak managemet where are you ??
Rio Grande Valley, CFI,CFII
blue streak 1Yes for emergency situations. Depending on how much the freight RRs are affecred the class 1s will get new excuses for delivery delays.
I don't know if this would qualify as an "emergency".
My question is - Who's Servers were down the prevented the initial testing and linking of PTC? Amtrak? Freight Carriers?
Why would Amtrak not use the servers of the Freight Carriers who are actually operating their trains?
The problem as I understand it - PTC is required to be linked and operative at a train's origin. Once underway, PTC can fail and be 'cut out' that will allow a train to continue to its destination.
zugmann blue streak 1 FRA needs to change PTC initiation protocols. So you want them to now nullify the billions of dollars of technology that was mandated ?
blue streak 1 FRA needs to change PTC initiation protocols.
So you want them to now nullify the billions of dollars of technology that was mandated ?
Yes for emergency situations. Depending on how much the freight RRs are affecred the class 1s will get new excuses for delivery delays.
Balt you are correct both Auto trains cancelled on Saturday. That is an almost all CSX operation except Sun Rail and Amtrak going north from SFA. I had just looked at CHI but we have 1, 2, 22 also cancelled. Have no idea if revelant but our CSX track only had one northbound all day and it was slow.
blue streak 1Now official 30, 48, 50, 59 cancelled. That makes it 100% LD trains cancelled out of CHI for today Satursay. Is Gardner & Harris happy ?. FRA needs to change PTC initiation protocols. That brings up the question . Where is the FRA and Sec of Transportation??
That brings up the question . Where is the FRA and Sec of Transportation??
Bull Hockey! You overlooked AutoTrain in both directions.
Now official 30, 48, 50, 59 cancelled. That makes it 100% LD trains cancelled out of CHI for today Satursay. Is Gardner & Harris happy ?. FRA needs to change PTC initiation protocols.
All Hias cancelled today. But Amtrak shines for one item. ACSES is working without a hick up. All NEC trains running OK with slowing NYP <> NRO work. beginning to wonder if system hack? As of 1800 EDT 3, 4, 5, 21 all ready cancelled out of CHI,
EDIT: 4 has been restored/
Amtrak Advisory | Server Issues Causing Delays
Due to a rail-industry control system that continues to experience intermittent communications issues, some Amtrak trains to and from Chicago have been canceled through late afternoon. We're working around-the-clock with our partners to continue troubleshooting and addressing these issues.
Affected customers have been notified and we are working to resolve this as soon as possible. Updates will be provided as available.
You may change your reservation via the Amtrak app or Amtrak.com. If your trip has been canceled, cancellation fees will be waived. Follow @AmtrakAlerts on Twitter for real-time service information and updates.
blue streak 1It is too early to know eactly how all this started and if freight RRs have had any problems. If this is an Amtrak problem some ITs need to be ????. Here is link to ongoing problems.
Has affected frieght RRs, too. Sounds like the I-ETMS system was giving fits.
I don't even know how much of that is Amtrak, vs WABTEC vs. Rockwell Collins or whomever is involved.
As of 1200 EDT almost all CHI regionals in and out cancelled. #s 1 & 22 cancelled at NOL & SAS. What LD at CHI are going to be cancelled this afternoon?
I'm not sure if this is related but my morning ride to work (Metra Southwest) was stopped at South Branch Bridge for about 5-10 minutes shortly after 7:00AM. The crew announced that it was due to PTC problems.
For some reason Amtrakhas been unable to connect to the PTC server for many trains especially CHI. This started early AM Friday and is continuing as of 0645 Saturday morning. Some long distance originators yesterday cancelled = 2, 3, 7, 21. All other LD departures from CHI in service distuption but running except 59 normal. West coast and east coast OK but almost all took origination delays. Almost all regionals out of CHI cancelled and 171 out of ROA delayed.
It is too early to know eactly how all this started and if freight RRs have had any problems. If this is an Amtrak problem some ITs need to be ????. Here is link to ongoing problems.
Amtrak Alerts (@AmtrakAlerts) / Twitter
Our community is FREE to join. To participate you must either login or register for an account.