Keep in mind my example was a local train on a branchline with (in the early sixties) an FM or Baldwin diesel, 5-6 cars, and a caboose, trundling along at around 10 MPH. Not a lot of walking for the brakeman at the end of the branch.
I will see if I can find the source of what I read about Flagmen (I think it was re the New York Central?), but my understanding was that some railroads used the term flagman, some 'second brakeman'. In any case, the source I read said of the three crewmen in the caboose (if there were three), the conductor would of course have the most seniority (being a promoted brakeman), the brakeman second, and the 'second brakeman' or flagman would be the least senior...often a newer employee 'learning the ropes' from the brakeman and conductor. If the train stopped unexpectedly and a flagman had to go out a half-mile or so back of the train (possibly in rain or snow) to ensure the rear was protected, it was a job they were going to make the least senior person do.