I think most of us adapt our styles and preferences to our scales, rather than start by saying "here is the sort of person I am, these are my strengths and preferences, and here is my completed benchwork ready to go: now, what scale is best for me?" Maybe that is the logical way we should approacy this, but not many of us do. Indeed how many of us are in the scale we are in mostly because that was the scale of some trainset we got as a gift?
So not many of us go about this in the order that you have chosen. But I think one question would be -- what kind of railfanning do you like best? The answer to that might dictate scale.
Both N and HO are well served in the way of track, rolling stock and structures, and both offer plenty of opportunities for almost any variety of model railroader, although i have to admit that operating sessions on N scale layouts where you are expected to read off car numbers is a real chore given my eyesight. But there are pretty sophisticated and realistic operating methods that do not rely so heavily on car numbers. And some guys don't use any operating system at all.
I also have to concede that rerailing N scale rolling stock remains a challenge for me as well. But if ease of rerailing was the main concern heck I'd go back to Lionel (and given what is available in three rail O these days I would probably have a heck of a good time with it, assuming I had the funds).
And yes I am well aware that the prime directive should be to not derail the durn things to begin with.
When I sit at the workbench assembling a kit or scratchbuilding a structure I am glad I am in HO and not in N. I am in awe of what modelers such as Jerry Gunderson or Keith Kohlmann can do in N scale in the way of super detailed rolling stock and structures. I mean, just look at this N scale modeling (which I have seen in person):
The size and bulk of HO fits my current abilities (or lack of same -- and they aren't getting any better as I age). Yet, as I try to visualize a large industrial area on the layout, with its own rail service, I wish I had chosen N because I hate making compromises.
Being a midwesterner most railfanning is of fairly contained views of a train coming down the track and to capture that on a model railroad, HO works well because the eye readily takes in on the model the same basic scene that it takes in trackside. If I was modeling more wide open spaces -- such as the west, or the flatter granger areas, or the Rockies or eastern mountains, where railfan photographers tend to back off the scene and capture more of the context, I really think I'd find the scenic advantages of N to be pretty powerful.
I think Mike Danneman's layout is the best advertisement ever for N scale. The interesting thing is that he has used his space wisely and resisted the temptation to cram it full of expensive trains and track -- the actual amount of track and turnouts and rolling stock (the stuff that really costs money in other words) is about the same as a much smaller but traditional layout.