Walking away from the Omelas

Why do the ones who walk away from the Omelas walk away? The ones who walk away are obviously disturbed by the child in the basement.  One reason they may walk away is because the do not approve of treating the child this way, and it is easier to walk away than to try and change what the Omelas are doing. Therefore they take the easy way out and just get away from it all. This makes them sound like terrible people but that is not my point. In one way they are taking a stand against the actions by making the choice to leave, and they are certainly doing more than the people who see the wrong doing and just continue living their lives without any change at all.