On rare occasions, if the usual judicial process is incapable of bringing a suspect to justice, he will turn a blind eye and allow "street justice" to run its course.
Another of Gibbs' quirks is his tendency to stop the building's elevator between floors if he needs to have a short, urgent, private conversation with someone.