Servomotor AND Button
Short answer: You have a blocking delay interrupting your button debounce time of 100ms.

Your button library has a debounce time of 100ms. This means you must be holding the button down for more that 100ms before it will register as pressed. 

You have a 3 second blocking delay at the end of your loop. When you call delay(XX), your processor cannot do anything during this time (like check for a button), it's just waiting. 

Think about how the code is running. In your main loop, you are calling the function button.loop(), and it's checking to see if your button is pressed. If it's pressed, it's starts a timer (your debounce). The goal of this timer is for another 100ms to pass by with your button still being pressed to make sure you actually pressed it. Button.isPressed() fails, and it moves to its next set of instructions, which is move the servo and wait 3 seconds, during which time it cannot check the button. Then it starts all over again, and it calls button.loop() again. This all happens MUCH faster than 100ms. From the start of void loop() to the time it takes for it to wind back up at the 3 second blocking delay is in the nano seconds. This never gives your button library the 100ms of "pressed down" button it needs for button.isPressed() to return true. 

I suggest spending some time looking at the behind the scenes of your button library and see how the debounce works. You can do that, or you can use the example debounce sketch off the main arduino website and learn how that works. The next thing you'd want to do is to learn how to make your code "wait" without using the blocking call delay(). This is done by using timers with millis(). A good way to learn this is the example arduino sketch "Blink Without Delay".

Hope that clears some things up.