How to remove the div surrounding the view block?
Enable Twig debugging as per this https://www.drupal.org/docs/8/theming/twig/debugging-twig-templates 
Then, see which template file is currently outputting your block view  
Create a new twig template file in your theme with the same name
Copy the content of the original file
Edit accordingly