Database of mineral with images for a neural network application
First of all rruff.info is an excelent site done by chemists to take data quickly, and you can totally trust the source.

Second, for images, most of the listed +5000 species are not shown in nature, and you cannot have pictures of all of them. You will only find on the net pictures from common minerals on nature.

Third it depends if your work is gonna be published or not, as you would need permission or CC licensed images if so.

Copyrigthed database of pictures and data:


Webmineral
Mindat.org


Mindat can share with you their data, and they are going soon to open them on CC format with an API, but the pictures they have belong to each author and many of them are copyrigthed (and gonna stay copyrigthed if they open data, you would need to ask permission individually for each picture).

CC license pictures:

I wondered how to obtain pictures from common minerals on CC license because I am writing a mineral app and my choise was Dr Lavinsky collection uploaded to wikipedia from his mineral shop iRocks.com. 

At his website they have plentifull of high quality well described pictures of common minerals you can maybe download with a script making GET petitions, and you can ask them for permission, even for the pictures that are not uploaded to the wiki. I did it myself for my mineral guide and they just ask me to quote iRocks.com; they are very fair people. Aparently they are happy to share if you quote his shop.

At Wikipedia you can find on CC license >2000 pictures uploaded from mindat.org, and >5000 uploaded from iRocks.com

You have the complete list of minerals with picture on Wikimedia Commons where to start to collect the data. I don't know if you could somehow extract all the images url using GET petitions with code. Myself I only took the iRocks pictures because my guide covered only 180 minerals, the descriptions Dr Lavinsky provides are valuable (example), and then I only needed to quote one author (I ended by taking them directly from iRocks and link on my work to each piece on his shop, they got very happy about that, they answered my mail with "many thanks").