Library item field doesn't show up the remove button
There is no dedicated delete button. Drupals default behavior (including paragraphs) is to automatically delete every empty field at the end of the list. The empty field you see is just a convinience for faster adding of a new item.

Side node: To delete e.g. the first item of the list, remove the content and drag it to the bottom.