Why Doesn't endObject work?
The process of ending an object is different to the process of creating it. Typically you delete it using a reference to the object, using the function piece.endObject()

Modifying your above code:

if key[bge.events.PAD5] == pressed:              # If 5 key pressed
    target = scene.objects['Empty1']             # Find where it goes
    new_piece = scene.addObject(shape)           # Create the piece..
    new_piece.worldPosition = target.worldPosition  # and place it.

if key[bge.events.PAD0] == pressed:
    # First up we retrieve the existing object. In this case we get it by name.
    # Make sure the name is the name of the object you want to remove
    existing_object = scene.objects.get(shape)
    # If it exists, then remove it
    if existing_object is not None:
        existing_object.endObject()