This is by design.  The thumbnails need to be available for reports and are not deleted.  When you delete an image the fullsize image is deleted in all three versions (if they exist).  The grayscale and sepia thumbnails are also deleted, although they will be rebuilt if needed for a report.  The only file left is the color thumbnail which should be fairly small depending on your image options.