Ok, I have been able to figure out these are ghost images of sorts.
So instead of being copies, as they don't exist anywhere in the folder, if I access the folder from my computer, so I tried something.
I deleted one picture off the phone (I must have managed to click on the original picture, otherwise it would have frozen on a "deleting screen" until I shut the phone off). Now when I rebooted the phone, instead of having 5 images of that one picture that I deleted. I have 4 images but all the images are broken and display no picture at all.
No clue how to solve this, I'm hoping to avoid hard resetting the phone.
I'm sorry you're having problems with pictures being duplicated on your device. Especially since the pictures don't even exist and cannot be deleted.
The first thing I'd like to see is if the problem is with the memory card. To test this, remove the memory card from your device and access your Gallery. Are the duplicate pictures still there? If not, most likely the memory card has been corrupted and should be re-formatted. You'll need to backup all the information on the memory card to the computer then follow the steps in How to format the SD card for Android.
If the issue is not with the media card, the next thing I'd investigate is a software glitch. The first thing we can do is clear out the data stored in the Gallery program. Use the Clear cache and clear data in any application - Android document and follow the steps for Android 2.2 / 2.3 / 3.0.
If that doesn't fix the issue, then the next step would be a master reset and testing without any applications installed from the Market.
Let me know if this helps!