It seems like it's something about the actual handling of the file that consumes two much memory (with the memory settings you've listed there should be no problem uploading the file).
Can you describe what you are foing with the file? How are you checking for duplicates?
In any case, I suggest the following:
* Create a small application that just uploads a file (and ignores it). Check that you the upload goes through.
* Gradually add more of the required behavior until you hit the OutOfMemory error. This will hopefully provide a hint for the cause of the problem.
For best results, use the Firefox browser..