Just giving up will only serve to make your memory worse. Like anything else you can improve your memory with practice and exercise.
There's a few free exercises at
The Memory Gym.
You may also wish to try hypnotherapy. With hypnotherapy you focus your mind on improving a faculty such as memory and it also heightens your visualization skills which greatly helps your power recall.