Nithya
I'm sorry that some people are rude and abrupt on the phone when you can't understand them immediately. For what it's worth, I think it's only natural for anyone in a foreign country to have to get used to understanding (and being understood), since there are so many aspects of language that differ from place to place, like accent, vocabulary, tone, etc.
It is very good that you are persevering nonetheless, and I don't think you necessarily need ESL lessons. Like Confused211 said, practice and time should solve your problems. Speaking on the phone can be especially difficult, since so many of the cues we use in face-to-face interactions, like lip-reading, body language, and facial expressions, are absent.
One thing you can try is try to repeat to the person what you THINK they said/want - if you get it wrong, hopefully they'll repeat their request. This will buy you some time as you try to process what they have said. Also, try to use humour or a light-hearted attitude to diffuse tension when you sense the conversation is not going so well. Sometimes, people respond better on the phone to a friendly voice.
Watching sub-titled movies is another great way to get a grasp of accents. Watch American movies with the English sub-titles on (sometimes called English for the Hearing-Impaired in the menu). When you hear dialogue, and read it at the same time, you will automatically become accustomed to the accent, since you will start to make connections between words as they are said in the new accent, and words as you know and can read them.
Some people have a great ear for accents, and some people really struggle. So, you may never become perfect at understanding all the different accents you hear in your volunteer job, but I think that's normal. Here are a few links to articles that may help you:
ESL Advices Blog: Best Way To Improve Your English: Using Voice Of America Podcasts to Improve Listening and Speaking Ability Audio for ESL/EFL: VOA's Special English Programs in a Podcast RSS Feed speech accent archive: browse
This is a good resource for you - click on "language/speakers" to browse recordings of different accents.
Irish tops 'World's Sexiest Accents' poll - Express India
The last link is a bit of fun - apparently the Irish are considered to have the world's sexiest accent!
Nithya, I hope this helps you in some way. Keep practising, and be confident in your own abilities. I think you are doing a wonderful thing by volunteering, and even though it may be hard sometimes, you are courageous to keep at it. Good luck!