Why you should do that? Humm, there are arguments why yo should get rid off or keep it, but it's a matter of personal choice, mainly. The methods to removing the meta name generator tag for Joomla varies between versions. I discussed couple of ways to do it in an earlier tip. Here is a new one, working in Joomla 2.5!
On Joomla 2.5 you don't necessarily need a plugin or a component to do this: you can simply add this code to your index.php file in your template.
JFactory::getDocument()->setGenerator('');
Put it just after the line
//no direct access
This method will vary greatly on what template you are using and from what company, so I cannot get too detailed. For example YooThemes has a different template structure then RocketThemes. You will have to noodle around a bit yourself in order to do this. The main advantage of editing the template code instead of installing a plugin is less overhead. The fewer plugins you have, the faster your site will be, and queries add up fast.