If God is all powerful, why can he not rid of all evil?

If he is all powerful, then evil shouldn't exist. It should be in his power to get rid of evil. Does he choose not to get rid of evil, making himself evil? Or can he just not get rid of evil at all, making him not actually all powerful?