In this age of increasing incivility, I just want to remind people that speaking up when you hear something you feel is demeaning, bigoted, or inappropriate can make a difference. A personal story: when I was a grad student, I worked with a professor who sometimes told racist jokes in social situations with his research group. This is something I don't particularly enjoy, and I was sure it was making the non-white students uncomfortable. Even though I wanted to keep working with this person, and even though I wanted this person to write me letters in the future, and even though this person was on my committee, after a little while, I decided that the next time he made one of these little jokes, I would say something.
I was very nervous about it--I had no idea how it would go, though this person is generally reasonable. I practiced to myself what I would say, and sure enough at lunch one day, I had my chance. Shortly after he had just told a racist joke, I was alone with him. I told him that I don't like those kinds of jokes, and that I would prefer if he didn't tell them when I was around. After that, he never told one in my presence (I have no idea what he did when I wasn't around). I think he didn't realize that such jokes can make people feel uncomfortable, since he didn't "mean" it. That said, it can be horrible to work in an environment where people (like your PI!) routinely make disparaging remarks about your culture or background. It is worth saying something to improve things, or even to make sure things don't get worse.
1. It is much, much easier to say something when the remarks/behaviors are about someone else. I am not sure how this would have played out if it was sexist humor in this situation, but I have definitely been told that I have no sense of humor and/or need to lighten up when pointing out troubling sexism to peers. Saying something about jokes that target you can also get you labeled as a troublemaker or complainer. So help out your colleagues--say something when they are the butt of the "joke", and hopefully someone will have your back as well.
2. I kept my comments about myself "I don't like" and about the jokes "that kind of humor" rather than calling the professor a racist, or implying that he was being cruel on purpose. I think people are less likely to feel defensive with this approach, and it is more likely to get results. As far as I could tell, the professor treated everyone fairly. It was just the "jokes".
3. Not every person is reasonable. This is not an option for everyone, but if it is, I recommend doing your bit for the social atmosphere. I certainly enjoyed working with him a lot more without having to listen to racist humor, but I was also fortunate, and this incident could have ruined an opportunity for me. I suppose in the worst case, it could have cost me my PhD. If a negative outcome will have very adverse affects, make sure you think things through. I am also talking about the occasional inappropriate remark or joke, not actual harassment, which is a much more serious problem (and likely will not get better with this approach).