Elitism is the idea that one is better than another because they have more money, resources, are more intelligent, have a better education and more opportunities than others. This may seem true in a short sighted way. However, elitism in higher education stifles its overall mission of helping people learn. When people are essentially locked out of opportunities because they were not born into wealthy families there is a level of dogmatic stunting in growth and depth that leads to inbred ideas that don't perform.
Think of how learning creates structures in our brain and how we understand the world around us. As we move through the education process we come to use certain mental pathways for solving problems and in the end continue to use those same pathways to solve problems. New ideas may be crowded out by conventional thinking.
Typically, in life this isn't a bad thing but in science it can be disastrous as new ideas are limited by the educational process. Some are not able to think outside the box and in the end it leads them to solving the same problems, living within the same mental frameworks, and in the end coming to deeper small solutions within the same line of thinking without ever changing the paradigm.
Elitism encourages cultural conformity from people who grew up with the same privilege and view themselves and their lifestyle as superior to others (It is equally likely that other defined groups also push for conformity). They are not necessarily "elite" but born with privilege. The elite person is the single mother who achieves a higher education, or a guy who invents something new in his garage. They are able to use ingenuity and grit to tackle unique problems.
In this case elitism is based on performance and actual existence in life. It is not based in the falsity of one's superior privilege rooted in things that made life easy for them. In higher education we need new ideas, new ways of doing things, and truly elite people who can think for themselves. A level of diversity is needed that can mix with resources to create new pathways for solving some of the world's problems.