A little diversity can go a long way. If all your guests had the same interests, opinions and lifestyles, conversation could be a little predictable. Guests from different age groups can also learn from and entertain each other as perspectives and experiences can be so different. My friends are from different countries and backgrounds and have different life experiences and I love them all. Of course, there is always one I had thought of as a friend who then proved to be otherwise but such is life.
Go ahead and invite people you like - if the room is full of mutual respect and friendship, it will be a great lunch.