Here’s an idea.
After every game you rank your opponent on a scale of 1/10 based on how enjoyable they were to play.
If they’re a sweat then they get ranked 1/10
Don’t take your rating personally but essentially if you’re a sweat people give you one star and stick you in a category where you play the other sweats
The people that offer a more casual experience get rated higher, and play other people with a similar casual play style.
Also add a feature where you can search for opponents similar to your ranking or change it if you do want the challenge of playing more skilled players.