Why is this happening?
It feels like you need 183 people (50% of 365) to get a match, but that's wrong.
You aren't comparing everyone to you. You are comparing everyone to everyone.
The Formula:
With 23 people, there are 253 lines connecting them.
With 50 people, there are 1,225 connections!
That spiderweb in the background? That represents the explosive growth of possible matching pairs.