Taking water weight fluctuations out of the equation because it has nothing to do with losing body fat, one single factor determines the rate of weight loss: the average daily calorie deficit. This is the difference between how much energy is expended (calories burned) and the amount supplied (calories consumed). The goal is to proceed at sustainable pace and one that does not compromise health or performance.
That said, generally speaking, the heavier you are the easier it is to shed those first few fat pounds simply because you can eat more than a lighter person (the heavier you are the more calories you burn). Example: because there are 3,500 calories in a pound of body fat, a 250 lb person that maintains body weight by consuming ~4,000 calories per day and wants to lose 3 pounds per week, would need to consume ~2,500 calories per day. This translates to a 1,500 daily calorie deficit.
If a smaller, 150 lb person maintains their weight by eating ~2,500 calories per day and wants to lose the same 3 pounds per week, they would only be able to consume 1000 calories a day, which would be miserable for most. Eventually, as this 150 pounder loses weight, their new maintenance calories would also be lower, making it extremely difficult to create a deficit of this size. So, weight loss happens at a much slower rate.
None of the above changes the fact that the rate of weight loss is solely determined by the calorie deficit because this 150 lb person could add exercise so they too could burn 4000 calories per day. This would allow them to eat 2,500 calories a day and still lose 3 pounds per week. So keep in mind, the leaner you get the more work and diligence it takes to lose weight. It’s much more comfortable to lose weight at a slower pace.