Answer: Maybe, but probably due more to lifestyle than physiology. No matter your age, the mechanics of weight loss are the same for everyone--eat less and move more. Make sure you’re eating fewer calories than you burn.
Let’s address the questions above one at a time. First, is it harder to lose weight or fat if you don’t have that much to lose? To your body, losing 1lb is the same as losing 20lbs (one just takes longer than the other). Realistically though, if you have less weight to lose the urgency to do it may be less, affecting your discipline and/or dedication. If you are less overweight, then your current lifestyle may be close to what you comfortably will tolerate. Any significant deviation from that is “just not worth it” and causes more grief than the pleasure of the weight loss. This is not to say that you cannot do it, but rather “do you want to do it?” As we age, we tend to become more comfortable in our lifestyle. Eating and social habits, vacations, sedentary recreation, the enjoyment of food and alcohol--all of these things may bring us joy and pleasure, and giving them up becomes hard to justify. In the end it comes down to what you are willing to do to look a certain way. The changes you may have to make in order to lose that last bit of fat may be a nuisance and interfere with your enjoyment of life.
As for the impact of age, there are certainly changes that occur over one’s life that reduce metabolism. For every 10 years after age 25, your metabolism slows 2% (if your resting metabolic rate was 1200 calories/day, this would reduce it by 24 calories). This is a minor reduction and easily offset by moving a little more (300 steps or 3 minutes on an exercise bike) or eating less (1 bite of about anything).
So, to wrap it up, it is not physiologically harder to lose 10 lbs versus 50 lbs or to do it if you are 50 years old versus 25. But, the leaner you are at any age, the more difficult it is to lose “the last 10 pounds”, due to the likelihood of nagging hunger and the lifestyle concessions needed to make it happen.