I can only personally speak to the hunt-seat equitation sect of horseback riding and can comfortably say that I see the sides of both arguments. Historically, horseback riding was seen as more of a male-oriented activity, by necessity and for sport. Horse racing was seen as the sport of kings. Women would often ride side saddle, with both legs over one side of the saddle instead of one on each side. It was seen as scandalous for women to part their legs over the horse. They were not given many opportunities to ride outside of necessity. Now, at least in America, women dominate the sport.
On one hand, the sport in America is dominated by women and consists of much fewer men, especially in the divisions outside of higher level show jumping. It also is the only section of sports in the Olympics (show jumping, cross country, and dressage are all featured) where there is no separation of genders. However, it is also a sport for predominantly rich white women. Not many compete in the sport just based on pure talent; there is a surprising amount of money involved. From lavish stables to horses worth up to half a million dollars or more, not just anyone can ride. The daughters of Springsteen, Gates, Jobs, Harvey, Olsen, Cyrus, and Bloomberg all appear as riders in top levels of competition.
The point is that while American equestrian culture is dominated by women, there is still the problem of privilege and power that exists. A lot of women in my riding world would probably say that they consider the sport to be feminist, but it might not look that way from the outside.
It still takes a lot of commitment and effort to become a good rider; no amount of money can help anyone with that. The sport becomes more of a lifestyle if you are doing it right. The problem here is that talent without the financial means usually does not get you anywhere unless you have the proper connections. So, in America, it is farther out of reach for minorities or anyone who may not be financially within the one percent. This leaves a lot of people excluded. A lot of the sport is still controlled in a more financial sense by rich white heterosexual men. Although, in America even men who ride and/or compete come under a false stereotype. It is often assumed that male riders are gay, considering the tight jodhpurs, polo shirts, expensive clothes, and tall black boots. This ridiculous stereotype exists because people associate the sport with female qualities. While people consider the sport to be feminine, that does not make it a feminist sport. Personally, I would define it as a rich white feminist sport, which cannot really be called feminist in any way. Even internationally, the sport, at its higher levels, is tough to compete in without the resources that so few people have.