I often hear people say that a horse is "wasted" if it is not ridden (or driven). Many people choose not to ride for a huge variety of reasons, this is a legitimate choice and one I made for myself a few years ago. Non ridden does not necessarily mean that the horse isn't spending quality learning or exercise time with their human it can incorporate groundwork, liberty and trail walking for example. Or maybe you don't do anything but love and care for your horse. It does not mean the person is anti riding. The original reason I chose not to ride was because my available time for riding is so incredibly ad hoc e.g. maybe I could ride Sunday, but then not for 3 weeks, then 4 months, then 4 days in a row. I felt this inconsistency unfair to my horses. I still have rideable horses but also have a collection that are unrideable, either aged, too small, too young or have health issues. I cannot say I will never ride again but the change to an unridden relationship with my horses has been amazing on many levels and feels good.
I value every second I spend with them and they do not have to do anything to "earn their keep". I do not believe a horse in the paddock is going to "waste" as long as they are loved and cared for. Non ridden is a lifestyle choice. If it feels right for you, embrace it.