Its still one of the better harems and more believable than those harems with milquetoast protagonists who just go with the flow. Ryuu is one of the best harem protags Ive seen. Not a self insert, flawed as hell, quirky, does what he wants instead of letting the girls drag him around.
A woman who doesn't want sex has no business getting married. Sex is pretty much central to marriage, otherwise, just be pals. She took his money and she agreed to the marriage. He gave up fortune to get her out of debt and lives in a crap house with a crap job. For her. He kept up his end. I see her as the problem, frankly.
(Every time I try to write this part, it becomes long winded.)
Basically, this is not how a relationship like this works in real life and I'm annoyed that Ryuu pushes Koharu even though she is obviously not comfortable and not enjoying it.