I don't consider this to be a harem manga at all. It slightly teases at first, but his interest is clear. There's really only one rival to spice things up.
For the other characters, they never progress romantically. The classmate confesses in the beginning, but after being immediately turned down she simply becomes a friend. The maid's mysterious air gave the impression of potential, but again nothing happens with her either. The "trap" character is also entirely just a friend (compared to, say, the blush moments in Yahari Ore no Seishun Love Comedy wa Machigatteiru), and now even has a crush of his own. The sister is just a sister, and the mother is just a loli-mom.
At first I'm hesitant to read this because of the harem. But after just three chapters, it seems to me that Masamune's world revolves around her.It's so obvious he's still in love with her though he doesn't want to realize it. I'm excited to see what his reaction would be if he found out that it's not Aki who called him "piggy" . I need developments between (▰˘◡˘▰)them