If you do not have an explicit don’t as don’t tell policy in place in your relationship, then having partners or families that your partner(s) don’t know about is wrong.
YOU ARE CHEATING.
Polyamory is about honesty. And you can’t pull the I didn’t know poly was an option bc there are plenty people out there living ethical non-monogamous lives without knowing there is a word for it or a community.
You can not justify cheating with a poly nature.