The few people I know in that situation either use "partner" where they're living together, or "Freddie's father" where they're not. In some cases, "Freddie's biological father".

Relationships can get complicated when one has three kids, is living with Man A, and the kids are by men B, C, and D, none of whom were married to you at the time of the relevant kid's conception, but may have been at other times. Actually, now I think about it, the person I'm thinking of has five kids....