It can be but it very much depends on personal circumstances.
Kallmann syndrome can be very severe with no pubertal development at all, leaving people physically very underdeveloped. This alongside the low hormone levels can make physical relationships difficult for both men and women with Kallmann syndrome. The level of physical development can vary a lot between patients so everybody's experience will be slightly different.
Early treatment and diagnosis can help but a lot of patients do have problems with their self image. This can be overcome once they meet the right partner. Physical relationships can be difficult to start but once the initial hesitation is overcome most patients with Kallmann syndrome can have an active sexual life.
Having an hormonal condition can affect emotions and it can be difficult for the partner to understand the feelings and emotions experienced by somebody with Kallmann syndrome but these can be overcome over time.
I think it is important to focus on the whole person, the whole personality. Partners will be attracted to the whole person and personality, regardless of the medical condition. Taking that first step to get physically involved with a partner can be very daunting for some patients but it is worth taking the risk. Support and advice from fellow patients can help.