Being faithful doesn't necessarily mean that you love only one person. The test of faithfulness is when you are faced with the inclination of love for another, but choose not to pursue or entertain it because of the love you have declared and committed to with your spouse.
What does this mean? It means that because you have committed and declared your love to someone, you will stick to it, even though at times you may feel that you don't love him or her anymore (but you really do). And it is this faithfulness that will grow into a deeper love and trust over time.
I don't know how weird this may sound, but I do think that I want my spouse to be someone I'm so close and comfortable with that I can tell him that I think another guy is cute without him getting upset, or me feeling trashy. I don't want to mind being told by him that he things another girl is pretty, and I don't want to get jealous. The thing is, to get to that kind of state, there must have been some kind if trust or faithfulness test, where I will have the knowledge and certainty that even if I'm not the prettiest, I'm the one that he has chosen. The thing is, to get to that kind of state, you must really believe in that love. It's like in the book "The Age of Innocence" where May tells her son that they can trust Newland Archer, because he gave up what he wanted most (Madame Olenska) for his earlier love and declaration for May.
That is commitment. That is faithfulness. And that is true love.
Do you figure that faithfulness to God is not just that we love Him because He is the only way? But also that we love him more and choose to love Him over all the other passions and idols in our lives.
There can be no faithfulness if there has been no choice.