He is right to a point. Passion is "concupiscence in action", our "desires out of control because of fallen human nature". Passions can rise up in any of us, for we all share that same nature, inherited from our first parents. We all have desires, and it is easy to let them run wild. Angelus gives us a decent definition of concupiscence - the tendency to sin that we all have inside, unwanted, unbidden, just waiting to be heard. We are meant to rise to singing of angels, not fall to the howling of animals, yet howl we do.
It is easy to obey our desires. We want to obey them. That's what makes them desires. But are we ruled by our passions? Do we have another choice? In some cases, a passion rises in us before our will is engaged. It is there driving us before we can consciously act. This "antecedent passion" lessons our freedom and can reduce our responsibility for sin. (Hardon) It is important to note, though, that it doesn't necessarily eliminate our responsibility. Angelus is right to a point. We obey these rising passions without thinking. We obey. However, habits and addictions can be broken, and we are called on to try, however long it takes. That is what Angelus misses - the answer to his question. That is the choice we have - to "reprogram" ourselves not to obey even these antecedent passions.
Angel is known for being "broody", but in this episode, Angelus broods over both the sleeping Buffy and the murdered Jenny Calendar. These desires - murdering Jenny, torturing Giles, and whatever designs he has on Buffy - are not sudden, unbidden howls from within. Not anymore. He dwells on them, long enough to draw them in detail. He has crossed the line from desires rising before the will is engaged, to consciously willed and developed desires. He is obeying not antecedent but consequent passion - desires that he is intentionally fostering.
What about us? It comes down to Angelus' question: "what other choice do we have?" We don't have a choice in where or when the sudden howls of desire come on us. Angelus is right in that much. We do have a choice, though, whether to accept or reject those inner voices. We can choose not to act on them, once our will is engaged, once we're aware of doing something wrong. We can choose, too, to work to overcome them - to break the habits that lead us into sin. Passion "speaks to us, guides us", but it need not rule us.
"...for the weapons of our battle are not of flesh but are enormously powerful, capable of destroying fortresses. We destroy arguments and every pretension raising itself against the knowledge of God, and take every thought captive in obedience to Chris." (2 Cor 10:4-5)
Hardon, John. 1999. "Hermeneutics". In The Modern Catholic Dictionary. Inter Mirifica.