Faith is a subject of much debate not only between atheists and theists, but even among theists themselves. The nature of faith, the value of faith, and the appropriate subjects of faith — if any — are topics of intense disagreement. Atheists frequently argue that it's wrong to believe things on faith while theists argue that not only is faith important, but that atheists also have their own faith. None of these discussions can go anywhere unless we first understand what faith is and is not.
Clear definitions of key terms are always important, but they are especially important when discussing faith because the term can mean very different things depending on context. This creates problems because it's so easy to equivocate about faith, starting an argument with one definition and finishing with another.
Faith as Belief Without Evidence
The first religious sense of faith is a type of belief, specifically belief without clear evidence or knowledge. Christians using the term to describe their beliefs should be using it in the same way as Paul: "Now faith is the substance of things hoped for, the evidence of things not seen." [Hebrews 11:1] This is the sort of faith Christians often rely upon when confronted with evidence or arguments that would disprove their religious beliefs.
This sort of faith is problematic because if a person really does believe something without evidence, even weak evidence, then they have formed a belief about the state of the world independent of information about the world. Beliefs are supposed to be mental representations about the way the world is but this means beliefs should be dependent upon what we learn about the world; beliefs shouldn't be independent of what we learn about the world.
If a person believes something is true in this sense of "faith," their belief has become separated from facts and reality. Just as evidence plays no role in producing the belief, evidence, reason, and logic can't disprove the belief. A belief that is not dependent on reality also can't be refuted by reality. Perhaps this is part of how it helps people endure the seemingly unendurable in the context of tragedy or suffering. It's also arguably why it's so easy for faith to become a motivation for committing unspeakable crimes.
Faith as Confidence or Trust
The second religious sense of faith is the act of placing trust in someone. It may involve no more than having faith in the words and teachings of religious leaders or it may be faith that God will fulfill promises described in scripture. This sort of faith is arguably more important than the first, but it's one which both theists and atheists tend to ignore in favor of the first. This is a problem because so much of what believers say about faith only makes sense in the context of this sense.
For one thing, faith is treated as a moral duty, but it's incoherent to treat any belief as a "moral duty." In contrast, having faith in a person who deserves it is a legitimate moral duty while denying faith to someone is an insult. Having faith in a person is a statement of confidence and trust while refusing to have faith is a statement of distrust. Faith is thus the most important Christian virtue not because believing that God exists is so important, but rather because trusting God is so important. It's not mere belief in the existence of God which takes a person to heaven, but trust in God (and Jesus).
Closely connected to this is the treatment of atheists as immoral merely for being atheists. It is taken for granted that atheists actually know that God exists because everyone knows this — the evidence is unambiguous and everyone is without excuse — so one has "faith" that God will be honorable, not that God exists. This is why atheists are so immoral: they are lying about what they believe and in the process are denying that God deserves our trust, allegiance, and loyalty.
Do Atheists Have Faith?
Claims that atheists have faith just like religious theists usually commit the fallacy of equivocation and that's why atheists vehemently dispute it. Everyone believes some things on meager or inadequate evidence, but atheists don't typically disbelieve in gods on "faith" in the sense of not having any evidence whatsoever. The sort of "faith" which apologists try to bring in here is usually just belief that falls short of absolute certainty, a confidence based on past performance. This is not "the substance of things hoped or" or "evidence of things unseen."
Faith as trust, however, is something that atheists have — as do all other human beings. Personal relationships and society as a whole wouldn't function without it and some institutions, like money and banking, depend entirely upon faith. It can be argued that this sort of faith is the foundation of human relationships because it creates the moral and social obligations which bind people together. It's rare to completely lack any faith in a person, even one who has proven to be generally untrustworthy.
By the same token, though, this sort of faith can only exist between sentient beings capable of understanding and agreeing to such obligations. You can't have this sort of faith in inanimate objects like a car, in systems like science, or even in non-sentient beings like goldfish. You can make assumptions about future behavior or place bets on future outcomes, but not have faith in the sense of investing personal trust in moral reliability.
This means that the moral virtue of Christian faith depends entirely on the Christian god existing. If no gods exist, there is nothing virtuous about trusting in any gods and there is nothing immoral about not trusting in any gods. In a godless universe, atheism isn't a vice or sin because there are no gods to whom we owe any allegiance or trust. Since faith as belief without evidence is neither legitimate nor a moral issue, we come back to the obligation of believers to provide sound reasons to think their god exists. In the absence of such reasons, atheists' disbelief in gods is neither intellectually nor morally problematic.