It's even worse if you just consider the very top. Out of 500 companies, there are only 24 female CEOs.
"It's kind of shocking really. With all of the attention the issue gets, you'd think companies would be doing better, " said Rita McGrath, a professor at Columbia Business School.
In recent years, there have been a number of high-profile female CEO hires, including Lockheed Martin CEO Marillyn Hewson and General Motors CEO Mary Barra, the first woman to ever lead a major auto maker.
Progress likely to be slow: But don't expect more women to break through the ranks unless there is a strong pipeline of female talent behind the CEO.
That's why CNNMoney analyzed the next four executive positions - chief financial officers, chief operating officers and other key roles at major companies. Women hold 16.5% of these four positions just below CEO in the S&P 500. Yet that's still a small pool of leaders to draw from.
"If you don't have women in the pipeline, they are not going to get the top job, " McGrath said.
The number of women running S&P 500 companies will actually fall to 23 in April when Maggie Wilderotter steps down after nearly a decade atop Frontier Communications.
Wilderotter told CNNMoney the number of female chief executives is far too low, especially given the motivation and credentials of women climbing today's corporate ladder.
"We didn't get to these jobs without being fearless and courageous at the same time as being good at what we do, " she said.
Stereotypes persist: So why aren't more women making it into the C-Suite?
"When we close our eyes and picture what a CEO looks like, too often the picture that comes to mind is that of a white man, " said Deborah Gillis, CEO of Catalyst, a nonprofit pushing to improve corporate gender diversity.
Women also don't have the critical support needed from high-ranking sponsors who can actively campaign for their advancement.
"It doesn't just mean being a mentor. Sponsorship means you are really fighting for that person, " said McGrath.
Mission-critical jobs go to men: Women are also lagging far behind their male counterparts when it comes to landing the critical profit-and-loss (P&L) jobs that serve as grooming roles for future leaders. Many successful women end up in roles in human resources and investor relations that are important yet don't serve as a gateway to the top of the C-Suite.
"We need to give women more full P&L experience and the task of managing brands - the nuts and bolts of the corporations, " Denise Morrison, CEO of Campbell Soup, told CNNMoney.
Morrison, the sister of Wilderotter, climbed the corporate ladder at Procter & Gamble and then Campbell by running brands, learning the supply chain and taking on other operational roles.
"There is a disproportionate number of women in staff roles. It's on the backs of both women and the corporations" to fix this problem, she said.
Gender diversity actually works: The push to get more females at the top isn't just about womens' rights. The best case for bringing gender diversity into the C-Suite is made by cold, hard cash.
Companies with a high representation of women board members significantly outperformed those with no female directors, according to a 2011 Catalyst analysis of financial results at Fortune 500 companies.
In other words, having leaders with diverse experiences and backgrounds often translates to financial success. That's largely due to the fact that women bring different skills than men and that can lead to more thoughtful deliberations about risk-taking and appealing to female consumers.
Wilderotter said women are "very good at multitasking, have a sixth sense with people, are nurturing and service-oriented."
Game-changer? Stay-at-home dads: Yet many women who want to have families are forced to give up their dreams for senior management positions due to the demands of the job.
"It's traditionally and will be for the foreseeable future an all-in enterprise to be a chief executive. It becomes your whole life, " said Stewart Friedman, director of the Wharton Work/Life Integration Project.
While women still bear the primary responsibility for childrearing, that dynamic is changing. A study by Friedman shows that 42% of the 2012 graduating class at Wharton Business School say they want to have children. That's down from nearly 80% in 1992.
At the same time, fewer men assume all of the household resources will be devoted to their careers at the expense of their spouse's, Friedman said.
"We are seeing more and more stay-at-home dads, enabling more women to go all-in in their careers, " he said.
Moving in the right direction: There are other reasons to be hopeful that the gender diversity problem will eventually get better.
In 2013 women held 16.9% of the board seats in the Fortune 500, up from 11.7% in 2000, according to Catalyst. That's important because these directors are the ones in charge of hiring CEOs and identifying future leaders in the pipeline.
It should be no surprise then that Thomas Saporito, CEO of executive search firm RHR International, has witnessed a major push by boards to hire more female senior executives.
"This is trending in a better direction, and there is a greater focus on development of women, but there is still much work to do - both at the top management level and in the boardroom, " he said.
Moreover, the hiring - and success - of leading women like Indra Nooyi at Pepsi and Phebe Novakovic at General Dynamics should help shatter the stereotype that only men can hold these top jobs.
Is legislation needed? Some European countries like Germany are attempting to close the gender gap by mandating a certain percentage of board seats go to women.