Rosh Hashana is the only major Jewish festival which falls on the first day of the Jewish (lunar) month; that is, at the new moon. Passover and Sukkot, the calendar's most prominent celebrations, both start on the 15th, at the full moon, while Shavuot and Yom Kippur take place while the moon is waxing.
This has both symbolic and practical significance. Jewish sources tend to see the moon as symbolic of the Jewish people, with the sun symbolizing the world in general, and foreign empires in particular. The sun shines powerfully and constantly (despite rare eclipses), representing the overwhelming power and constancy of global superpowers throughout history (though their identity changes).
The moon, though the same apparent size as the sun, is always much dimmer, shining with cool reflected light, visible primarily at night when the sun is absent. Most significantly, the moon waxes and wanes, cycling through phases from invisibility to full strength. This is seen as symbolic of the Jewish people; though at times weak to the point of insignificance and always much less prominent than the powerful nations, we will in due time be blessed to return to our full strength.
Thus, Passover and Sukkot, celebrating the redemption of the Jewish nation from Egypt, are appropriately scheduled when the moon is full. Furthermore, both are set for the equinoxes, when the day and night are equal, with the sun and the full moon sharing the 24-hour cycle evenly. The moon and the sun, like the day and the night, or the winter and the summer, are at a rare point of balance.
Rosh Hashana, by contrast, is primarily not about the Jewish people in particular. It marks the creation of the world with a coronation of God as the King of the Universe and Supreme Judge. All creatures pass before Him in judgment; all beings acknowledge his sovereignty. Appropriately, on Rosh Hashana the moon is nowhere to be seen. It is a day of universality, of the constancy and power of the sun.
More on the practical implications in a separate post, God willing.