Messages, messages: Why e-mail at Cornell was sluggish in September

Over a million messages zip in and out of Cornell's e-mail system on a typical weekday during the semester. Most of the time, the four machines that make up the system are very quick, each processing messages in under 5 seconds. But mix in 200,000 more messages a day and several mass mailings to the campus community, followed by a few (thwarted) mail bombs, and the result is the e-mail sluggishness seen in September.

For three weeks after the Sept. 11 tragedy, Cornell Information Technologies (CIT) saw a dramatic, sustained jump in e-mail and Internet usage. Up to 20 percent more e-mail was being sent during the daytime -- the hours in which the system had already been near capacity.

"Some of our equipment is near the end of its life cycle. We had planned to boost the capacity by upgrading in the spring," said Don Mac Leod, CIT assistant director of client/server systems. "Instead, we did one of those upgrades in that first week. It helped stabilize the system, but the volume of messages continued to exceed capacity, and we had messages backlogged at times."

Also during the first week, President Hunter Rawlings needed to communicate rapidly with the entire campus about coping with the Sept. 11 events. A Cornell-built technology called Message Blaster ensured that all users would receive a copy of the messages as soon as they checked their e-mail.

"Message Blaster is reserved for emergency-type situations," said Jim Howell, CIT e-mail system administrator. "We had never needed to use this tool repeatedly for a few days in a row. It amounted to another 200,000 extra messages, which stressed our e-mail system even more."

The next week, the e-mail system was hit by a few attempted mail bombs. A mail bomb tries to bring down a site by flooding it with e-mail. In these cases, Cornell was meant to be a relay, not the target. By using Cornell to route attacks on another site, the mail bombs would be harder to trace. Blocking these attempts further taxed the e-mail system. CIT ultimately tracked down and stopped the sources of the attacks.

Although delivery of messages was slower during those three weeks, the e-mail system never failed and no messages were lost. To support what was clearly an essential communications tool, the e-mail team worked long hours, manually adjusting and tweaking the system to keep performance as high as possible. By Oct. 1, the demand for e-mail returned to normal levels and so did the speed.

The "@cornell.edu" column is edited by Beth Goelzer Lyons of Cornell Information Technologies (CIT). Please send suggestions to citnews@cornell.edu . For more technology news, visit the CIT News web site http://www.cit.cornell.edu/citpubs/news/.