On the evening of April 17 came the first of many back-and-forth messages concerning the campus phone lines. At 5:10 p.m., the first email was sent to Hamline students from the university, containing a message that stated that, “campus phone lines are down. For emergencies: call 6122979480 or security@hamline.edu. Please watch for more updates.”
Students had noticed issues contacting Hamline Public Safety (HPS) for over two hours before the all-campus alert, however, according to HPS, they were deemed officially out a quarter after 4 p.m. Around 5:30 p.m. Hamline’s Information Technology (IT) department was contacted and informed that HPS had no consistent success receiving or outsourcing phone calls.
From the time IT was informed about the issue until 7:30 p.m., Director of Infrastructure Systems and Services Anthony Schroeder spent time trying to diagnose the cause of the issue, to which he discovered that the internal phone system, which is run by the international technology company Avaya and managed by another tech corporation Black Box, on campus was not functioning as expected. From the Hamline side of the line, things seemed to be working correctly, however, the line from the outside to the inside was deemed corrupted.
“[Schroeder, Avaya and Black Box] went through a number of hours between 7:30 and about 10:00 [p.m.], trying to isolate what the issue was, and they didn’t have success doing that, so the decision that Anthony and Avaya and Black Box made was that they would basically shut down and entirely reboot our telecommunications infrastructure rather than investing more time trying to find the solution,” Hamline Librarian and Senior Lecturer Terry Metz said.
At 9:22 p.m., amid the search, the third email of the evening was sent out, explaining how external phone calls were unable to be received but internal phone calls were, meaning that students had to either call the phone number mentioned in the first email or reach out to security@hamline.edu to get in contact with the HPS dispatch team.
Despite saying that the lines would be up shortly, it was not until 6:22 a.m., the following morning that it was announced that the technical team was able to fix the issue at hand. As of now, the phone lines are operational and able to receive both external and internal calls. However, it does not end there.
“So now what’s happening is over the next three or four days, both Lumen and Blackbox/Avaya and Anthony are going to be going through logs to try to find out if there is more information where they can find out what actually caused this undesirable outcome to happen,” Metz said.
This is the first time anything of the sort has happened here in the ten years Metz has been employed, meaning that this is not a common issue and that whether this will happen again is up in the air. This campus-wide incident has brought to light a proposal that the IT departments have been suggesting to the university.
Hamline’s technology dates back to the early 1990s, a time when landlines and physical telecommunication equipment were at their prime. Because of this outdated technology, IT has suggested transitioning from hardlines to softlines, where they would transfer campus telecommunications to the cloud. This would mean that communications would be done through the same technology as everyday smartphones and computers rather than the traditional landline.
This process has yet to begin and will take several years to implement, as Metz explains, the department would rather have it done correctly and confidently over a while rather than done quickly and be uncertain about their work.
“The priority that Anthony made, and I think rightly so, was better to get the service back than spend another day trying to drill down and diagnose it and have another 12 hours of not having phone access,” Metz said. “We’re going to be moving to something else in the future because, like everything in life, new things are coming.”