Academics

Researcher's novel work on machine learning receives international recognition

Ting He, associate professor of computer science and engineering. Credit: Penn State College of Engineering / Penn StateCreative Commons

UNIVERSITY PARK, Pa. — In 2019, Ting He, associate professor of computer science and engineering at Penn State, and her fellow researchers published a paper in the Institute of Electrical and Electronics Engineers (IEEE) Journal on Selected Areas in Communications detailing the ways in which they — successfully and for the first time — optimized a specific type of machine learning technique known as federated learning. Now, that work has been recognized with the 2021 IEEE Communications Society Leonard G. Abraham Prize.

The paper, “Adaptive Federated Learning in Resource Constrained Edge Computing Systems,” was selected for the award based on its “quality, originality, utility, timeliness and clarity of presentation,” according to the society’s website.

“Federated learning has broad applications in scenarios where data are generated in a distributed manner and cannot be shifted to a centralized data center for training models due to a variety of reasons: limited bandwidth, privacy constraints and so on,” He said. “This award recognizes the importance of optimizing the performance of this federated learning process in order to unlock the value in distributed data.”

In order to explain federated learning, He describes geographically distributed surveillance cameras tasked with detecting different types of vehicles — by color or shape, for example — at their respective locations. To do this, the system must train something called a "classifier" to accurately categorize the vehicles. Instead of collecting data from the cameras at a data center and training the classifier there, which was the traditional approach, federated learning allows each camera to train the classifier locally. 

However, if each camera were to run a training algorithm based only on the local images, the quality would be compromised because only a fraction of the data would be used. To avoid this issue, according to He, the cameras periodically upload the models they trained so far to a central server, called the parameter server, which then aggregates the local models into one and pushes the combined model back to the cameras.

“It’s kind of like taking the average across the different classifiers, and then the server is going to broadcast back the aggregated model to the individual cameras,” He said. “The camera will then start the next iteration based on the server-broadcasted model. So, it’s just an iterative process, but with two levels: [the cameras] doing local iteration, and the server doing global iteration.”

According to He, there have been many follow-up works on optimizing various aspects of the federated learning process, but her initial work was one of the earliest to discuss optimizing the hyperparameters of federated learning such as the frequency of global aggregation.

The camera example illustrates some of the military surveillance and national security implications of the research, but He said the applications extend beyond that.

“This technique applies generally when you have distributed data sources that have some computational power, but not a lot, a central server with some bandwidth, but not enough, and then you need to train a machine learning model in this federated manner,” she said. “Then our technique will apply to tune the frequency of global aggregation in order to achieve a desirable trade-off between your computational costs and communication costs.”

This research was the result of a collaboration between researchers in the United States and United Kingdom. It was performed under the Distributed Analytics and Information Science International Technology Alliance Program that is jointly funded by the U.S. Army Research Laboratory and the U.K. Ministry of Defence.

 

Last Updated August 09, 2021

Contact