Impact statement: FedGKT can efficiently train large deep neural networks (CNNs) in resource-constrained edge devices (such as smartphones, IoT devices, and edge servers). Unlike past FL approaches, FedGKT demonstrates the feasibility of training a large server-side model by using many small client models. FedGKT preserves the data privacy requirements of the FL approach but also works within the constraints of an edge computing environment. Smartphone users may benefit from this technique because their private data is protected, and they may also simultaneously obtain a high-quality model service. Organizations such as hospitals, and other non-profit entities with limited training resources, can collaboratively train a large CNN model without revealing their datasets while achieving significant training cost savings. They can also meet requirements regarding the protection of intellectual property, confidentiality, regulatory restrictions, and legal constraints. As for the potential risks of our method, a client can maliciously send incorrect hidden feature maps and soft labels to the server, which may potentially impact the overall model accuracy. These effects must be detected and addressed to maintain overall system stability. Second, the relative benefits for each client may vary. For instance, in terms of fairness, edge nodes which have smaller datasets may obtain more model accuracy improvement from collaborative training than those which have a larger amount of training data. Our training framework does not consider how to balance this interest of different parties.