Mobile artificial intelligence-generated content (AIGC) is an innovative technology that can support the evolution and updating processes of virtual twins (VTs) in human digital twin (HDT) systems. With a reliable and efficient automatic information generation process, the requirement for a timely physical-to-virtual synchronization in HDT can be satisfied. While such an AIGC-enabled HDT system can facilitate modelling high fidelity VTs, generating rare disease data and providing timely customized services, it may suffer from poor understanding of contexts, lack of creativity, and various security and privacy concerns. In this paper, we propose a new framework, which integrates federated learning (FL) and semantic communication (SemCom) to enhance performance in such a system while improving accuracy and convergence properties. Such an integrated FL-enhanced SemCom (FLeS) solution, however, comes with its own challenges. First, we present a holistic architectural framework for the proposed FLeS paradigm for mobile AIGC-enabled HDT systems and discuss the associated design requirements and challenges. We later present some key technologies necessary to realize such a solution before elaborating on some technical issues to suggest future directions. We believe that this article will open up new research opportunities and motivate new research efforts toward incorporating FLeS techniques for mobile AIGC, especially in emerging mobile services such as HDT.