基于双标签混合Transformer的骨骼关键点重定向方法OA

To address the problem of image generation distortion caused by skeletal proportion differences in human motion transfer,a dual-label hybrid Transformer(DLHFormer)architecture based on skeletal keypoints was proposed to achieve direct motion transfer between different skeletons.First,a DLHFormer encoder was used to accurately decouple and encode the dynamic labels and static labels of the target person's skeletal keypoint motion sequences.Then,an initial pose predictor was used to predict the initial skeleton pose of the target skeletal keypoint sequence associated with the motion sequence,eliminating the limitations of predefined poses.Finally,motion retargeting was realized in the framework based on the encoded static labels of the source person's skeleton and the dynamic labels of the target person's skeleton.Experiments were conducted on a cross-character image dataset constructed based on Mixamo.The results show that after integrating the proposed method into MagicPose,the average L1 loss was reduced from 3.69 to 3.42,PSNR was increased from 19.92 dB to 20.33 dB,SSIM was increased from 0.996 9 to 0.997 5,LPIPS was reduced from 0.195 to 0.183,and FID was reduced from 75.5 to 68.9.In the skeletal keypoint retargeting task,when combining spatial and temporal positional encoding,PCK reached 0.993 and MPJPE was reduced to 2.95.Therefore,the proposed method can effectively maintain human body structure proportions,improve keypoint localization accuracy,and enhance generated image quality in cross-skeleton motion transfer.