| 摘要: |
| 目的 由于跨模态数据集有限和模态异构表征问题，利用跨模态检索算法解决实际应用问题一直是当前多模态研究中的一大研究方向。方法 提出了一种面向传统服饰的细粒度跨模态检索算法，解决传统服饰跨模态检索的单模态表征和跨模态表征一致的问题。在单模态特征表征方面，沿用DCMH使用深度学习的方法对初始数据进行特征提取;在跨模态表征一致方面，新增自监督语义网络，以自监督的方式对应标签信息提取细粒度信息，并将其用于图文哈希学习的监督，从而得到更好的图文哈希表征。通过在传统服饰数据集上与其他方法进行对比实验，验证算法的有效性。结论 有关此方面的应用探索，有利于解决互联网时代中国传统服饰文本、图像处理等的保护性难题，为未来纹样检索中的工作做铺垫，实现中国传统服饰的创新性传承和发展。 |
| 关键词: 跨模态检索 哈希 细粒度 自监督 |
| DOI：10.19554/j.cnki.1001-3563.2021.22.005 |
| 分类号:TB472 |
| 基金项目:新疆维吾尔自治区重点实验室开放课题(2017D04009) |
|
| Fine-grained Cross Modal Retrieval Algorithm for Traditional Clothing |
|
ZHAO Hai-ying1, XIANG Xiang1, LI Jie1, ZHANG Jia-wei2
|
|
(1.School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China;2.Xinjiang Laboratory of Education Cloud Technology and Resources, Xinjiang Teacher's College, Urumqi 830043, China)
|
| Abstract: |
| Due to the limitation of cross modal data sets and the heterogeneous representation of modes, it is always a major research direction to solve the practical application problems by using cross modal retrieval algorithm. In this paper, a fine-grained cross modal retrieval algorithm for traditional clothing is proposed to solve the problem of consistency between single-mode representation and cross mode representation of traditional clothing cross modal retrieval. In the aspect of single-mode feature representation, DCMH is used to extract features from the initial data using deep learning method; In the aspect of cross modal representation consistency, a new self-supervised semantic network is added to extract fine-grained information from label information in a self-supervised way, and it is used to supervise the hash learning of images and texts, so as to obtain better hash representation of images and texts. The effectiveness of the proposed algorithm is verified by comparing with other methods on traditional clothing data sets. The application and exploration of this aspect will help solve the protective problems of the text and image processing of traditional Chinese clothing in the Internet era, and realize the innovative inheritance and development of traditional Chinese clothing. |
| Key words: cross modal retrieval hashing fine-grained self-supervised |
渝公网安备 50010702501716号