UI layers merger: merging UI layers via visual learning and boundary prior

查看全文 作　　者：Yunnong [1,5]CHEN;Yankun [4]ZHEN;Chuning [2]SHI;Jiazhi [2]LI;Liuqing [2,3,5]CHEN;Zejian [1,3,5]LI;Lingyun [2,3,5]SUN;Tingting [4]ZHOU;Yanfang [4]CHANG

机构地区：[1]School of Software Technology,Zhejiang University,Hangzhou 310027,China;[2]College of Computer Science and Technology,Zhejiang University,Hangzhou 310027,China;[3]Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies,Hangzhou 310027,China;[4]Alibaba Group,Hangzhou 311121,China;[5]Zhejiang-Singapore Innovation and AI Joint Research Lab,Hangzhou 310027,China

出　　处：《Frontiers of Information Technology & Electronic Engineering》索引2023年第24卷第3期,共15页

基　　金：Project supported by the National Key R&D Program of China(No.2018AAA0100703);the National Natural Science Foundation of China(Nos.62006208 and 62107035);the Ng Teng Fong Charitable Foundation in the form of ZJU-SUTD IDEA Grant;the Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies。 摘　　要：With the fast-growing graphical user interface(GUI)development workload in the Internet industry,some work attempted to generate maintainable front-end code from GUI screenshots.It can be more suitable for using user interface(UI)design drafts that contain UI metadata.However,fragmented layers inevitably appear in the UI design drafts,which greatly reduces the quality of the generated code.None of the existing automated GUI techniques detects and merges the fragmented layers to improve the accessibility of generated code.In this paper,we propose UI layers merger(UILM),a vision-based method that can automatically detect and merge fragmented layers into UI components.Our UILM contains the merging area detector(MAD)and a layer merging algorithm.The MAD incorporates the boundary prior knowledge to accurately detect the boundaries of UI components.Then,the layer merging algorithm can search for the associated layers within the components’boundaries and merge them into a whole.We present a dynamic data augmentation approach to boost the performance of MAD.We also construct a large-scale UI dataset for training the MAD and testing the performance of UILM.Experimental results show that the proposed method outperforms the best baseline regarding merging area detection and achieves decent layer merging accuracy.A user study on a real application also confirms the effectiveness of our UILM. 关键词：User interface(UI)to code UI design lint UI layer merging Object detection

相关文献

参考文献（31）
引证文献（1）
耦合文献（0）

参考文献(31)

引证文献(1)

互联网出版许可证新出网证(渝)字10号全国400电话 - 免长途话费

UI layers merger: merging UI layers via visual learning and boundary prior

参考文献(31)

引证文献(1)

用户提示