arrow
Return

Code generation from a graphical user interface via attention-based encoder-decoder model

delete2021-05-18
delete5
PRE
AI
W
Wen-Yin Chen
P
Pavol Podstreleny
Y
Yung-Yao Chen
K
Kai‐Lung Hua *
DOI:10.1007/s00530-021-00804-7delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Code generation from graphical user interface images is a promising area of research. Recent progress on machine learning methods made it possible to transform user interface into the code using several methods. The encoder-decoder framework represents one of the possible ways to tackle code generation tasks. Our model implements the encoder-decoder framework with an attention mechanism that helps the decoder to focus on a subset of salient image features when needed. Our attention mechanism also helps the decoder to generate token sequences with higher accuracy. Experimental results show that our model outperforms previously proposed models on the pix2code benchmark dataset.
Keywords:
Code generation
Encoder-decoder
Attention
Graphical user interface
AI Summary

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.

Journal

Multimedia Systems cover
Multimedia Systems
IF:
3.1
Papers:
2.8K
Citations:
2.7K

Organization

N
National Yang Ming Chiao Tung University
Scholars:
2.5W
Papers: 2.3W
Citations: 2.2W
N
national taiwan university of science & technology
Scholars:
8.8K
Papers: 8.7K
Citations: 9
National Taipei University of Education cover
National Taipei University of Education
Scholars:
203
Papers: 290
Citations: 307
researcher View more organizations
Cited Papers

Cited Papers

ImageNet Large Scale Visual Recognition Challenge
err2015-04-11
err2.7W
PREAI
errRussakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C.; Fei-Fei, Li
errShare
errSave
Acid catalysis in hydrolytic reactions
err1929-01-01
err0
PREAI
errJ. N. Brönsted; W. F. K. Wynne-Jones
errShare
errSave
Intelligent deployment of UAVs in 5G heterogeneous communication environment for improved coverage
err2017-05-01
err99
PREAI
errSharma, Vishal; Srinivasan, Kathiravan; Chao, Han-Chieh; Hua, Kai-Lung; Cheng, Wen-Huang
errShare
errSave
errShare
errSave
A comparative study of data fusion for RGB-D based visual recognition
err2016-04-01
err56
PREAI
errSanchez-Riera, Jordi; Hua, Kai-Lung; Hsiao, Yuan-Sheng; Lim, Tekoing; Hidayati, Shintami C.; Cheng, Wen-Huang
errShare
errSave
researcher View more