Sea ice motions play an important role in the polar climate by transporting pollutants, heat, water and salt as well as changing the ice cover. Numerous physics-based models have been constructed to represent the sea ice dynamical interaction with the atmosphere and ocean. Here we propose a new data-driven deep-learning approach which utilizes a convolutional neural network (CNN) to model how Arctic sea ice moves in response to surface winds given its initial ice velocities and concentration a day earlier. Results show that CNN computes the sea ice response with a correlation of 0.82 on average with respect to reality, which surpasses a set of pixel-based predictions, such as persistence (PS), linear regression (LR), random forest (RF), multiple layer perceptrons (MLP) and CICE, a leading physics-based model. The superior predictive skill of CNN suggests the important role played by the connective patterns of the predictors of the sea ice motion.