Shanjiang Tang

Shanjiang Tang's photo 

Assistant Prof./Master Supervisor

School of Computer Science& Technology
Tianjin University

Contact

55-B517, CS, Beiyang Campus, JinNan District, Tianjin, China, 300350 [map]
Tel: +86-13652068936
tashj [at] tju [dot] edu [dot] cn
http://cs.tju.edu.cn/faculty/tangshanjiang

Shanjiang Tang's research interest

About Me

I am an Assistant Professor in the School of Computer Science& Technology, Tianjin University. I received my Ph.D degree from Nanyang Technological University in 2015. I received the B.Eng. and M.Sc. degrees from School of Software Engineering and School of Computer Science& Technology at Tianjin University in 2008 and 2011, respectively.

My general research interests primarily focus on large-scale computing systems, big data, and cloud computing, with special emphasis on the resource management and job scheduling for Hadoop/YARN system. Specifically, I am interested in designing new scheduling and resource allocation algorithms, analyzing their performance, and implementing them in large-scale computing systems. I am also interested in problems at the intersection of computing systems and economics.


I'm looking for undergraduate and graduate students. If you are interested in big data, parallel computing and cloud computing. Please contact with me.

News

Publications   [Entry@DBLP] [Google scholar]

Journal Articles

  1. Shanjiang Tang, Zhaojie Niu, Bingsheng He, Bu-Sung Lee, Ce Yu, “Long-Term Multi-Resource Fairness for Pay-as-you Use Computing Systems,” IEEE Transactions on Parallel and Distributed System, 2017. [Supplementary Document]

  2. Xi Chen, Chen Wang, Shanjiang Tang, Ce Yu, Quan Zou,“CMSA: a heterogeneous CPUGPU computing system for multiple similar RNADNA sequence alignment,” BMC Bioinformatics, 2017.

  3. Chen Wang, Ce Yu, Shanjiang Tang, Jian Xiao, Jizhou Sun, Xiangfei Meng,“A General and Fast Distributed System for Large-scale Dynamic Programming Applications,” Parallel Computing, 2016.

  4. Zhaojie Niu, Shanjiang Tang, Bingsheng He, “ An Adaptive Efficiency-Fairness Meta-scheduler for Data-Intensive Computing, ” IEEE Transactions on Services Computing, 2016.

  5. Peng Ren, Shanjiang Tang, Fang Fang, et al., “Gait Rhythm Fluctuation Analysis for Neurodegenerative Diseases by Empirical Mode Decomposition,” IEEE Transactions on Biomedical Engineering, 2016.

  6. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, “Fair Resource Allocation for Data-Intensive Computing in the Cloud,” IEEE Transactions on Services Computing, 2016. [Supplementary Document]

  7. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, “Dynamic Job Ordering and Slot Configurations for MapReduce Workloads,” IEEE Transactions on Services Computing, 2015. [Supplementary Document]

  8. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, “DynamicMR: A Dynamic Slot Allocation Optimization Framework for MapReduce Clusters,” IEEE Transactions on Cloud Computing, 2014. [Supplementary Document][slides]

  9. Shanjiang Tang, Ce Yu, Jizhou Sun, Bu-Sung Lee, Tao Zhang, Zheng Xu, and Huabei Wu, “EasyPDP: An Efficient Parallel Dynamic Programming Runtime System for Computational Biology,” IEEE Transactions on Parallel and Distributed System, 2012. [Supplementary Document][slides]

Conference and Workshop Proceedings

  1. Shanjiang Tang, Ce Yu, Chao Sun, Jian Xiao, Yinglong Li, “ QKnober: A Knob-based Fairness-Efficiency Scheduler for Cloud Computing with QoS Guarantees,” in the 16th International Conference on Service Oriented Computing (ICSOC'18), Nov 2018.

  2. Hao Fu, Shanjiang Tang, Bingsheng He, Ce Yu, Jizhou Sun, “ GLP4NN: A Convergence-invariant and Network-agnostic Light-Weight Parallelization Framework for Deep Neural Networks on Modern GPUs,” in the 47th International Conference on Parallel Processing (ICPP'18), August 2018.

  3. Shanjiang Tang, Bingsheng He, Shuhao Zhang, Zhaojie Niu, “ Elastic Multi-Resource Fairness: Balancing Fairness and Efficiency in Coupled CPU-GPU Architectures,” in the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'16), Nov 2016. [slides]

  4. Zhaojie Niu, Shanjiang Tang, and Bingsheng He, “ Gemini: An Adaptive Performance-Fairness Scheduler for Data-Intensive Cluster Computing,” in CloudCom'15, Dec 2015. [slides]

  5. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, “ Towards Economic Fairness for Big Data Processing in Pay-as-you-go Cloud Computing,” in CloudCom 2014 (Ph.D. Consortium), Singapore, Dec 2014. [slides]

  6. Shanjiang Tang, Bu-Sung Lee, Bingsheng He and Haikun Liu, ‘‘Long-Term Resource Fairness: Towards Economic Fairness on Pay-as-you-use Computing Systems,’’ in the 28th International Conference on Supercomputing (ICS'14), Munich, Germany, June 2014. [slides]

  7. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, ‘‘Dynamic slot allocation technique for MapReduce clusters,’’ In IEEE Cluster 2013, Indiana , USA, Sept 2013. [slides]

  8. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, ‘‘MROrder: Flexible Job Ordering Optimization for Online MapReduce Workloads, ’’ In Euro-Par 2013, Aachen, Germany, Aug 2013. [slides]

  9. Jun Du, Ce Yu, Jizhou Sun, Chao Sun, Shanjiang Tang, and Yanlong Yin, ‘‘EasyHPS: A Multilevel Hybrid Parallel System for Dynamic Programming. ’’ in IPDPS Workshops 2013, Boston, USA, May 2013.

  10. Shanjiang Tang, Bu-Sung Lee, and Bingsheng He, ‘‘ Speedup for Multi-Level Parallel Computing,’’ in IPDPS Workshops 2012, Shanghai, China, May 2013. [slides]

  11. Shanjiang Tang, Ce Yu, Bu-Sung Lee, Chao Sun, and Jizhou Sun, ‘‘ Adaptive Data Refinement for Parallel Dynamic Programming Applications,’’ in IPDPS Workshops 2012, Shanghai, China, May 2012. [slides]

  12. Zhen Xu, Jizhou Sun, Ce Yu, Huabei Wu, Xiaojing Meng, Shanjiang Tang, ‘‘ A Visual Model-Driven Rapid Development Toolsuite for Parallel Applications,’’ in CSIE, 2009, Los Angeles CA, March 2009.

Book Chapter

  1. Shanjiang Tang, Bingsheng he, Haikun Liu, Bu-sung Lee, “Resource Management in Big Data Processing Systems,” invited chapter in Big Data: Principles and Paradigms. (Eds. R. Buyya, R. N. Calheiros, and A.V. Das), Academic Press, June 2016.

  2. Chen, Wang, Shanjiang Tang, Ce Yu, “Parallel Dynamic Programming for Large-scale Data Applications,” invited chapter in Horizons in Computer Science Research, Volume 14. (Eds. Thomas S. Clary), Nova Science Publishers, Feb 2017.

Patents

  1. Chao Sun, Jizhou Sun, Shanjiang Tang, et al., Multi-level parallel programming method, CN201010205530, Nov 2010.

  2. Ce Yu, Shanjiang Tang, Jizhou Sun, et al., Parallel programming model system of DAG oriented data driving type application and realization method, CN200910312089, May 2010.

  3. Chao Sun, Jizhou Sun, Shanjiang Tang, et al., Visual modeling and code skeleton generating method for supporting design of multinuclear parallel program, CN201010171361, Nov 2010.

  4. Ce Yu, Jizhou Sun, Zhen Xu, Huabei Wu, Shizhong Liao, Xiaojing Meng, Shanjiang Tang, et al., MPI parallel programming system based on visual modeling and automatic skeleton code generation method, CN200910067715, June 2009.

Research Fundings

  1. Efficient Spark-based Query Processing Techniques for Big Spatial Data in Supercomputing Systems (2018.4 - 2021.03), 200K, PI.

  2. Parallel and Distributed Computing (2018XRG-0027, 2018.1 - 2019.12), 100K, PI.

  3. Fair Resource Allocation for Big Data Processing in Multi-tenant Cloud Computing System (NSFC 61602336, 2017.1 - 2019.12), 210K, PI.

Projects

I focus on systems and algorithms for large-scale data-intensive computing. My projects include:

MRYARN: Pay-as-you-go is a popular billing model based on users' resource usage in the cloud. A user's demand is often changing over time, indicating that it is difficult to keep the high resource utilization all the time for cost efficiency. Resource sharing is an effective approach for high resource utilization. In view of the heterogeneous resource demands of workloads in the cloud, multi-resource allocation fairness is a must for resource sharing in cloud computing. MRYARN is proposed for multi-resource fair allocation on the cloud. It ensures that each user in cloud computing can at least get the amount of total resources as that under the exclusively non-sharing environment in the long term. Moreover, MRYARN can guarantee that no users can get more amount of total allocated resources over time by lying their demands. Finally, MRYARN has a mechanism to discourage users to submit cost-inefficient workloads, especially when there are some idle resources they truly do not need.(homepage)(TPDS'17)

LTYARN: Life is not fair, but with a little help, existing large-scale data processing systems (e.g., YARN, Spark, Dryad) can be, ensuring resource sharing between users. However, past work on fair sharing considered memoryless fairness, an instantaneous fair share without historical information considered. When it comes to cloud computing (i.e., pay-as-you-use computing), it fails to satisfy the service-as-you-pay fairness (i.e., the total service that each user enjoys should be proportional to her payment) from a long-term view. Long-Term Resource Fairness (LTRF) generalizes max-min fairness for this case. LTYARN implements LTRF for YARN in cloud computing. (homepage) (demo) (ICS'14) (CloudCom'14)(TSC'16)

DynamicMR: Hadoop MRv1 uses the slot-based resource model with the static configuration of map/reduce slots. Due to the pre-configuration of distinct map slots and reduce slots which are not fungible on slave nodes, slots can be severely under-utilized, which significantly degrades the performance. Although YARN was proposed to address this problem by giving a new resource model of 'container' that either map and reduce tasks can run on, we keep the slot-based model by proposing an alternative technique called Dynamic Hadoop Slot Allocation (DHSA). It relaxes the slot allocation constraint to allow slots to be reallocated to either map or reduce tasks depending on their needs. Our experiments show that it consistently outperforms YARN by about 2% ~ 9% for multiple jobs due to the ratio control mechanism of running map/reduce tasks. Second, the speculative execution can tackle the straggler problem, which has shown to improve the performance for a single job but at the expense of the cluster efficiency. In view of this, we propose Speculative Execution Performance Balancing (SEPB) to balance the performance tradeoff between a single job and a batch of jobs. Third, delay scheduling has shown to improve the data locality but at the cost of fairness. Alternatively, we propose a technique called Slot PreScheduling that can improve the data locality but with no impact on fairness. Finally, by combining these techniques together, we form a step-by-step slot allocation system called DynamicMR that can improve the performance of Hadoop MRv1 significantly while maintaining the fairness. (homepage) (TCC'14) (Cluster'13)

MROrder: In Hadoop MRv1, different job submission orders will bring significantly varied performance results. MROrder is an automated MapReduce job ordering optimizaton prototype system. It targets at the online MapReduce workloads where MapReduce jobs arrives over time for various perfomane metrics, such as makespan, total completion time. Users just need to input some simple arguments. For example, users need to designate the job ordering performance metric( e.g., makespan, total completion time). The MROrder then starts to perform job ordering optimization automatically for online MapReduce jobs, based on user's configuration. (homepage) (Euro-Par'13) (TSC'15)

EasyPDP: To tackle the growing volume of genomic data, an efficient and easily-programmed high performance computing system is needed. EasyPDP is a parallel dynamic programming runtime system and an abstract programming model for computational biology and scientific computing applications. As a runtime system, it handles low-level thread creating, mapping, resource management, and fault tolerance issues automatically regardless of the system characteristics or scale. As an programming model, it allows users to describe applications and specify concurrency from the high level, without concerns about the complex details of parallel programming. (homepage) (demo) (TPDS'12) (IPDPS Workshop'13) (IPDPS Workshop'12) (Parallel Computing'16)

Open Source

Almost all of my work is open source:

Others

Personal Activities

I had experiences on a number of programming contests since I was a undergraduate student. I participated in the ACM International Collegiate Programming Contest in 2005 and 2006, and won 'Golden Prize' in the 31th ACM/ICPC Asia Programming Contest VS Google Cup on Programming Invitational Tournament of National College Students. I've now stopped doing contests, but I still love algorithmic and mathematical problems.

During my master study in 2009, I had internship at IBM CRL over four months during the summer holiday, where it germinates my strong interest on research.

I enjoy reading, swimming, fitness and playing badminton.

Professional Activities

PC Member for

Reviewer for

Interesting Links