| 魏波* ** ***,刘晓东*,阎雨庭*,赵晓芳*,袁雨馨* ***,唐宏伟*.细粒度的流计算执行效率优化方法[J].高技术通讯(中文),2021,31(1):21~30 |
| 细粒度的流计算执行效率优化方法 |
|
| |
| DOI：10.3772/j.issn.1002-0470.2021.01.003 |
| 中文关键词: 流计算引擎； 执行效率优化； 细粒度； 元组 |
| 英文关键词: stream computing engine, optimization method of execution efficiency, fine-grained, tuple |
| 基金项目: |
| 作者 | 单位 | | 魏波* ** *** | | | 刘晓东* | | | 阎雨庭* | | | 赵晓芳* | | | 袁雨馨* *** | | | 唐宏伟* | |
|
| 摘要点击次数: 2050 |
| 全文下载次数: 1395 |
| 中文摘要: |
| 流计算引擎如Storm被广泛地应用于大数据实时处理中，以提高数据处理的执行效率，但该引擎因统计信息粒度粗、无法准确定位性能瓶颈，造成了难以提高数据处理的执行效率问题。为此，本文提出了一种细粒度流计算执行效率优化方法，该方法以元组为中心进行细粒度的性能分析，包含性能瓶颈识别算法和性能瓶颈缓解算法，支持量化地选择最优参数配置以提高执行效率。实验结果表明，在3个标准程序32个不同配置场景下，该方法能够准确地识别流应用的性能瓶颈，识别率为100%，应用的执行效率提高了70%。 |
| 英文摘要: |
| The stream computing engine Storm is widely used in the real-time processing of big data to improve the execution efficiency of data processing. However, due to the coarse granularity of statistical information, the engine cannot accurately locate the performance bottleneck, which makes it a big problem to improve the execution efficiency of data processing. To solve the problem, a fine-grained optimization method for execution efficiency of stream computing is proposed, which focuses on tuples for fine-grained performance analysis, including performance bottleneck identification algorithm and performance bottleneck mitigation algorithm, and supports quantitative selection of optimal parameter configuration to improve execution efficiency. The experimental results show that under 32 different configuration scenarios of three standard programs, the proposed method can accurately identify the performance bottleneck of flow applications, the recognition rate is 100%, and the application execution efficiency can be improved by 70%. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
