How does serverless deal with the resource supply demand of k8s in the offline scenario

itread01 2021-01-14 14:11:12
serverless deal resource supply demand

This paper is compiled from ** Han Pei, expert product manager of Tencent cloud native product team ** stay Techo The sharing content of the cloud native theme of the developer conference ——Kubernetes Mixing part and elastic container . This sharing is mainly divided into three parts :** Based on K8s The application mix of 、 The key to improve the effect of application mixing 、 The value of elastic container to mixed part clustering .**![]( Discussion K8s The mixed part of this topic , Because we found that , In business K8s After melting , Mixed department and resource utilization is an unavoidable topic for the operation and maintenance team , This topic has always been the focus of our Tencent cloud native team . First , Beyond all doubt ,Kubernetes Its system capability and its influence as an engine to promote cloud native ecology are very powerful , It helps a lot of advanced ideas in the production environment of large-scale practical , Including micro Services 、 Automatic expansion and contraction 、CICD、 Service Grid 、 Application of mixing part, etc . Some of this is in the existing K8s You can get very good support in the system , For example, microservices 、 Automatic expansion and contraction . Some rely on K8s Integration with ecological capacity , such as CICD、 Service Grid , Just rely on it K8s And some communities DevOps 、servicemesh Get through the system , But most of them are well integrated in the ecosystem , Usually we don't have to do too much work . But our topic today ——**K8s The application mix under the architecture **, It's a special field , On the one hand, most of the enterprise infrastructure is upgraded to cloud native architecture , Usually, it will naturally support some mixed abilities , And bring some obvious benefits , For example, the improvement of resource utilization . Containerization and K8s It opens a window for the whole industry to enter the large-scale mixed application department . And on the other hand , But when we really get into this field , Even standing on K8s On my shoulder , Mixed division is still a great challenge to enterprise architecture capability .![]( Before containerization , Deploy applications on physical or virtual servers , Resource utilization is usually very low , One is that many applications have tidal phenomena , Second, most servers can only deploy one application , Instead of K8s In that way, multiple nodes are deployed on one node . But containerization is managed to K8s After cluster , Many times , We will find that the utilization rate of resources is still not high . Above , It's a K8s Typical resource curve of clustering online business , The top blue line is container resources request Application value , The red line is the actual execution curve value of the container , We see request and usage There is a big gap between gap, This is because the evaluation of container resources cannot be completely accurate , in addition , There's also a difference between peaks and troughs , Eventually, the average utilization rate is not high .![]( Is that K8s No way ? Of course not ,K8s Although we haven't solved all the problems in helping us carry out the application mix , But it's definitely one of the best alternative platforms . Excellent system capability makes K8s Naturally, it is suitable for mixing , Including the online service hybrid department and the hot offline hybrid department in the industry . Tencent has also adopted K8s turn , In many scenarios, it significantly improves resource utilization . A computing volume of the scale of Tencent , In addition to being familiar with star apps , There is also a huge amount of computing power in the service support 、 Offline computing, etc . By mixing these applications and some products and services with obvious tidal phenomena , The improvement of resource utilization rate is very significant .![]( In the industry ,Google Based on K8s The forerunner of Borg System , From 2015 So far, a number of papers related to the mixed department have been published . In a paper released last year , You can see Borg The ability of the support is approaching 60% Of CPU Resource utilization . Compare it with 2011 Years and 2019 The mixed effect of two years , You can see , In addition to the increase in utilization , The most significant change , First, the granularity of application classification is finer , Second, application execution at all levels is more stable . Especially the second point , Obviously feel Borg At the support level of the mixing section , Such as scheduling enhancement 、 Resource forecasting and recycling 、 Progress in mechanisms such as task avoidance .![]( What is the key to improve the mixing effect ? First , We need to be clear about two issues .** The first question , What's the purpose of the Ministry ?** The purpose of the hybrid department is to ensure the quality of online business services , Realize the reuse of idle resources , Improve the overall utilization of resources . Ensuring the service quality of online business is the premise , Make it unaffected , Without that premise , There's no point in increasing utilization .** The second question , What kind of application is suitable for mixed parts ?** There are two types of applications that are suitable for mixing : One is the periodic application with high computational requirements , Usually offline computing tasks . One is the application that is easy to waste resources , It's usually something that takes a long time to execute 、 Online services with tidal phenomena . But should pay attention to , Some online services are highly sensitive to certain resources , This kind of service is the biggest challenge to the hybrid system , Because a little carelessness will deviate from the purpose of the Ministry , It affects the quality of online service , The gain is not worth the loss .![]( After identifying these two issues , Let's take a look at the mechanisms that the hybrid system needs to have . It's usually divided into three layers : One is to make feature portraits of mixed applications 、 The application management layer for grading and allocating resource quotas . This layer defines the level of application , The timing of the Ministry , And through quotas to ensure that the allocation of resources is not out of control . The second is to schedule mixed cluster 、 Isolation 、 The core system of resource reuse and avoidance . This layer is the core of the mixed department , It basically determines what kind of mixed part effect our clustering can achieve . Finally , We also need a set of adaptive automatic operation system .![]( The basic principle of hybrid division is the redistribution of idle resources . Usually , There are two sources of idle resources . There will be fragmented resources in the cluster , This is caused by the granularity of resource allocation , These fragmented resources can be allocated to smaller granular applications . in addition , The resources of online service application are usually larger than the resources actually used , With the application of portrait , Predict the peaks and troughs of these services , These idle resources can be allocated to other applications . Based on this background , The two core sub modules of the hybrid part are extended :** Resource reuse and task avoidance **. As the name suggests , Resource reuse is to forecast the above two kinds of idle resources 、 The recycling mechanism , Real time Reallocation for mixed application use . And mission avoidance , It is to detect whether the core online service has been affected by the mixed department . In the event of interference , Trigger the conflict handling mechanism immediately , Press and reschedule .![]( You can say that , These two modules determine the effect of the mixing part and the application scope of the mixing part . Except for the theoretical problems , There are also some important points to consider : In order to ensure the mixing effect , Frequent cluster real-time situation prediction and resource recovery , It puts an extra burden on clustering itself , How to find a balance between resource reuse as much as possible and resource recovery frequency as low as possible ? And , In order to ensure the quality of online service , Task avoidance is often unavoidable , This reduces the execution efficiency of secondary priority applications , High load may lead to frequent retries and heaps of tasks , And it could drag down the entire cluster .![]( To solve these problems , Tencent cloud native team has been thinking and trying , At present, a more advanced way is through serverless A container is an elastic container , To expand the resource pool of the whole mixed department . Elastic container is a server free container product launched by Tencent cloud . It supports a capability , Similar to open source virtual kubelet The way , But it's more powerful than open source solutions 、 More suitable for production . It supports in an existing K8s In cluster, virtual nodes are deployed pod The schedule is elastic container . The schedule is flexible pod Compared with the others in the original bundle set pod Network intercommunication , If it's connected service ,service They can also communicate with each other . So whether it's existing workload The expansion of 、 It's still new workload, Can be scheduled in a smooth way . And this capability does not generate additional maintenance costs for clustering .![]( The core value of this ability to the Ministry lies in : It expands the cluster resource pool without cost , Reducing the risk of resource conflict , The redundancy and applicability of mixed cluster are improved . in addition , When conflicts such as insufficient resources are detected , In many scenarios, you don't need to abort the secondary priority task , It's capacity expansion or rescheduling depending on the situation , Continue the task on the elastic container , Adhere to the principle of not interrupting started tasks as much as possible , Improve the efficiency of the whole system .![](** Several typical scenes of this kind of Mixed Cluster :**1、 Fill in the task when the load is low , Do more , Improve cluster resource utilization .2、 In case of online service disruption , Block related nodes , Expel secondary priority tasks to virtual nodes , Let it continue .3、 When cluster resources are tight , Block related nodes , Depending on the situation , If it's a compressed resource crunch , such as CPU、IO etc. , Then suppress the second priority task ; If it's incompressible, resources are tight , Like memory 、 Storage, etc , Then the secondary priority task is expelled to the virtual node ; In this case, all new Pod Schedule to virtual nodes , No more pressure on clustered fixed resources , Avoid avalanches . This 3 One example can't cover all mixed scenes yet , But it's been upgraded K8s The application scope of cluster mixing part . We are also continuing to explore other ways to do better . Also welcome friends with ideas to discuss and share . Finally , Mixing is a process of continuous optimization . All the major factories have invested a considerable amount of time in the research of the mixing department , Just started to spread out in large quantities . With the development of Technology ,K8s The mixing effect will be better and better , There will be more and more applicable scenarios . Thank you !Kubernetes Mixing part and elastic container ( this paper ) PPT Download method , Please reply the keyword in the original background of Tencent cloud on the public account “**EKS**” Get .>【 Tencent cloud native 】 Cloud said new products 、 Cloud research new technology 、 Travel new life 、 Cloud View information , Scan the code and pay attention to the public account of the same name , Get more dry goods in time !!![](

  1. Spring boot static resource configuration principle (step by step source analysis, detailed and easy to understand)
  2. 400万Docker镜像中,51%的镜像存在高危漏洞
  3. Head first design pattern -- 10. Iterator pattern
  4. A few pictures, take down the HTTPS
  5. Simple use of pyecharts module
  6. [azure redis cache] discussion on the functionality of azure redis
  7. Installation of SVN under Linux
  8. Sorting out knowledge points of MySQL Cluster
  9. rocketmq-cpp-client Visual Studio 2019 编译
  10. rocketmq-cpp-client Visual Studio 2019 编译
  11. RBAC authorization mode of k8s
  12. Remember to use it once Asp.Net The development process of core webapi 5.0 + dapper + MySQL + redis + docker
  13. Java Concurrent Programming points
  14. Explain Java I / O flow in detail
  15. Linux system builds springboot project environment and deploys it
  16. Easy to understand JS object-oriented, by the way understand prototype and__ proto__
  17. Summary of java basic knowledge
  18. . net cloud native architect training camp (module 2 basic consolidation rabbitmq mastransit detailed explanation) - learning notes
  19. The architecture of MySQL
  20. MySQL security management, database maintenance and performance improvement
  21. Redis basic command
  22. Summary of MySQL articles
  23. 2、 Create k8s cluster in 5 seconds
  24. data自定义属性在jQuery中的用法
  25. Linux常见解压缩
  26. Detailed explanation of HBase basic principle
  27. 1、 Why and how to learn k8s
  28. Java advanced (29) -- HashMap set
  29. java中大文件上传
  30. Weblogic 2017-3248 analysis of Java Security
  31. Kubernetes official java client 8: fluent style
  32. Explain the function of thread pool and how to use it in Java
  33. Programming software tutorial video Encyclopedia: C + + / Java / Python / assembly / easy language (with tutorial)
  34. Description of dependency problem after javacv is updated to 1.5. X and how to reduce the size of dependency package
  35. Java reflection & dynamic agent
  36. Building Apache 2.4 + php7 + mysql8 in centos7 environment
  37. Summary of Java multithreading (1)
  38. Oracle AWR report generation
  39. Four magic functions of mybatis, don't step on the pit!
  40. A 16-year-old high school student successfully transplanted Linux to iPhone and posted a detailed guide
  41. Centos7 one click installation of JDK1.8 shell script
  42. Mounting of file system in Linux (centos7)
  43. How does serverless deal with the resource supply demand of k8s in the offline scenario
  44. Detailed explanation of HBase basic principle
  45. Spring security oauth2.0 authentication and authorization 4: distributed system authentication and authorization
  46. Redis performance Part 5 redis buffer
  47. JavaScript this keyword
  48. Summary of Java multithreading (3)
  49. Sentry(v20.12.1) K8S 云原生架构探索, SENTRY FOR JAVASCRIPT 手动捕获事件基本用法
  50. Sentry(v20.12.1) K8S 云原生架构探索, SENTRY FOR JAVASCRIPT 手动捕获事件基本用法
  51. (10) Spring from the beginning to the end
  52. Summary of Java multithreading (2)
  53. Spring source notes! From the introduction to the source code, let you really understand the source code
  54. A stormy sunny day
  55. Zookeeper (curator), the implementation of distributed lock
  56. Show the sky! Tencent T4's core Java Dictionary (framework + principle + Notes + map)
  57. Spring boot project, how to gracefully replace the blank value in the interface parameter with null value?
  58. Spring boot project, how to gracefully replace the blank value in the interface parameter with null value?
  59. docker+mysql集群+读写分离+mycat管理+垂直分库+负载均衡
  60. docker+mysql集群+读写分离+mycat管理+垂直分库+负载均衡