Drill-On-YARN

1. Introduction to Drill-On-YARN

1. Function
   • start up
   • stop
   • Expansion
   • shrink
   • failover
2. start process
   • Download the drill community package, perform the necessary configuration, execute the drill-on-yarn.sh start command, and start the drill-on-yarn client
   • client
     • Upload the community package of drill to the file system (eg: hdfs)
     • Set the startup AM context, including environment, commands, resources
     • Submit the application, start RM
   • RM requests a container from NM, starts AM, and starts the process
     • Download the community package of drill in the file system (hdfs) to the local and decompress it automatically
     • Execute the set command to start AM
   • AM and RM negotiate resources, when resources are negotiated (ie allocated to containers)
     • Set the context of container startup, including environment, commands, resources, then start the container, start the process
       • Download the community package of drill in the file system (hdfs) to the local and decompress it automatically
       • Execute the command to start drillbit to start drill
3. stop
   • Graceful stop: cancel all tasks (pending, requesting, running) -> stop container -> stop NMClient -> unregisterApplicationMaster -> stop AMRMClient
   • Forced stop: The yarn client directly kills the application yarnClient.killApplication
4. Capacity expansion: re-execute the request container again, when there are no available resources in the cluster, it will keep waiting
5. Shrinking: In the container that has been started, select one at random and execute stop
6. failover: When the started container ends (there are many reasons, such as: drill exits), it will try to re-initiate the container request (there is a limit on the number of retries, and it may not be restarted in place)

2. Start the process

3. Other functions