add profiling log for livy sessions

[tovganesh]

adding profiling log for:

• opening a livy session
• submitting a SQL and getting results in a livy session

Sample profling log:

12:47:20.104742 [debug] [MainThread]: Spark adapter: Starting livy session
12:47:20.104811 [debug] [MainThread]: Spark adapter: Using HTTP auth
12:47:20.105054 [debug] [MainThread]: Spark adapter: Creating new livy session
12:47:22.108724 [debug] [MainThread]: Spark adapter: Obtained new livy session: 696
12:47:22.109197 [debug] [MainThread]: Spark adapter: 696: Polling to check if session is idle
12:47:23.234932 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:23.235438 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:26.390059 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:26.390702 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:29.574515 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:29.574948 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:32.753653 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:32.754680 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:36.000985 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:36.001457 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:39.153043 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:39.153496 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:42.461602 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:42.462020 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:45.656239 [debug] [MainThread]: Spark adapter: 696: Session state is starting
12:47:45.656524 [debug] [MainThread]: Spark adapter: 696: Sleeping 2 seconds before next poll
12:47:48.895976 [debug] [MainThread]: Spark adapter: 696: Session state is idle
12:47:48.896601 [debug] [MainThread]: Spark adapter: Created new livy session: 696
12:47:48.896782 [debug] [MainThread]: Spark adapter: New Livy session created: 696
12:47:48.896937 [debug] [MainThread]: Spark adapter: 696: Livy session connected
12:47:48.897080 [debug] [MainThread]: Spark adapter: 696: Submitting query: select version()
12:47:50.119863 [debug] [MainThread]: Spark adapter: 696: 0: Awaiting query results
12:47:51.271620 [debug] [MainThread]: Spark adapter: 696: 0: Query status: running
12:47:51.272071 [debug] [MainThread]: Spark adapter: 696: 0: Sleeping 2 seconds to poll for status
12:47:54.568626 [debug] [MainThread]: Spark adapter: 696: 0: Query status: running
12:47:54.569131 [debug] [MainThread]: Spark adapter: 696: 0: Sleeping 2 seconds to poll for status
12:47:57.849903 [debug] [MainThread]: Spark adapter: 696: 0: Query status: running
12:47:57.850233 [debug] [MainThread]: Spark adapter: 696: 0: Sleeping 2 seconds to poll for status
12:48:01.088801 [debug] [MainThread]: Spark adapter: 696: 0: Query status: running
12:48:01.089888 [debug] [MainThread]: Spark adapter: 696: 0: Sleeping 2 seconds to poll for status
12:48:04.501021 [debug] [MainThread]: Spark adapter: 696: 0: Query status: available
12:48:04.501661 [debug] [MainThread]: Spark adapter: 696: 0: Query execution over
12:48:04.501967 [debug] [MainThread]: Spark adapter: SPARK VERSION 3
12:48:04.502697 [debug] [MainThread]: Spark adapter: 696: Submitting query: select 1 as id
12:48:05.696463 [debug] [MainThread]: Spark adapter: 696: 1: Awaiting query results
12:48:06.873680 [debug] [MainThread]: Spark adapter: 696: 1: Query status: available
12:48:06.874227 [debug] [MainThread]: Spark adapter: 696: 1: Query execution over
12:48:06.877520 [debug] [MainThread]: Spark adapter: Connection.close()
12:48:06.877841 [debug] [MainThread]: Spark adapter: Connection.close()
12:48:09.142920 [debug] [MainThread]: Spark adapter: cleanup_connections
12:48:09.143032 [debug] [MainThread]: Spark adapter: Closing the livy session: 696

[TapasSenapati]

Look good. few minor comments.