how to bind pandas datas on map

[vikashsingh009]

• Dremio client version:
• Dremio version:
• Python/Arrow version:
• Operating System:

Description

i have tried below code in jupiter note book to bind data on map

What I Did

here is mine phython code

from pyarrow import flight
import pyarrow as pa
import pydeck as pdk
import json
import pandas as pd

class HttpDremioClientAuthHandler(flight.ClientAuthHandler):

def __init__(self, username, password):
    super(flight.ClientAuthHandler, self).__init__()
    self.basic_auth = flight.BasicAuth(username, password)
    self.token = None

def authenticate(self, outgoing, incoming):
    auth = self.basic_auth.serialize()
    outgoing.write(auth)
    self.token = incoming.read()

def get_token(self):
    return self.token

username = 'visur'
password = 'BizRuntime@123'
sql = '''select * from construction.sample'''

client = flight.FlightClient('grpc+tcp://127.0.1.1:47470')
client.authenticate(HttpDremioClientAuthHandler(username, password))
info = client.get_flight_info(flight.FlightDescriptor.for_command(sql))

reader = client.do_get(info.endpoints[0].ticket)
batches = []
while True:
try:
batch, metadata = reader.read_chunk()
batches.append(batch)
except StopIteration:
break
data = pa.Table.from_batches(batches)
print(data)
df = data.to_pandas()

schema = pa.Schema.from_pandas(df)
print(schema)
layer = pdk.Layer(
'HexagonLayer', # type positional argument is here
df,
get_position=['lng', 'lat'],
auto_highlight=True,
elevation_scale=50,
pickable=True,
elevation_range=[0, 3000],
extruded=True,
coverage=1)

Set the viewport location

view_state = pdk.ViewState(
longitude=-1.415,
latitude=52.2323,
zoom=6,
min_zoom=5,
max_zoom=15,
pitch=40.5,
bearing=-27.36)

Combined all of it and render a viewport

r = pdk.Deck(layers=[layer], initial_view_state=view_state)
r.to_html('hexagon-example.html')

[rymurr]

Hey @vikashsingh009 I am not familiar with pydeck. Is there any flight related errors that you can share?

[vikashsingh009]

@rymurr ,i able to solve issues was like Data schema not on correct format ,now able to use dremio flight and bind read data using pandas..

but i am facing issues on deploying https://github.com/dremio/dremio-cloud-tools/tree/master/charts/dremio

it show unable to identifi cluster,and dremio not start after that,but if i remove flight-connector jar its able to start,did we tested flight conector on dremio cluster

[rymurr]

Hey @vikashsingh009 I am not great with Kubernetes, did you use the new charts? https://github.com/dremio/dremio-cloud-tools/tree/master/charts/dremio_v2