GraphQL Apache Arrow Flight Transport vs ASGI idea

[davlee1972]

So I was thinking about GraphQL schemas over the weekend and how they fit pretty well in the Apache Arrow Column format. The Apache Arrow project has its own RPC protocol called Arrow Flight which uses gRPC + HTTP2 + custom payloads which sends columnar batches of data across the wire.

This could be incorporated into a new pyarrow.graphql file like how asgi is added in asgi.graphql.

I decided to fork the Ariadne Repo and put together some benchmarks using an Arrow GraphQL Server.

This would include:

A python arrow flight server receiving the Query and running graphql_sync to produce a python object (lists of 1k, 100k, 10 million, etc. random ints, floats and strings)
Convert the python object to an Arrow columnar object.
Send the Arrow columnar object back to the client

A python arrow client sending the GraphQL Query as an Arrow Flight Ticket.
Client converts the Arrow columnar object into a standard python dictionary.

I ran both the client and server on my local PC, but I suspect (will test later) that splitting the client and server across a network will really show how Arrow Flight improves transport speed.

Even on my local PC the Arrow Flight results were roughly 33% faster than ASGI.

Here's the code and how to run it:

https://github.com/davlee1972/ariadne_arrow/blob/arrow_flight/benchmark/test_asgi_server.py
Standard ASGI server. Start it up with uvicorn --host=0.0.0.0 test_asgi_server:app

https://github.com/davlee1972/ariadne_arrow/blob/arrow_flight/benchmark/test_arrow_flight_server.py
New Arrow Flight server. Start it up with python test_arrow_flight_server.py

https://github.com/davlee1972/ariadne_arrow/blob/arrow_flight/benchmark/test_asgi_arrow_client.py
Benchmarking script. Pass in length of lists to test and server host.

python test_asgi_arrow_client.py -l 10 -s 127.0.0.1
python test_asgi_arrow_client.py -l 1000 -s 127.0.0.1
python test_asgi_arrow_client.py -l 10000000 -s 127.0.0.1

Arrow flight was faster in all 3 test cases.

10 records it was 0.002 seconds faster
10000 records it was 0.025 seconds faster
10 million records it was 14 seconds faster or 33%.

===============

ASGI Fetch Time for 10 records:0.00400233268737793
ASGI Convert JSON to Dictionary Total Time:0.005000591278076172

Server side data generation time:0.0
Actual ASGI minus server side:0.005000591278076172

Arrow Flight Fetch Time for 10 records:0.0030961036682128906
Arrow Flight Convert Arrow to Dictionary Total Time:0.0030961036682128906

Server side data generation time:0.0
Actual Arrow Flight minus server side:0.0030961036682128906

===============

ASGI Fetch Time for 10000 records:0.14105224609375
ASGI Convert JSON to Dictionary Total Time:0.14405202865600586

Server side data generation time:0.05375194549560547
Actual ASGI minus server side:0.09030008316040039

Arrow Flight Fetch Time for 10000 records:0.11061930656433105
Arrow Flight Convert Arrow to Dictionary Total Time:0.11061930656433105

Server side data generation time:0.04554128646850586
Actual Arrow Flight minus server side:0.0650780200958252

===============

ASGI Fetch Time for 10000000 records:73.41773104667664
ASGI Convert JSON to Dictionary Total Time:76.54971837997437

Server side data generation time:32.58742570877075
Actual ASGI minus server side:43.96229267120361

Arrow Flight Fetch Time for 10000000 records:60.792750120162964
Arrow Flight Convert Arrow to Dictionary Total Time:60.792750120162964

Server side data generation time:31.123136520385742
Actual Arrow Flight minus server side:29.66961359977722

[davlee1972]

Here are the test results with the client script running on a laptop over home WIFI to the same server on a windows desktop..

Over the network Arrow really improved transport speed which was what I expected, but performance should be scaling up as data size increases. I need to talk to the Arrow folks. I suspect that list structures are not being serialized into vectors.

10 records it was 0.049 seconds faster or 80% faster
10000 records it was 0.109 seconds faster or 66% faster
10 million records it was 54 seconds faster or 66% faster.

Also here is the data structure that is sent across the wire..

pyarrow.Table
data: struct<test_lists: struct<float_list: list<item: double>, int_list: list<item: int64>, length: int64, string_list: list<item: string>, time_spent: double>>
child 0, test_lists: struct<float_list: list<item: double>, int_list: list<item: int64>, length: int64, string_list: list<item: string>, time_spent: double>
child 0, float_list: list<item: double>
child 0, item: double
child 1, int_list: list<item: int64>
child 0, item: int64
child 2, length: int64
child 3, string_list: list<item: string>
child 0, item: string
child 4, time_spent: double

data: [
-- is_valid: all not null
-- child 0 type: struct<float_list: list<item: double>, int_list: list<item: int64>, length: int64, string_list: list<item: string>, time_spent: double>
-- is_valid: all not null
-- child 0 type: list<item: double>
[[13.500371672273381,17.747395152140353,28.973205439157457,1.361443415643098,19.029191125636135,14.62284718057391,18.44333922481529,7.906278860251386,14.402464768126993,5.826040531772251]]
-- child 1 type: list<item: int64>
[[23,3,21,15,20,4,10,16,23,25]]
-- child 2 type: int64
[10]
-- child 3 type: list<item: string>
[["qypsupwtxy","vrxptpspyt","qpvruwsuqq","ywwpyxrvrt","wswutpxxqv","tsyypstxvv","ytprpqsxsx","wtwsxvprvu","suwtrvqvwp","wtsrwywwty"]]
-- child 4 type: double
[0]]

===============

ASGI Fetch Time for 10 records:0.05800437927246094
ASGI Convert JSON to Dictionary Total Time:0.059921979904174805

Server side data generation time:0.0
Actual ASGI minus server side:0.059921979904174805

Arrow Flight Fetch Time for 10 records:0.011480569839477539
Arrow Flight Convert Arrow to Dictionary Total Time:0.011480569839477539

Server side data generation time:0.0
Actual Arrow Flight minus server side:0.011480569839477539

===============

ASGI Fetch Time for 10000 records:0.17502927780151367
ASGI Convert JSON to Dictionary Total Time:0.18400907516479492

Server side data generation time:0.03500080108642578
Actual ASGI minus server side:0.14900827407836914

Arrow Flight Fetch Time for 10000 records:0.07013273239135742
Arrow Flight Convert Arrow to Dictionary Total Time:0.07013273239135742

Server side data generation time:0.03199934959411621
Actual Arrow Flight minus server side:0.03813338279724121

===============

ASGI Fetch Time for 10000000 records:100.53821158409119
ASGI Convert JSON to Dictionary Total Time:113.22756385803223

Server side data generation time:30.61162829399109
Actual ASGI minus server side:82.61593556404114

Arrow Flight Fetch Time for 10000000 records:59.43856120109558
Arrow Flight Convert Arrow to Dictionary Total Time:59.43856120109558

Server side data generation time:30.664714813232422
Actual Arrow Flight minus server side:28.77384638786316

[davlee1972]

In an ideal scenario a forked version of GraphQLPlayground would also communicate with a GraphQL server using Arrow flight and this REST style JSON response could be abandoned.

Arrow flight clients can be written in Python, Java, Javascript, Ruby, etc..

Streamlit recently switched to using Arrow to move data from Python environments into Javascript web components.
https://blog.streamlit.io/content/images/2021/07/legacy-vs-arrow-2-1.png#shadow
https://blog.streamlit.io/all-in-on-apache-arrow/

[davlee1972]

Here's a similar graphQL implementation using Go and Arrow Flight.

GraphQL and Apache Arrow: A Match Made in Data
https://www.youtube.com/watch?v=5N97TzY_tis