@ZoranPandovski Hi, I created the issue for updating the Airtable Handler.

Draft PR: #10108

@ZoranPandovski @MinuraPunchihewa Hi, for this issue, I am planning on not only modifying the handler to use the SDK but also transforming the Airtable handler from Database handler to an API (Application) handler because it seems more intuitive for me since it uses restful api calls.

However, before proceeding further, I just want to discuss with you whether you agree on this because I think it is a big change and I don't want to delete/redo existing functionalities that may have some intended purposes.

The pyairtable has formulas which are used to filter and query records. In the current implementation, we are retrieving all records, merge them into pandas.Dataframe and use duckdb to do SQL operations. If we want to use their formulas, then I think API (Application) handler is a better fit but it's more involved to implement.

Final decision

I will just make the handler to use the Python SDK (pyairtable) as intended for this issue and in the future we can think about whether it is worth to use the formulas.

Hey @HahaBill,
Sorry for the late reply, I am in agreement with you that it is probably best to convert this to an API handler. If possible, go ahead and make this update.

Hey @HahaBill, Sorry for the late reply, I am in agreement with you that it is probably best to convert this to an API handler. If possible, go ahead and make this update.

No worries! Alright, I will then convert the handler into API handler.