OpenTrails.js

[alanjosephwilliams]

Description

A javascript module for converting a feed of OpenTrails files (GeoJSON and CSV) into pure JSON, for use in client-side focused web and mobile apps, like To The Trails.

cc @jmoe @rclosner

[lovehandle]

I'm still of the opinion that the job of converting between widely supported formats is the role of the API endpoint. If, for example, the service had 5 consuming applications -- why would you force each individual application to perform the same conversion process between CSV, JSON, XML, etc?

In terms of this module -- CSV to JSON converters exist, and having one that specifically returns Trail data would add little value, IMO.

If we're not able to get alternate formats from the API, I say we just integrate the conversion process into the Trails mobile source code until the endpoint can return formats other than CSV.

[alanjosephwilliams]

@rclosner that seems like a reasonable perspective.

I suppose the question is to whether a module that converts OpenTrails (GeoJSON/CSV) into a single, generic JSON blob will results in data ready to be consumed by multiple applications.

In other words, to you think that apps like Trails Mobile are using OpenTrails in a generic way, or an idiosyncratic way, or somewhere inbetween?

[lovehandle]

I suppose the question is to whether a module that converts OpenTrails (GeoJSON/CSV) into a single, generic JSON blob will results in data ready to be consumed by multiple applications.

It will not, IMO. An API is a wrapper around a datastore. One of the fundamental roles of an API is to provide translation from a datastore into formats that applications can consume. Assuming the datastore is an aggregate of CSV files, the API is only providing half of its value -- unified access to a repository of data.

In other words, to you think that apps like Trails Mobile are using OpenTrails in a generic way, or an idiosyncratic way, or somewhere inbetween?

It's a good question, and I'll defer to you on that. I was operating under the assumption that the API will be a canonical source of Trail data. In which case it's responsible for providing access to the data in a variety of formats. In the case that it's not, and it's literally just a one off that's backing our mobile trails application then it will be less important for it to provide data in alternate formats (JSON, XML, and CSV to name a few).

One of the niceties in Rails is that it makes it really easy to transform data from a datastore into a variety of formats:

def index
  @trails = Trail.all
  respond_to do |format|
    format.csv { render text: @trails.to_csv }
    format.js { render json: @trails.to_json }
  end
end

Assuming that the datastore has been persisted into a database, we're literally talking about adding an extra line to the API as opposed to writing a 50-100 line module.

At any rate, (since we're not going to get JSON from the API in this release) for now let's skip the module and just add a post processor on the request from the app that transforms the API responses from CSV to JSON.

@jmoe, @alanjosephwilliams -- sound reasonable?

[jmoe]

Ryan makes good points about API's in a general sense. The mechanics of serving up different formats in Rails is very convenient yes, however there are lot of nuances to designing and supporting an API long term. We know this first hand with Transit & Trails and so we are being very deliberate about what we make available publicly and when. Unfortunately our timelines did not sync up perfectly in this case.

We most definitely intend to provide a full REST API to our backend. No doubt about that and it will serve the purpose Ryan describes. However, one issue is that To the Trails is just one of many applications that are already committed to support. Another issue is that our data model is in many ways for rich than the OpenTrails spec and in some ways presents the relationships a little differently. For that reason the OpenTrails data model cannot be our public API. So the thinking was that we would simply serve up the OpenTrails compliant data and let OpenTrails compliant apps do the lifting on parsing and translating for now. Since this is an open standard, that code to do that parsing from the component files to a internal data structure would be useful to anybody building an outdoor application.

To Ryan's point though, it would be very simple for me to standup JSON formats for the CSV files. I had not actually considered that, I was thinking more in terms of a fully functional CRUD API.

I do think a module, that give OpenTrails documents, would parse and provide a simple API for accessing trail objects would be very useful to the community.

So, if you guys think it will be helpful and save time I am willing to stand up steward.json, named_trails.json endpoints. Is that all we are talking about here? It will still be the responsibility of the app to build the relationships between the objects internally. Make sense?

Thanks for your support on this Ryan, hope things are going well out there!

[jmoe]

Hey folks,

opentrails/stewards.json and opentrails/named_trails.json are now availabe on production.

Hope that helps!