Cv2 3435 alegre with presto for audio

[DGaffney]

Description

This ticket integrates alegre's presto usage into Check-API for dealing with callbacks from storage requests on alegre. As stated in the ticket, I think we can actually get away with a pretty narrow implementation:

At the point that the webhook for storing data is executed, we don’t actually need to make a callback-based search query - we can use the search functionality nearly exactly as written today. When a new item is added in Check-API, we send a request to Alegre to store it, which then executes a callback against presto. Alegre receives the response from Presto, and at that point, we send a callback in turn from Alegre back to Check-API. At that point, when we receive the callback on Check-API from Alegre, we know that the data is now properly stored within Alegre - so we can just synchronously send a request to Alegre to do the lookup as it exists today. In fact, we should probably…. entirely rollback the portion of changes on Alegre for the GET requests (e.g. search-side) for now. The overwhelmingly common flow is for Alegre GETs to be used when we already know that the item is now safely stored, so in those rare cases where we aren’t sure, we should route through the new webhook setup instead of making everything be doubly-called-back. Make sense?

References: 3435

How has this been tested?

I've updated and fixed the tests on alegre to assert an expected message packet to be sent to alegre, and I'm mapping that directly to the added webhook, and then patching directly into existing functionality. Should be pretty minor if any difference downstream of that.

Things to pay attention to during code review

We will need to extensively test full service-to-service integrations just because this is involving so many different repos....

Checklist

• I have performed a self-review of my own code
• I have added unit and feature tests, if the PR implements a new feature or otherwise would benefit from additional testing
• I have added regression tests, if the PR fixes a bug
• I have added logging, exception reporting, and custom tracing with any additional information required for debugging
• I have commented my code in hard-to-understand areas, if any
• I have made needed changes to the README
• My changes generate no new warnings
• If I added a third party module, I included a rationale for doing so and followed our current guidelines

[computermacgyver]

There are functions such as Bot::Alegre.get_items_with_similar_media that expect Alegre to return similar results based on the a media file specified as a URL. My reading of meedan/alegre#340 is that this no longer works unless the audio has been stored already in Alegre with that URL. This assumption does not hold when this function is called from Smooch Search (https://github.com/meedan/check-api/blob/c983b834a859c4052553b8cf102e13f79ba78577/app/models/concerns/smooch_search.rb#L167C5-L167C5) and likely also when it is called from Check Web search.

Currently Alegre will temporarily hash and store an Audio file to perform the search, but this will no longer be the case after meedan/alegre#340 is merged. We therefore need to identify such instances and either (a) create exceptions for audio to fail gracefully or (b) solve the more general case of how we'll use callbacks for tipline/check-web search or (c) ensure that Alegre already fails gracefully in such cases but also logs and ideally alerts the calling service.

[caiosba]

I'm going to review this once I'm able to run it locally, in order to make some tests.

[computermacgyver]

Looking great

[caiosba]

Thanks Devin! We're almost there! I added a couple of requests about the tests, but other than that, please also do:

• Rebase this branch against latest develop
• Add alegre_token: dev to config/config.yml.example

[computermacgyver]

Great work.

[codeclimate]

Code Climate has analyzed commit 60ad614 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (100% is the threshold).

This pull request will bring the total coverage in the repository to 100.0% (0.2% change).

View more on Code Climate.

[caiosba]

I pushed a few cosmetic changes, simplified one test and added one more test. Good to go after Alegre is merged!