Skip to content

Editing Recording and Re recording API calls with VCR (WIP)

Jump to bottom
James Miller edited this page May 23, 2022 · 8 revisions

search-gov uses the VCR gem to record and play back API and other web responses in tests.

Configuration

VCR is configured in:

  • spec/support/shared_vcr_setup.rb - shared config for RSpec & cucumber
  • spec/support/vcr_setup.rb - RSpec-specific config
  • features/support/vcr_setup.rb - cucumber-specific config

Recording new cassettes

When running specs locally, if you see new files pop up in spec/vcr_cassettes/, or features/vcr_cassettes/, that indicates that we’ve made and recorded requests. At that point, you should either (in order of preference):

  • tweak the spec/feature to avoid making unnecessary requests
  • stub the request with Webmock
  • commit the cassettes so that VCR can stub the response in future test runs

Re-recording cassettes

Basic Usage

Currently, cassettes are not automatically re-recorded. If necessary, you can update them using the following process:

Remove some or all of the existing cassettes:

$ rm -rf spec/vcr_cassettes features/vcr_cassettes

Checkout the main branch of I14y and run the server in test mode (required for the cucumber tests that hit the i14y server):
$ RAILS_ENV=test rails s -p 8081

Then, switch to usasearch and create a new branch from main:
$ git checkout -b cassettes/{date_today}

Then run the tests:
$ rake

When tests are done running, you should see all the yml files in spec/vcr_cassettes/ and /features/vcr_cassettes/ modified:

$ git status
modified:   spec/vcr_cassettes/api_bing_docs_search_as_json.yml
modified:   spec/vcr_cassettes/api_bing_docs_search_as_json/with_advanced_query_operators.yml
modified:   spec/vcr_cassettes/api_bing_docs_search_run/when_enable_highlighting_is_disabled.yml
...etc

If your tests have passed, you can commit and push your updated cassettes.

If your tests have NOT passed, refer to the Gotchas/Debugging sections below.

Gotchas

Many of the old specs were written expecting search results to meet certain criteria. While those criteria were reasonable, there may be some unexpected failures. For example, many tests expect something along the lines of "when I search for 'foo', the title of the search first result should match /foo/"...which is usually, but not alway, the case. We have slowly been updating the specs to be less specific and more flexible. Some examples of changes we've made:

BAD (too specific):

result["FileSize"].should eq 500

GOOD (flexible):

result["FileSize"].should be_an Integer

BAD (expectation set on one search result):

expect(normalized_response.results.first.title).to match(/\xEE\x80\x80White House\xEE\x80\x81/)

GOOD (expectation set on at least one of the search results):

expect(normalized_response.results.map(&:title).join).to match(/\xEE\x80\x80White House\xEE\x80\x81/)

Debugging

To see additional info on when/why VCR is recording (or not recording) cassettes, you can uncomment the debugging line in spec/support/vcr_setup.rb or features/support/vcr_setup.rb:

config.debug_logger = STDOUT
Clone this wiki locally