Chef generate cookbook creates tests in a different location than where kitchen-inspec looks for tests

[ricardolupo]

Issue Description

The default cookbook generators in chef-dk do not create the necessary scaffolding to work with kitchen-inspec.

By default, kitchen-inspec looks for tests in the cookbooks in these locations, in order of preference:

• ./test/recipes/default/
• ./test/integration/default/inspec/

However, chef generate cookbook lays down tests in /test/recipes/, without the default subdirectory.

Current behavior

In order to run Inspec tests, people must either:

1. manually move the files to the proper directories required to work with kitchen-inspec;
2. manually edit the .kitchen.yml file to point to the default created tests in ./test/recipes/default_test.rb

Desired Behavior

• Inspec tests can be run on a freshly generated cookbook without modification.
• Error message when no tests are found by kitchen-inspec is clarified, so that it is obviously referring to a path, and not a filename.

ChefDK Version

Chef Development Kit Version: 0.16.28

Platform Version

Mac OS/X latest

Replication Case

• chef generate cookbook foo
• cd foo
• delivery local smoke
• Note the output:

-----> Setting up <default-ubuntu-1604>...
       Finished setting up <default-ubuntu-1604> (0m0.00s).
-----> Verifying <default-ubuntu-1604>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-ubuntu-1604> (0m0.28s).
...
-----> Setting up <default-centos-72>...
       Finished setting up <default-centos-72> (0m0.00s).
-----> Verifying <default-centos-72>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-72> (0m0.26s).

• mkdir test/recipes/default && mv test/recipes/default_test.rb test/recipes/default/default_test.rb
• delivery local smoke
• Note the output:

Chef Delivery
Running Smoke Phase
-----> Starting Kitchen (v1.10.2)
-----> Verifying <default-ubuntu-1604>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing

Target:  ssh://[email protected]:2222

  ○  1 skipped
     This is an example test, replace with your own test.
  ○  1 skipped
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-ubuntu-1604> (0m0.32s).
-----> Verifying <default-centos-72>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing

Target:  ssh://[email protected]:2200

  ○  1 skipped
     This is an example test, replace with your own test.
  ○  1 skipped
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-centos-72> (0m0.29s).
-----> Kitchen is finished. (0m1.67s)

[charlesjohnson]

Edit: Initially I closed this ticket, based on a false reading. After talking with @ricardolupo I've revised the description and initial report. Thanks Lupo!

[ricardolupo]

The challenge here is that kitchen-inspec is actually looking for tests in the ./test/recipes/default directory. chef generate cookbook foo creates a sample test in ./test/recipes, which still leaves the user to create the default subdirectory. As a user, I expect the chef command to create all the scaffolding necessary for my cookbook.

Test case:

09:51:46-rlupo~/cookbooks/dev/foo (master)$ kitchen verify
-----> Starting Kitchen (v1.10.2)
-----> Setting up <default-centos-72>...
$$$$$$ Running legacy setup for 'Docker' Driver
       Finished setting up <default-centos-72> (0m0.00s).
-----> Verifying <default-centos-72>...
       Use `/Users/rlupo/cookbooks/dev/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-72> (0m0.73s).
-----> Kitchen is finished. (0m2.65s)
09:51:57-rlupo~/cookbooks/dev/foo (master)$ mkdir /Users/rlupo/cookbooks/dev/foo/test/recipes/default
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default
default/         default_test.rb
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default
default/         default_test.rb
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default_test.rb /Users/rlupo/cookbooks/dev/foo/test/recipes/default/default_test.rb
/Users/rlupo/cookbooks/dev/foo/test/recipes/default_test.rb -> /Users/rlupo/cookbooks/dev/foo/test/recipes/default/default_test.rb
09:52:44-rlupo~/cookbooks/dev/foo (master)$ kitchen verify
-----> Starting Kitchen (v1.10.2)
-----> Verifying <default-centos-72>...
       Use `/Users/rlupo/cookbooks/dev/foo/test/recipes/default` for testing

Target:  ssh://[email protected]:32778

  ○  User root should exist; User root This is an example test, r... (1 skipped)
     This is an example test, replace with your own test.
  ○  Port 80 should not be listening; Port 80 This is an example ... (1 skipped)
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-centos-72> (0m0.81s).
-----> Kitchen is finished. (0m2.32s)

[charlesjohnson]

Updated the title and description of this issue to be more descriptive. Thanks for reporting this, @ricardolupo!

[ricardolupo]

add trailing / to ensure no confusion that default is a directory

15:19:42-rlupo~/cookbooks/dev/foo (master)$ kitchen verify
-----> Starting Kitchen (v1.10.2)
-----> Verifying ...
Use /Users/rlupo/cookbooks/dev/foo/test/recipes/default for testing

Target: ssh://[email protected]:32778

[qubitrenegade]

Another side effect is #1027 which causes foodcritic to fail.

I think it makes the most sense for chef generate cookbook to put the inspec files in /test/integration/default/inspec/ as this will keep with the current structure allowing us to have different suites and will allow for easier transition from ServerSpec to InSpec.

[cheeseplus]

Now the generator explicitly generates an inspec_tests key with a location and we've reverted the changes that made this confusing in the first place, closing.