Test using capture.hsh fails

[chrisportela]

Seems like it's trying to use .stdout an old way, where the website now shows examples with result.context.stdout?

hush/src/runtime/tests/data/positive/capture.hsh

Lines 6 to 7 in 7fa2dc4

	std.assert(result.stdout == "123\n456\n")
	std.assert(result.stderr == "")

This failed when I was trying to build master as a nix package and it automatically ran cargo test. I have disabled the checks for now.

[caioraposo]

result.context is available if there was an error inside the capture block. The website example ensures result's type is error before accessing the context field:

std.assert(std.type(result) == "error")
std.assert(result.context.stdout == "Hello world!\n")
...

[chrisportela]

For sure, I am guessing what the solution is which is why I didn't make a PR to "fix" the problem.

That said something does seem to be wrong because I had a failure only for that test.

("0.1.5a" is just my way of saying newer than 0.1.4)

error: builder for '/nix/store/4mw43qfh6x2qphix55n9p0xfzzccpchp-hush-0.1.5a.drv' failed with exit code 101;
       last 10 log lines:
       > thread 'runtime::tests::test_positive' panicked at 'Panic in /private/tmp/nix-build-hush-0.1.5a.drv-0/source/src/runtime/tests/data/positive/capture.hsh (line 6, column 18): index ("stdout") out of bounds', src/runtime/tests/mod.rs:82:35
       > note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
       >
       >
       > failures:
       >     runtime::tests::test_positive
       >
       > test result: FAILED(B. 16 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 15.31s
       >
       > error: test failed, to rerun pass '--bin hush'
       For full logs, run 'nix log /nix/store/4mw43qfh6x2qphix55n9p0xfzzccpchp-hush-0.1.5a.drv'.

[caioraposo]

That's strange, I can't reproduce it from the current master. I am missing something?

[gahag]

I'm also not able to reproduce the issue on my linux box, but we can see some useful info from the error output above. In particular:

src/runtime/tests/data/positive/capture.hsh (line 6, column 18): index ("stdout") out of bounds'

Can you please run the following and tell us the output?

echo 'std.print(${ echo 321 | rev | cat; echo 456 })' | cargo run

I get the following (expected) result:

    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/hush`
@[ "stdout": "123\n456\n", "stderr": "" ]

[gahag]

Chances are your machine does not have cat, rev or echo. That would cause the test to fail.

[chrisportela]

Since this is a nix package I assumed it was very possible one of the binaries was not in the path when building, but running your command returns the expected result and running cargo test ... fails as reported earlier.

Running echo 'std.print(${ echo 321 | rev | cat; echo 456 })' | cargo run

   Compiling hush v0.1.4 (/nix/store/rkz54y039ngi19whvbn8hkk58sf5k4q2-source)
    Finished dev [unoptimized + debuginfo] target(s) in 14.47s
     Running `/tmp/nix-shell.mZYGUg/tmp.bE2ZkayaHc/debug/hush`
@[ "stdout": "123\n456\n", "stderr": "" ]

And running the tests

M1M-CPortela:rkz54y039ngi19whvbn8hkk58sf5k4q2-source chris.portela$ RUST_BACKTRACE=full cargo test --release --target-dir $TARGET_DIR
    Finished release [optimized] target(s) in 0.04s
     Running unittests src/main.rs (/tmp/nix-shell.mZYGUg/tmp.bE2ZkayaHc/release/deps/hush-ff353bacd5821acf)

running 17 tests
test syntax::lexer::tests::test_byte_literals ... ok
test syntax::lexer::tests::test_command_block ... ok
test syntax::lexer::tests::test_expansions ... ok
test syntax::lexer::tests::test_invalid_tokens ... ok
test syntax::lexer::tests::test_number_literals ... ok
test syntax::lexer::tests::test_simple_function ... ok
test syntax::lexer::tests::test_token_kind_size ... ok
test syntax::lexer::tests::test_string_literals ... ok
test semantic::tests::test_positive ... ok
test runtime::tests::test_asserts ... ok
test semantic::tests::test_examples ... ok
test semantic::tests::test_negative ... ok
test syntax::tests::test_examples ... ok
test runtime::tests::test_negative ... ok
test syntax::tests::test_positive ... ok
test syntax::tests::test_negative ... ok
test runtime::tests::test_positive ... FAILED

failures:

---- runtime::tests::test_positive stdout ----
thread 'runtime::tests::test_positive' panicked at 'Panic in /nix/store/rkz54y039ngi19whvbn8hkk58sf5k4q2-source/src/runtime/tests/data/positive/print.hsh (line 76, column 42): index ("stdout") out of bounds', src/runtime/tests/mod.rs:82:35
stack backtrace:
   0:        0x100879806 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h898273157692f900
   1:        0x10078d9fa - core::fmt::write::ha637189176d33ea0
   2:        0x100884bbc - std::io::Write::write_fmt::he7370ddd9ebfe147
   3:        0x100894f0d - std::panicking::default_hook::{{closure}}::ha72aeb45233f96e3
   4:        0x100894b76 - std::panicking::default_hook::h99919d760c06bd6a
   5:        0x1008958c0 - std::panicking::rust_panic_with_hook::h5652bc681b06f370
   6:        0x100879fc4 - std::panicking::begin_panic_handler::{{closure}}::h80a37f33d1cc831e
   7:        0x100879f39 - std::sys_common::backtrace::__rust_end_short_backtrace::h70b3b7caedc36ea6
   8:        0x100895345 - _rust_begin_unwind
   9:        0x1008dab03 - core::panicking::panic_fmt::h77104dabd677246a
  10:        0x100820bf2 - core::panicking::panic_display::hac2f7dd07aa765a7
  11:        0x1008206fc - hush::tests::util::test_dir::run::h8afb01b70dce6317
  12:        0x100812f65 - core::ops::function::FnOnce::call_once::hf94d47e430f4109e
  13:        0x1007be958 - serial_test::serial_core_with_return::hb0f43644d8881b1b
  14:        0x10076d640 - core::ops::function::FnOnce::call_once::hb31cf4ecc79da9bb
  15:        0x100822de6 - std::sys_common::backtrace::__rust_begin_short_backtrace::h6d0be4d88654d2ab
  16:        0x1008c7868 - test::run_test_in_process::h914445ab8d755e5e
  17:        0x1008ad88d - std::sys_common::backtrace::__rust_begin_short_backtrace::h4fc5c3c21d9852c2
  18:        0x1008b1252 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb416cf0e955bf431
  19:        0x10089e9b9 - std::sys::unix::thread::Thread::new::thread_start::h0799c011c520d3c5
  20:     0x7ff800b054e1 - __pthread_start


failures:
    runtime::tests::test_positive

test result: FAILED. 16 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.26s

error: test failed, to rerun pass '--bin hush'

which for env, cat, and rev

M1M-CPortela:rkz54y039ngi19whvbn8hkk58sf5k4q2-source chris.portela$ which echo
/nix/store/3mfkgajns47hfv0diihzi2scwl4hm2fl-coreutils-9.1/bin/echo
M1M-CPortela:rkz54y039ngi19whvbn8hkk58sf5k4q2-source chris.portela$ which rev
/usr/bin/rev
M1M-CPortela:rkz54y039ngi19whvbn8hkk58sf5k4q2-source chris.portela$ which cat
/nix/store/3mfkgajns47hfv0diihzi2scwl4hm2fl-coreutils-9.1/bin/cat

[gahag]

Hmmm, that is really weird. Note that in the last output you provided, it was another test that failed, but seemingly for the same reason. I guess we should try to replace the ${ ... }.stdout with the following in all files in src/runtime/tests/data/positive:

let result = ${ ... }
std.print(result)
result.stdout

That way, we can get the error output from the command block, and hopefully have a clue on why it's failing.