ts tests failing on clean repository with dash

[davidag]

In Debian testing/bullseye, clone the ts repository and run tests/suite as suggested in README:

~/projects/ts$ ./test/suite                                                                    
............F........................................F.....F.....F...

[./test/suite:201] test_ts_runs_cleanly_with_no_tests
  unequal output:
  3c3
  < 0 pass 0 fail 0 skip 0 s
  ---
  > 0 pass 0 fail 0 skip -1 s
F

[./test/suite:783] test_include
  unequal output:
  3c3
  < P [test/examples/include -> test/examples/include a:3] test_include_a
  ---
  > P [test/examples/include:3] test_include
  5,7c5
  < P [test/examples/include -> test/examples/include b:3] test_include_b
  <
  < P [test/examples/include -> test/examples/include b -> test/examples/include_c:3] test_include_c
  ---
  > P [test/examples/include:3] test_include
F

[./test/suite:816] test_undeclared_variable
  unequal output:
  2c2
  <   test/examples/undeclared_variable: line 8: var: unbound variable
  ---
  >   test/examples/undeclared_variable: 8: /home/david/projects/ts/bin/ts: var: parameter not set
F

[./test/suite:862] test_ts_readme_common_tests
  unequal output:
  1,4d0
  < P [./test_grep_abc -> ./common_tests:1] test_it_should_pick_lines_with_abc
  <
  < P [./test_sed_abc -> ./common_tests:1] test_it_should_pick_lines_with_abc
  <
F

65 pass 4 fail 0 skip -1 s

As already noted in the README, dash has some strange behaviors but I wonder if ts own tests should pass on it, being the default sh shell in Debian/Ubuntu distributions.

[thinkerbot]

Hi @davidag, thanks for taking note. I've known about these failures for a long time but haven't done anything about them -- working out the little differences between the shells is surprisingly hard around the edges. This is good motivation to take a second look. As a starter...

test_ts_runs_cleanly_with_no_tests: Looks like dash doesn't have the SECONDS environment variable. I have done my best to be non-shell-specific but perhaps this is a bash thing that snuck in. I probably can and should have used date as it's part of the POSIX spec.

docker run -it --rm  debian:bullseye /bin/sh
# /bin/dash
# echo $SECONDS

# /bin/bash
root@2773bf3b4a38:/# echo $SECONDS
3

test_undeclared_variable: Looks like this is a case of my tests making assumptions about the output format. Surely can be fixed.

The others will take a little more time to look into.

FYI I'm often not able to prioritize working on this project but it's a favorite of mine and seeing you pointed these out, I do intend to get to them. Thanks again!

[davidag]

FYI I'm often not able to prioritize working on this project but it's a favorite of mine and seeing you pointed these out, I do intend to get to them. Thanks again!

I'm glad it serves as motivation! It's definitely not easy to handle so many small differences between shells. At the same time, this is a very nice project and I'm sure its usage could be more widespread if it supported the dash shell.

Thank you for your kind and detailed response!

[thinkerbot]

I just pushed a couple commits -- I ended up normalizing the test_ts_runs_cleanly_with_no_tests issue. README explains why I didn't go with date after all. Maybe someday. Still working on how to fix test_undeclared_variable.

[thinkerbot]

@davidag when you get a chance please try this again from master. All the tests are passing for dash, AFAICT. I still want to circle back to the timing issue, as it would be great to have that feature working for all shells.

[davidag]

It's working now, great work!