Testing Command Line Applications using Tomty and RakuDist

So you have a command line application written on Raku and shipped as a Raku module to an end user. You’re quite happy with unit tests get run as a part of  zef install process.

But say, you need something  extra. For example, some post deployment tests that check your tool runs properly on a small set of scenarios. Here we enter  Tomty – Raku testing framework.

With a very simple code it’s possuble to add extra tests and run those tests under RakuDist – API to test Raku modules.

Zero Install

As tests are going to  be  run by RakuDist API, no dependencies to install is required, RakuDist will take care about execution environment, you just need to create tests and push them to Git as a part of .rakudist folder,  as this:

mkdir .rakudist/.tomty && git add .rakudist && git commit -a -m "my tests"

You still may want to install Tomty to debug/run tests locally :

zef install Tomty

Hello world example

Every test consists of two parts – high level wrapper written on Raku and underlying script written on a language of your choice (any Sparrow compatible language*)

* Perl,Bash,Ruby,Powershell,Python,Raku for the moment

Raku wrapper:

​nano .rakudist/.tomty/test-01.pl6

task-run "tasks/some-task";

Test script:

nano .rakudist/tasks/some-task/task.bash

echo "Hello World"

In this example it’s just a simple echo command, of course for real modules you would have a command line of  an application.

Let’s give it a run:

cd .rakudist/ && tomty --all --verbose

[test-01] ....... 
23:12:04 02/21/2020 [repository] index updated from file:///home/melezhik/repo/api/v1/index
23:12:04 02/21/2020 [tasks/some-task] Hello World
=========================================
(=: / [1] tests in 1 sec / (1) tests passed

At the very basic level Tomty just runs an external script and checks it exit code.

Task checks

Tomty allows to create so called task checks – Raku regexp based DSL to check a script output, in simplest form it could be just a plain string check:

nano .rakudist/some-task/task.check

Hello World

Now, Tomty not only checks if a script exits with zero exit code but also verifies it’s output:

tomty --all --verbose

[test-01] ....... 
23:16:16 02/21/2020 [repository] index updated from file:///home/melezhik/repo/api/v1/index
23:16:16 02/21/2020 [tasks/some-task] Hello World
[task check] stdout match <Hello World> True
=========================================
(=: / [1] tests in 1 sec / (1) tests passed

Script parameters

Raku wrappers not just call scripts,  they might pass some parameters:

task-run "tasks/some-task", %(  
  name => Raku 
)

With a simple modification a script could handle passed parameters:

.rakudist/tasks/some-task/task.bash

name=$(config name)
echo "Hello {$name}"

You can even handle parameters within task checks:

.rakudist/tasks/some-task/task.check

generator: <<CODE
!bash
  name=$(config name)
  echo "hello ${name}"
CODE

To know more about how to write Sparrow scenarios follow documentation.

More tests

Eventually more tests are added by creating new Raku wrappers inside .rakudist/.tomty directory and respected scripts inside .rakudist/tasks folder:

.tomty/test-02.pl6
.tomty/test-03.pl6
...

The whole test suite as usually is executed by tomty --all command

Conclusion

We’ve ended up with a simple file structure to test command line application distributed as a Raku module:

.rakudist/
├── tasks
│   └── some-task
│       ├── task.bash
│       └── task.check
└── .tomty
    └── test-01.pl6
3 directories, 3 files

If you want to see more examples, take a look at my pull request for App::Mi6 application.

I would love to hear a feedback and ideas on possible usage of the toolset.

—

Thank you for reading