Outthentic – quick way to develop user scenarios


Outthentic is a development kit for rapid development of users scripts and test scenarios. Outthentic is an essential part of Sparrow system. Let’s see how easy script development may be by using Outthentic framework.

Project boilerplate

First of all let’s create a project to hold all our scripts.

$ mkdir tutorial
$ cd tutorial/

Ok, now let’s create our first script. We are going to use Bash language here. But Outthentic plays nice with many languages(*), we will see it later.

(*) these are Bash, Perl5, Python and Ruby

Let’s say  want  to create a simple script to check nginx status:

$ touch story.check
$ cat story.bash
service nginx status

Let’s understand what we have done so far. We have created a story file called “story.bash” and empty story check file “strory.check”.

Story file is a plain bash script to do useful job. Story check file could contain some check rules to verify stdout from story file. Right now we don’t want to verify story stdout so we just leave story check file empty.

Outthentic requires that every script be paired with story check file.

Now let’s run the script, or if say in Outthentic terminology – run the story. Let’s get a strun – console utility that executes scenarios in Outthentic:

$ strun 

 at 2017-02-08 15:47:56
 * nginx is running
ok    scenario succeeded

Ok. Good. All should be clear from reading strun output. We see that nginx is running. At least this is what “service nginx status” tells us. What’s happening under the hood when we invoke “strun” ?

Strun – a [s]tory [r]unner – utility that runs story file “story.bash” and then checks if it’s exit code is 0. In case of successful exit code  strun prints “scenario succeed” in it’s report. Overall status “STATUS SUCCEED” line means all the scripts comes from out project are succeed.

Right now there is the only one script – “story.bash”, very soon though we will see that there are might be more than one script in outthentic project.

But before diving into more details about scenario development let me show how strun reports when scenario fails to succeed, let’s shut nginx down and re-run the story:

$ sudo /etc/init.d/nginx stop
$ strun 

 at 2017-02-08 15:57:25
 * nginx is not running
not ok    scenario succeeded

Check lists

Check lists are rules written in Outthentic::DSL language to verify stdout comes from story script. Recalling that we left story file empty, now let’s add some check rules to it:

$ cat story.check 
nginx is running

Now let’s start nginx over again and re-run our story:

$ sudo /etc/init.d/nginx start
$ strun 

 at 2017-02-08 16:02:38
 * nginx is running
ok    scenario succeeded
ok    text has 'nginx is running'

Good, we see new line appeared at strun report:

ok    text has 'nginx is running'

Strun executes “story.bash” script and then checks if it’s STDOUT include the string “nginx is running”.

You may use Perl5 regexs in check rules as well:

$ cat story.check 
regexp: nginx\s+is\s+running

Outthentic::DSL make it possible a lot of other complex checks, but let’s go ahead and see how we can use check rules in our scripts development.

So far this type of check looks meaningless, as “service nginx status” seems to do all the job and if it’s succeed there no need to track stdout to verify that nginx is running, unless you are true paranoid and want to add double checks 🙂

But let’s rewrite our story scenario to see how useful story checks might be. What if instead of consulting  of “service nginx status” command we want to lookup at the processes list happening at our server?

$ cat story.bash 
ps uax | grep nginx

$ cat story.check 
nginx: master
nginx: worker

Now let’s give it run and see results:

$ strun 

 at 2017-02-08 16:13:19
root     21274  0.0  0.0  85884  1332 ?        Ss   16:02   0:00 nginx: master process /usr/sbin/nginx
www-data 21275  0.0  0.0  86220  1756 ?        S    16:02   0:00 nginx: worker process
melezhik 21406  0.0  0.0  17156   944 pts/1    R+   16:13   0:00 grep nginx
ok    scenario succeeded
ok    text has 'nginx: master'
ok    text has 'nginx: worker'

Ok. Now we see our check rules (“nginx: master” and “nginx: worker”) are verified which means nginx server processes “appear” at processes list as nginx master and nginx process. It is more detailed information in comparison with those getting from simple “service nginx status” command.

What is more important “ps uax|grep nginx” might succeed with exit code zero but this does not mean that nginx server is running ( guess why? ). And this is where check rules become handy. Now let’s summarize.

Check rules VS exit code.

Sometimes you don’t have to define any check rules to verify that your script succeed, obviously most of  modern software provides a valid exit code you can rely upon. But sometimes a normal ( zero ) exit code does not mean an overall success. This test shows the idea. It is pretty simple but could be considered as basic example for such a test scenarios where you want to “grep” some information from script stdout to verify that everything goes fine. Actually this is what people usually do when hitting “foo|grep baz” command.

Another good example when exit code can’t be a good criteria is insertion into database. Say first time you insert record it does not exists and you are ok when script doing insertion  and return zero exit code. Next time you run script record already exists and script throws bad exit code and proper message ( something like record with given ID already exists … ). If after all you care only about record existence  you can’t rely on exit code here. So alternative approach could be verify script work by messages appeared at stdout:

$ cat story.check

regexp: record (created|already exists)

Outthentic suites

As I said at the beginning there are might be more than one script in the project. In terms of Outthentic we can talk about outthentic project or outthentic suite – a bunch of related stories. Strun utilizes directories to tell one story from another. Let’s add new story to our suite to start nginx service, we will reorganize directory layout on the way:

$ tree 
├── check-nginx
│   ├── story.bash
│   └── story.check
└── start-nginx
    ├── story.bash
    └── story.check

The content of  check-nginx/* files remains the same. This is a story to check nginx state. The content of start-nginx/story.bash file is pretty simple:

$ cat  start-nginx/story.bash
sudo service nginx start

We leave file start-nginx/story.check empty.

Strun uses “–story” option to set a story to run. If no “–story” option is given strun tries to run file story.bash (*) at current working directory:

$ strun  --story start-nginx

start-nginx/ at 2017-02-08 16:52:27
ok    scenario succeeded
(*) Or actually one of four files if exists – story.pl, story.bash, story.py, story.rb – as you can guess it relates to the language you write scenarios – Perl5, Bash, Python or Ruby.

Having more than one story at your project help you to split large task into small independent scripts to be running distinctly. But sometimes we want take another approach – call one scripts from others. Let’s see how we can achieve this.

Story modules

Story modules ( or in short just a modules ) are scripts being called from other scripts.
When gets called modules might being given an input parameters aka story variables.

Consider an example of simple package manager.

Let’s say we want to write a script to install packages taken from input list passed as string of space separated items:

"package-foo package-bar package-baz"

Outthentic provides very flexible API to handle command line input parameters, so we can pass package list by “–param” option:

$ strun --param "package-foo package-bar package-baz"

Now let’s split our task into two simple scripts. One – to parse input parameters and another to install given package. The overall project structure will be:

$ tree
├── hook.bash
├── meta.txt
└── modules
    └── install-package
        ├── story.bash
        └── story.check

Let’s explain a new project structure.

First of all we notice file called “hook.bash”. Hooks are way to extend strun functionality. Under the hood hooks are simple scripts to be executed before story file.

Second thing, If we look at the project root directory we don’t find neither story file nor story check files here. It’s ok. Existence of  file called “meta.txt” informs strun that this is meta story. Meta story is outthentic story which does not have  a story file at all.

Meta file is just plain text file. It could be empty. But you may place some helpful info here to be dumped when story executed:

$ cat meta.txt 
simple package manager

Hooks and meta stories are quite described at Outthentic documentation in “Hooks API” section, but let’s go ahead.

The last new thing we can notice at our project is a directory  “modules/install-package” with content very similar to the content of outthentic story ( story file and story check file there ). Well, everything kept under “modules/” directory is treated as story-modules.

Story modules as I already told are the usual outthentic stories but being called from other stories or if to be accurate from hook files. Let’s see how this happens:

$ cat hook.bash 
for p in $(config packages); do
  run_story install-package package $p

This  simple bash code does following:

1. Parses input parameters using ubiquitous “config” function provided by Outthentic
2. Splits packages string by spaces and for every item calls a story module named “install-package”:

run_story install-package package $p

Story module being passed an input parameter or story variable named “package” having the name of the package being installed.

Let’s see how story module is implemented, it’s very simple:

$ cat modules/install-package/story.bash 
package=$(story_var package)
echo install $package ...

What we do in “modules/install-package/story.bash” script?

1. Parse story input parameter by using handy “story_var” function
2. Run install command (*) for given package.

(*) For demonstration purposes we don’t run real package install here , using yum or apt-get package manager.

Let’s summarize. Story modules are very useful when design your script system. This mechanism encourages you to split a complex task into simple ones and make code reuse via “scripts-libraries”.

A plenty of information about story modules could be found at Outthentic docs in “Upstream and Downstream stories” section.

Now let’s run our story suite:

$ strun  --param packages='nginx mysql perl'

@ simple package manager

modules/install-package/ params: package:nginx at 2017-02-09 11:29:23
install nginx ...
ok    scenario succeeded

modules/install-package/ params: package:mysql at 2017-02-09 11:29:23
install mysql ...
ok    scenario succeeded

modules/install-package/ params: package:perl at 2017-02-09 11:29:23
install perl ...
ok    scenario succeeded

In next section we’ll see how supply our suites with default configuration.


Sometimes it’s useful to provide a sane default for our script parameters. Outthentic comes with a lot of ways to do this. Let’s show one.

Consider a script which looks if running nginx listening to a given port:

$ cat story.bash 
sudo netstat -nlp|grep nginx
$ cat story.check

Running a suite we see that nginx is available at 80 port as we expected:

$ strun 

 at 2017-02-09 12:22:48
tcp        0      0    *               LISTEN      21899/nginx     
tcp6       0      0 :::80                   :::*                    LISTEN      21899/nginx     
ok    scenario succeeded
ok    text has ''

Say, if nginx listen to other port and we want to make this parameter configurable for the script. Not a problem:

$ cat story.check 
generator: <

Generators are way you can built check list in run time. We can see now that port variable is passed as input parameters. Now let’s provide a sane default for port:

$ cat  suite.ini
port 80

Later if want override default setting we can say:

$ strun --param port=443

Outthentic provides other methods to handle script configuration among them are JSON/YAML/Config::General/Command Line formats and nested parameters. Please follow documentation at section “Suite Configuration”

There is more than one language to write your script

And finally as I told at the very beginning you are free choose many language to develop scripts by Outthentic framework. This is the list of supported languages:

* Per5
* Bash
* Python
* Ruby

This is how hook file for  package manager script could be written on Perl5:

$ cat hook.pl 
for my $p ( split /\s+/, config()->{packages}) {
  run_story("install-package", { package => $p });

Outthentic provides unified API for all listed languages to make it script development easy and simple:

  • Handling input parameters
  • Developing mutli scripts systems using story modules and “–story” option
  • Enabling configuration with reach support of well known formats like Config::General/YAML/JSON/Command line

Script distribution

This article only describes how one can use Outthentic in scripts development. If you want to distribute your scripts use Sparrow – outthentic scripts manager.

For further reading I would recommend you comprehensive article –  “Sparrow plugins evolution”

Script examples presented at the paper could be found here.

Regards. The author of Sparrow/Outthentic – Alexey Melezhik

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s