Introduction
Outthentic is a development kit for rapid development of users scripts and test scenarios. Outthentic is an essential part of Sparrow system. Let’s see how easy script development might be in the Outthentic framework.
Bits and pieces of theory
First of all let’s create a project for all our scripts.
$ mkdir tutorial
$ cd tutorial/
Ok, now let’s create our first script. We are going to use Bash language here. But Outthentic plays nice with many languages (*), we will see it later.
(*) These are Bash, Perl5, Python and Ruby.
Let’s say we want to create a simple script to check status of nginx web server:
$ touch story.check
$ cat story.bash
service nginx status
Let me explain what we’ve done so far.
We have created a script story.bash and empty check file story.check .
In Outthentic there is term story which is just an abstraction for some script and its check file. We may call scripts as story scenarios and check files as story check files. We may also refer to the story script and the story check file as story data or story files.
To make Outthentic tells one story from another we should put story files into different directories. Technically speaking story is just a directory with some story files inside.
When we say “run or execute the story” it means we execute story script and apply rules from story check file to verify script stdout.
Another good explanation of stories is that they are elementary units of Outthentic framework to build a bigger things like Outthentic suites or projects.
Conversely, Outthentic project or suite is just a container with Outthentic stories.
That’s enough of theory. Let’s get back to our small script.
Here, in the example, the story scenario is a small Bash script to do useful job. Story check file could contain some check rules to verify stdout emitted by the script. Right now we don’t want to verify script stdout so we just leave story check file empty (*).
(*) In the latest versions Outthentic check files are no longer obligatory, so if you’re not going to validate scripts stdout just don’t create check file.
Now let’s run the script, or like we would say in Outthentic terminology – run the story.
Let’s get a strun – console client that executes scenarios in Outthentic stories:
$ strun
at 2017-02-08 15:47:56
* nginx is running
ok scenario succeeded
STATUS SUCCEED
Ok. Good. All should be clear from reading of strun’s report. We see that nginx is running. At least this is what service nginx status command tells us. What’s happening under the hood when we invoke strun ?
Strun is a [s]tory [r]unner – utility that runs story script story.bash and then checks if its exit code is 0. In case of successful exit code strun prints “scenario succeed” in its report. Overall “STATUS SUCCEED” line means that all the project’s scripts have succeeded. Right now there is the only one script – story.bash, very soon though we will see that there are might be more than one script in Outthentic project.
But before diving into details about scenario development let me show how strun reports when some scenario fails, let’s shut the nginx down and run our story again:
$ sudo /etc/init.d/nginx stop
$ strun
at 2017-02-08 15:57:25
* nginx is not running
not ok scenario succeeded
STATUS FAILED (256)
Check lists and check files
Check lists are rules written in Outthentic::DSL language to verify stdout emitted by story script. Do you remember that we left story file empty? Now let’s add some check rules:
$ cat story.check
nginx is running
Now let’s start nginx over again and re-run our story:
$ sudo /etc/init.d/nginx start
$ strun
at 2017-02-08 16:02:38
* nginx is running
ok scenario succeeded
ok text has 'nginx is running'
STATUS SUCCEED
Good, we see new line has appeared at the strun report:
ok text has 'nginx is running'
Strun executes story.bash script and then checks if script’s stdout include the string “nginx is running”.
You may use Perl5 regexs in the check rules as well:
$ cat story.check
regexp: nginx\s+is\s+running
Outthentic::DSL make it possible a lot of other complex checks, please follow this tutorial to see me examples. But for now let’s just see how we can use check rules in our scripts development.
So far this type of check is meaningless, as it seems that the command service nginx status do all the job and if it succeeds there no need to analyse the stdout to verify that nginx is running, unless you are true paranoid and want to add double checks :).
But let’s rewrite our story scenario to see how useful story checks could be. What if instead of “consulting” of service nginx status command we want to look up at the processes list at our server? Let’s rewrite our story:
$ cat story.bash
ps uax | grep nginx
$ cat story.check
nginx: master
nginx: worker
Now let’s give it run and see the results:
$ strun
at 2017-02-08 16:13:19
root 21274 0.0 0.0 85884 1332 ? Ss 16:02 0:00 nginx: master process /usr/sbin/nginx
www-data 21275 0.0 0.0 86220 1756 ? S 16:02 0:00 nginx: worker process
melezhik 21406 0.0 0.0 17156 944 pts/1 R+ 16:13 0:00 grep nginx
ok scenario succeeded
ok text has 'nginx: master'
ok text has 'nginx: worker'
STATUS SUCCEED
Ok. Now we see that our check rules ( “nginx: master” and “nginx: worker” ) are working and “verifying” that the nginx server processes are seen at the processes. It is much more detailed information in comparison with those getting from simple “service nginx status” command.
What is more important the command ps uax|grep nginx might succeed with exit code zero but this does not mean that nginx server is running ( guess why? because of the grep command itself appeared in process list! ), and this is where check rules become handy – to verify that some commands succeed of fail even thought they don’t drop proper exit code.
Let’s compare check rules against simple exit codes.
Check rules vs exit codes.
Sometimes you don’t have to define any special check rules to verify that your script succeeds, obviously most of the modern software provides valid exit codes you can rely upon. But sometimes a normal ( zero ) exit code does not mean that command succeeds. The previous example shows the idea. It is pretty simple but could be considered as a “template” for this type test scenarios where you want to “grep” some information from script stdout to verify that everything goes fine. Actually this is what people usually do when typing $cmd|grep foo command in a terminal.
Another good example when exit code could’t be a good criteria is insertion into database. Say, first time you insert record it does not exists and you are ok when script doing insertion and return zero exit code. Next time you run script record already exists and script throws bad exit code and proper message ( something like the table record with given ID already exists … ). If you only need to ensure that records with given ID gets inserted into database, you can write the following check rule and will be safe:
$ cat story.check
regexp: table record (created|already exists)
Outthentic suites
As I said at the beginning there are might be more than one script in the Outthentic project. In terms of Outthentic we can talk about Outthentic projects or outthentic suites – a bunch of related outthentic stories. Strun uses directories to tell one story from another. Let’s add new story to the project we created before, we have to reorganize the directory layout:
$ tree
.
├── check-nginx
│ ├── story.bash
│ └── story.check
└── start-nginx
├── story.bash
└── story.check
The content of check-nginx/* files remains the same. Check-nginx is a story to check nginx web server status.
Now there is new story – start-nginx as you can imagine to run nginx server.
The content of start-nginx/story.bash file is pretty simple:
$ cat start-nginx/story.bash
sudo service nginx start
We leave the content of file start-nginx/story.check empty.
Strun client has --story option to set a story to run.
$ strun --story start-nginx
start-nginx/ at 2017-02-08 16:52:27
ok scenario succeeded
STATUS SUCCEED
If no --story option is given strun will run file story.bash (*) at current working directory. So we can create default story which just says that user should choose one of two stories to run – check-nginx or start-nginx :
$ cat story.bash
echo usage: strun --story (start-nginx|run-nginx)
(*) Or actually one of four files if exists – story.pl, story.bash, story.py, story.rb – as you can guess it relates to the language you write scenarios on – Perl5, Bash, Python or Ruby.
Having more than one story in the project allow to have many small scripts which then you can run independently. But sometimes we want to take another approach – call one scripts from others. Let’s see how we can achieve this.
Story modules
Story modules ( or in short just modules ) are scripts being called from other scripts.
When gets called modules might being given an input parameters aka story variables.
Consider an example of a simple package manager.
Let’s say we want to write a script to install packages taken as the input string of space separated items:
script "package-foo package-bar package-baz"
Outthentic provides highly effective API to handle command line parameters, so we can pass package list by --param option:
$ strun --param packages="package-foo package-bar package-baz"
Now let’s split our task into two simple scripts. One – to parse input parameters and another to install given package. The overall project structure will be:
$ tree
.
├── hook.bash
└── modules
└── install-package
├── story.bash
└── story.check
Let’s explain the new project structure.
First of all we notice file named hook.bash. This is the hook. By using hook we can extend strun functionality. Under the hood hooks are just simple scripts to be executed before story scenario. Hooks functionality is described at the Outthentic documentation in the Hooks section. At the moment all we have to know about hooks is they are scripts that get run before story scenario.
The directory modules/install-package holds the new outthentic story install-package. When we place story files under modules/ directory we define story modules.
Story modules are the usual outthentic stories which are called from other stories by using hook files. Let me show how it works:
$ cat hook.bash
for p in $(config packages); do
run_story install-package package $p
done
This simple Bash code does following:
1. Parses input parameters using config() function provided by Outthentic
2. Splits input string by spaces and iterates for packages list, calling story module install-package passing package name as parameter:
run_story install-package package $p
Let’s see how story module is implemented, it’s very, again Bash script:
$ cat modules/install-package/story.bash
package=$(story_var package)
echo install $package ...
What’s happening in install-package/story.bash script?
1. Package name is assigned to variable by using using Outthentic story_var()
2. Package install command is executed (*).
(*) For demonstration purposes we don’t run real package install here.
Let’s summarize.
* Story modules are very useful when design your script system.
* This mechanism foster to split a complex task into simple ones and make code reuse via “script libraries” pattern.
You may found more information about Outthentic modules at the documentation pages, section Run stories from other stories
Let’s run our story suite to see all in action:
$ strun --param packages='nginx mysql perl'
modules/install-package/ params: package:nginx at 2017-02-09 11:29:23
install nginx ...
ok scenario succeeded
modules/install-package/ params: package:mysql at 2017-02-09 11:29:23
install mysql ...
ok scenario succeeded
modules/install-package/ params: package:perl at 2017-02-09 11:29:23
install perl ...
ok scenario succeeded
STATUS SUCCEED
In next section we’ll see how we may add configuration to our suites.
SUITE CONFIGURATION
It is extremely useful to provide a sane default for script input parameters.
Outthentic has a lot of tools to do this. Let me show the one.
Consider a script which prove if nginx server is listening to the given http port:
$ cat story.bash
sudo netstat -nlp|grep nginx
$ cat story.check
0.0.0.0:80
When we run the story we will see that nginx is available at 80 port as we expected:
$ strun
at 2017-02-09 12:22:48
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 21899/nginx
tcp6 0 0 :::80 :::* LISTEN 21899/nginx
ok scenario succeeded
ok text has '0.0.0.0:80'
STATUS SUCCEED
Now we want to make the port parameter configurable for the script:
$ cat suite.yaml
port: 80
Once we define configuration file called suite.yaml at the top of project directory strun will read it and configuration data will be available via config() function:
$ cat story.check
generator: << CODE
!bash
port=$(config port)
echo 0.0.0.0:$port
CODE
This check file shows the example of generators – special DSL to generate check rules on runtime. More information about generators could be found at the Outthentic::DSL documentation, at the generators section.
We can override default values by passing command line parameters:
$ strun --param port=443
Indeed Outthentic provides may sophisticated and efficient methods to configure your scripts. With well recognized formats support like JSON, YAML and Config::General. Please follow Outthentic documentation to read more, the section suite configuration
There is more than one language to write your script
And finally as I told at the very beginning you are free to choose between several language when developing scripts in Outthentic framework. Outthentic API is implemented for the following languages:
* Per5
* Bash
* Python
* Ruby
For example, this is how the hook file in the package manager suite could be written on Perl5:
$ cat hook.pl
for my $p ( split /\s+/, config()->{packages}) {
run_story("install-package", { package => $p });
}
Outthentic provides universal API for the all listed languages:
- Handling input parameters
- Developing mutli scripts systems using story modules
- Enabling configuration with reach support of well known formats
Scripts distribution
The next step is to distribute your scripts written on Outthentic framework. Sparrow is outthentic scripts manager allow you to share your scripts across any Linux boxes provided that Perl5 is installed.
Further reading
For further reading I would recommend you comprehensive article – “Sparrow plugins evolution”
Script examples
Script examples presented at the paper could be found here.
