Tests

Argentum got a special syntax to define tests right in the source files along this with code that needs to be tested, and a special mode of compilation, designed to support those tests.

How to Define Tests in Code

Argentum test is a special global function, which is defined not with keyword fn but instead with keyword test.

Currently tests have no parameters and no result. In later language revisions the elements in parentheses and between the parentheses and curly braces will be used for various tests attributes, but now we have to define our tests as functions with no parameters and no result (see line 7 of the following example).

using json;
using tests { assert }
...
class MyClass { ... }
fn myFn(s str) int { ... }

test myFnTest (){                              //<-- test
   assert(myFn("Some input data") == 42);
   ...
}

log("application started {myFn("some real data")}");

The test body can contain whatever you want: instantiate classes, call functions perform any actions. There is a handy function in tests module: assert the checks condition and ends application on failure.

How to Build and Run Tests

When you build and run an application having tests (and it doesn't matter if it is release or debug mode), this application will run as if it contains no tests at all. In the above example this application just prints "application started...".

In order to use tests, we need to add a command line parameter -T with attribute "." to the compiler invocation:

agc -src "./myAppDir" -start myModule -O3 -o "t.obj" -T .

In this case compiler completely ignores the main application entry point and instead builds a special application that executes each test defined in this module and all modules directly or indirectly used from this module.

For each test our compiled program first prints to the log the name of this test in form: "moduleName_testName" and then executes the test body. So you (or automation tools you use) could always understand which test is broken.

Async and Multithreaded Tests

Your test can be a simple single-threaded code that synchronously performs actions and quits as function returns, or it can register a main application object and initiate execution in asynchronous or even multi-threaded way:

test asyncTest() {
   t = Object;             // create a dummy object instance
   sys_setMainObject(t);   // register it as an app global state
   t ~~ asyncAction(){     // post it an asyncronous deferred action
     sys_log("async hello");     // do something inside this action
     sys_setMainObject(?Object); // reset global state to null (this quits our async test)
   }
}

If we compile, link and run this test, it prints "async hello". So tests are not just plain functions, instead they are miniature application entry points. Such async tests executes one after another, when previous asynchronous test sets an empty global state, instead of quitting the application, Argentum runs the next test. Btw, even though each test handles its own asynchronous and multi-threaded mode, all tests share the same set of global constants, that are initialized before the first test and destroyed after the last one. So constants are shared between tests.

How to Define Which Tests to Build/run

Sometimes you don't want to include all tests in all used modules, or you want to compile one specific test or a group of tests. This is feasible. The command line key -T has a parameter, for which we passed a dot (".") in our first example. Actually it's a regular expression, And compiler adds to the executable only tests having full names containing this regular expression:

agc ... -T .               <-- all tests
agc ... -T "array_*"       <-- all tests from module array
agc ... -T "network_json*" <-- all JSON-related tests of the network module
agc ... -T myModule_asyncTest <-- one exact test

Argentum compiler automatically excludes from the production (non-test) executable all functions that haven't been called directly or indirectly starting from the entry point. Also it excludes all classes that haven't been instantiated and all methods that are not used. This means that you can freely create functions and classes needed only for testing purposes. They will be excluded from the actual application build. And the opposite is true as well: when you build in test mode, only selected tests and their call graphs are actually included in the test build.

How to Mock

Argentum compiler allows to define multiple directories for the source code as command-line parameters. And when it searches for some module, it checks these directories in the given order. This means that we can create a a set of mocking modules for http-client, file systems, databases etc. and put them in a special mocks directory. Now all we have to do is to call the compiler with needed directories and test names:

agc -src ./mocks -src ./myapp ... -T myTestUsingMocks

This allows us to create executable statically linked to mock classes and mock functions defined in the mock modules.

It is common to have extended API in mock classes that allows the tune up the behavior of mocks. And the opposite: we usually do not implement the full set of APIs of real classes in mocks.

This is not a problem in Argentum: when it compiles in production mode, it parses all tests but don't check the tests against names or type integrity, it merely requires the tests to be free from basic syntax errors. If test calls nonexistent function or use nonexistent data types it doesn't prevent compiler from building the production code, because tests code is never presented in the production executable. This allows the code of tests to use API that exists only in mock classes.

In contrary when compiled in test mode, Argentum skips the main program entry point and compiles the call tree of each test, which can freely use extended API of mocks.

Since both mock modules and test names are defined in the same command line, they can be automated using the same CI/CD scripts.

How to Debug Tests

Tests are just applications. They can be compiled with or without debug information with or without any optimizations. If test suite fails, perform the following steps:

  • Compile the failed test with these command-line switches:
    • into a separate executable for this test (-T testname)
    • with debug info (-g)
    • and no optimization (-O0)
  • Link it with debug version of ag-runtime (see: bin\build-debug-ffi.bat)
  • Run it in a debugger (open executable in VisialStudio as a project)
  • Set breakpoint at src\runtime\ag-assert.c: ag_fn_tests_assert function
  • Run and check the call stack and variables.
  • Step by the program if needed.

Leave a Reply

Your email address will not be published. Required fields are marked *