<div class="title-block" style="text-align: center;" align="center">

# Bluespec Compiler - Test suite

---

</div>

This is a test suite for the compiler, simulator, and tools for the
**Bluespec Hardware Description Language** as found in the [bsc repository].

[bsc repository]: ../README.md
[Bluespec Inc]: https://bluespec.com
[DejaGnu]: https://www.gnu.org/software/dejagnu/

---

## Requirements

This test suite uses the [DejaGnu] testing framework, which is written in
Expect, which uses Tcl.  The following command will install those packages
on Debian or Ubuntu:

    $ apt-get install dejagnu

DejaGnu version 1.6.3+ requires a POSIX shell to run one of its scripts.
If you are using a non-compatible shell, you may need to assign `CONFIG_SHELL`
in the environment to a POSIX shell, such as `CONFIG_SHELL=/bin/sh`.

The test suite also uses a number of other tools to run and test programs:

    $ apt-get install csh grep m4 make perl pkg-config time

To simulate the Verilog generated by BSC, a Verilog simulator is required.
By default, the test suite expects Icarus Verilog, which can be installed
on Debian or Ubuntu with the following command:

    $ apt-get install iverilog

When a Verilog simulator is not available, simulation tests can be disabled
by assigning `VTEST=0` in the environment or on the command line when running
the `make` commands shown below.

Tests for BSC's SystemC generation require SystemC headers and libraries for
compilation and linking.

SystemC can be installed on Debian 10 (buster) or later with the following
command:

    $ apt-get install libsystemc-dev

When SystemC is not available, these tests can be disabled by assigning
`SYSTEMCTEST=0` in the environment or on the command line when running
the `make` commands shown below.

---

## Running the test suite

All of the following commands are executed from the `testsuite`
subdirectory of the `bsc` repository.

### Specifying the BSC installation to test

There are many ways to run tests in the suite, but the simplest is:

    $ make TEST_RELEASE=/path/to/bsc/inst check

This will run the suite on the BSC installation pointed to by `TEST_RELEASE`.

Actually, an even simpler command is possible.  If an `inst` subdirectory
exists in the `bsc` repository containing this testsuite (that is, `../inst`),
the Makefile can detect that and
implicitly assign `TEST_RELEASE` if you have omitted it:

    $ make check

If you omit `TEST_RELEASE` and there is no `inst` subdirectory of the
parent `bsc` repo, the Makefile will report an error.

### Extra tools

By default, the Makefile in `bsc/src/comp/` will not build and install
the tool `showrules` or the developer tools.  These tools can all be
installed with the `install-extra` target in `bsc/src/comp/` (or
separately with `install-showrules` and `install-utils`).

#### showrules

The test suite has tests for the `showrules` tool in `bsc.showrules`.
These tests are disabled if `showrules` is not found in the
`TEST_RELEASE` `bin` directory.

#### Developer tools

The test suite is currently able to use the following developer
tools, if they exist in the `TEST_RELEASE` `bin` directory:
`dumpbo`, `vcdcheck`, and `bsc2bsv`.  There is also a `dumpba`
tool, which is not being used by the test suite, but could be
usefully added.

After each compilation that generates a `.bo` file, the suite
can perform a sanity check by running `dumpbo` on the file
and checking for an error exit code.  There are also a small
number of tests that explicitly run `dumpbo` as part of their
testing.

The suite does not currently perform a sanity check on
generated `.ba` files, but that would be possible with
`dumpba` as a future extension.

After each Bluesim simulation that generates a `.vcd` file,
the suite can perform a sanity check by running `vcdcheck`
on the file and checking for an error exit code.  There are
also a small number of tests that explicitly run `vcdcheck`
as part of their testing.

One test (`bsc.bugs/b611/`) checks for a bug in `bsc2bsv`
but otherwise this tool is not used.  There are no tests
for the related tool, `bsv2bsc`.

### Running some or all tests

If you want to perform a quick check, the `smoke` target will
run a small subset of tests:

    $ make smoke

The set of tests that are run is specified in `Makefile`.

On a clean repository, the `check` target runs most tests,
but not all.  Several tests have been deemed to be long
tests and not essential to run.  These have been placed
in `bsc.long_tests/` and they have been disabled.  There
is a `Makefile` in that directory which has targets for
enabling and disabling tests, as well as showing the
currently enabled tests.  At the top tevel of the suite,
the `fullcheck` command will enable these tests and then
run `check`:

    $ make fullcheck

After that command has been run, making `check` will run
all the tests, so remember to disable the tests afterward
if you don't want to run them (or use git's `clean` command
to restore the repo).

### Running tests in a subdirectory

If you want to run only the tests in a given subdirectory,
you can change to that directory and do:

    $ make localcheck

The `localcheck` target will only run the tests in the
current directory.

If you want to test the current directory and recurse into
its subdirectories, you can do:

    $ make check

This will run tests in the current directory and print a report
and then, if there are no failures in that report, it will
immediately run the tests in the subdirectories and print a
report for those.  (So be aware that there will be two reports,
if you want to see all the stats.)

### Running tests in parallel

There are also `checkparallel` and `fullparallel` targets,
which are versions of `check` and `fullcheck` that run the
tests in parallel.

> **NOTE**: These targets may have issues to be fixed.

### Disabling types of tests

If you do not want to run Bluesim simulations, you can disable
those tests (or parts of tests) by assigning `CTEST=0`.
For example:

    $ make CTEST=0 check

If you do not want to run Verilog simulations, you can disable
those tests (or parts of tests) by assigning `VTEST=0`.
For example:

    $ make VTEST=0 check

If you do not want to run tests that require SystemC, you can
disable those tests by assigning `SYSTEMCTEST=0`
For example:

    $ make SYSTEMCTEST=0 check

### Specifying the Verilog simulator

BSC can be used to link Verilog files into a simulation executable.
When run this way, the choice of Verilog simulator is specified
with the `-vsim` flag.  For example, to specify Icarus Verilog as
the simulator, the flag would be `-vsim iverilog`.

This is how all Verilog simulation is done in the test suite.
The argument to `-vsim` is provided with the `TEST_BSC_VERILOG_SIM`
environment variable:

    $ make TEST_BSC_VERILOG_SIM=cvc64 check

The default value is `iverilog`.

---

## Adding new tests

TBD