‹ Executors   Inline Benchmarking ›

Running ScalaMeter tests from SBT is easy – just follow these simple steps.

  • Add a ScalaMeter dependency to your project in SBT. Open build.sbt and add the following line:

    resolvers += "Sonatype OSS Snapshots" at
  "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies += "com.storm-enroute" %% "scalameter" % "[version]"

Where [version] should be the desired ScalaMeter release. You can find the exact maven dependencies for each ScalaMeter release in the download section. Alternatively, you can include ScalaMeter library as a manual dependency by downloading it and placing it into the lib directory of your project. See the download section for instructions on how to download the JAR file manually.

  • SBT needs to know that there is a new test interface in your project. Open build.sbt again and add the following lines:

      testFrameworks += new TestFramework(
    "org.scalameter.ScalaMeterFramework")
  
  logBuffered := false

Latest sbt versions run tests in parallel by default. If there are multiple regression tests in project, than they will be executed in parallel during sbt test, potentially making results useless.

To disable this behaviour:

parallelExecution in Test := false

should be included in build configuration.

And voila – you should be able to run ScalaMeter from the SBT shell now.

To run all the tests:

> test

To run a single test:

> test-only mypackage.MyScalaMeterTestName

To run tests with arguments:

> test-only mypackage.MyScalaMeterTestName -- <arguments>

Separating normal tests and benchmarks

The previous section describes the simplest possible SBT configuration. However, in most cases, you will want to separate unit tests from benchmarks, as benchmarks are usually executed less frequently.

This example template shows how to declare a separate folder for ScalaMeter benchmarks. This enables running the benchmarks with bench:test and bench:test-only.

SBT/command line test arguments

Below is a list of useful arguments. These can be used both from SBT and when running ScalaMeter from the command line.

Since ScalaMeter 0.7, running a performance test by default produces verbose output about the status of the test. ScalaMeter shows information about running your benchmarks, warmup times, measuring time, etc. The -silent flag disables this verbose output:

> test-only mypackage.MyScalaMeterTestName -- -silent

In versions prior to ScalaMeter 0.7 the verbose output was off by default. The -verbose flag enabled verbose output.

> test-only mypackage.MyScalaMeterTestName -- -verbose

The -CresultDir option changes the directory the results are generated into:

> test-only mypackage.MyScalaMeterTestName -- -CresultDir tmp2

Starting from ScalaMeter 0.4 you can selectively run only certain tests from your test suite.

The -CscopeFilter <test-name-regex> option runs only the benchmarks whose name starts with <test-name-regex>. For example:

> test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq'

> test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq\.foreach'

> test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq\.foreach.*'
‹ Executors   Inline Benchmarking ›