Running ScalaMeter tests from SBT is easy – just follow these simple steps.
Add a ScalaMeter dependency to your project in SBT. Open
build.sbtand add the following line:
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots" libraryDependencies += "com.storm-enroute" %% "scalameter" % "[version]"
[version] should be the desired ScalaMeter release.
You can find the exact maven dependencies for each ScalaMeter release
in the download section.
Alternatively, you can include ScalaMeter library as a manual dependency
by downloading it and placing it into the
lib directory of your project.
See the download section for instructions on how to download
the JAR file manually.
SBT needs to know that there is a new test interface in your project. Open
build.sbtagain and add the following lines:
testFrameworks += new TestFramework( "org.scalameter.ScalaMeterFramework") logBuffered := false
Latest sbt versions run tests in parallel by default.
If there are multiple regression tests in project, than they will be executed in
sbt test, potentially making results useless.
To disable this behaviour:
parallelExecution in Test := false
should be included in build configuration.
And voila – you should be able to run ScalaMeter from the SBT shell now.
To run all the tests:
To run a single test:
> test-only mypackage.MyScalaMeterTestName
To run tests with arguments:
> test-only mypackage.MyScalaMeterTestName -- <arguments>
Separating normal tests and benchmarks
The previous section describes the simplest possible SBT configuration. However, in most cases, you will want to separate unit tests from benchmarks, as benchmarks are usually executed less frequently.
This example template
shows how to declare a separate folder for ScalaMeter benchmarks.
This enables running the benchmarks with
SBT/command line test arguments
Below is a list of useful arguments. These can be used both from SBT and when running ScalaMeter from the command line.
Since ScalaMeter 0.7, running a performance test by default produces verbose output
about the status of the test.
ScalaMeter shows information about running your benchmarks, warmup times,
measuring time, etc.
-silent flag disables this verbose output:
> test-only mypackage.MyScalaMeterTestName -- -silent
In versions prior to ScalaMeter 0.7 the verbose output was off by default.
-verbose flag enabled verbose output.
> test-only mypackage.MyScalaMeterTestName -- -verbose
-CresultDir option changes the directory the results are generated into:
> test-only mypackage.MyScalaMeterTestName -- -CresultDir tmp2
Starting from ScalaMeter 0.4 you can selectively run only certain tests from your test suite.
-CscopeFilter <test-name-regex> option runs only the benchmarks whose name starts
> test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq' > test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq\.foreach' > test-only mypackage.MyScalaMeterTestName -- -CscopeFilter 'Seq\.foreach.*'