Enum.HasFlag performance with BenchmarkDotNet

BenchmarkDotNet allows you to test the performance on .NET methods. So let’s answer a question: is the Enum.HasFlag method really that slow?

Table of Contents

Just a second!
If you are here, it means that you are a software developer. So, you know that storage, networking, and domain management have a cost .

If you want to support this blog, please ensure that you have disabled the adblocker for this site. I configured Google AdSense to show as few ADS as possible - I don't want to bother you with lots of ads, but I still need to add some to pay for the resources for my site.

Thank you for your understanding.
- Davide

I’ve recently published two articles about some of the things you probably didn’t know about Enums. I had too much to write that I split the original article into 2 parts: here’s the first part and here’s the second.

In both articles I talked about flagged enums This is a way to define Enum types that allow you to join multiple values inside a single variable.

enum Beverage
{
    Water = 1,
    Beer = 2,
    Tea = 4,
    Wine = 8
}

// and, somewhere else
var beverage = Beverage.Water | Beverage.Tea;

The key point is that all enum values are power of 2 (so 1, 2, 4, 8 and so on).

To check if a variable includes a specific value, you can use the HasFlags method:

if(beverage.HasFlag(Beverage.Water)){
// do something
}

But many users reported that the HasFlag method in really inefficient, and it’s better to work at bit-level:

if((beverage & Beverage.Water) == Beverage.Water){
// do something
}

Is it true? Is there so much difference? Let’s check it using a benchmark.

BenchmarkDotNet

To test performance you can act in two ways: build you own framework or use an existing one.

BenchmarkDotNet is a .NET framework that tracks methods performance and analyses them to give you statistical results.

Let’s build the application!

I’ve created a Console application in .NET Core 3.1. Then I’ve installed BenchmarkDotNet via NuGet:

Install-Package BenchmarkDotNet and Install-Package BenchmarkDotNet.Annotations

All you need to do is create a class with certain attributes that tell the framework how to process benchmarking, how to setup the environment and so on. The benchmarking is defined in a single class, and it’s run on the Main method of the console application:

class Program
{
    static void Main(string[] args)
    {
        var summary = BenchmarkRunner.Run<EnumsBenchmark>();
    }
}

public class EnumsBenchmark
{
    [Benchmark]
    public void MethodToBeBanchmarked()
    {
        // add some logic here
    }
}

By running the application, BenchmarkDotNet calls several times the MethodToBeBanchmarked method and calculates some values, such as the mean value for the execution time and several other statistical values, like standard deviation and error.

The default output is the console: you can customize the output to have the result printed in a CSV file, in a JSON format and also in an R plot.

Our first benchmarking

It’s time to add some code!

To create the first benchmarking, I’ll create a array where every item is the “union” of 2 random Enum values.

[GlobalSetup]
public void Setup()
{
    tests = new List<Beverage>();
    int[] values = new int[] { 1, 2, 4, 8 };

    for (int i = 0; i < 50; i++)
    {
        Beverage firstElm = (Beverage)values.GetRandomFromArray<int>();
        Beverage secondElm = (Beverage)values.GetRandomFromArray<int>();
        tests.Add(firstElm | secondElm);
    }
}

This is marked with the GlobalSetup attribute. As the official documentation states,

A method which is marked by the GlobalSetup attribute will be executed only once per a benchmarked method after initialization of benchmark parameters and before all the benchmark method invocations.

Well, the documentation is pretty straightforward! The last step is to add the methods to be analysed:

[Benchmark]
public void RunWithHasFlag()
{
    tests.Select(x => x.HasFlag(Beverage.Tea)).ToList();
}

[Benchmark]
public void RunWithBitOperator()
{
    tests.Select(x => (x & Beverage.Tea) == Beverage.Tea).ToList();
}

That’s it: switch the build configuration to Release and run the program!

The result is provided in the console as a table:

As you can see, the two ways took almost the same time: 2.87ns against 290ns.

Are we finished yet? Nah, let’s do something more.

Parameterized values

You can also set up parameterized values, by adding a field enriched with the Params attribute. We can use it to set up different sizes of the array: bigger arrays bring to a more precise average value.

[Params(50, 100, 200)]
public int Size { get; set; }

and, in the Setup method, just replace the end value in the for loop.

As you see, nothing changes if we change the array size.

So, was everyone wrong about the performance issue?

Testing multiple runtimes

That’s simple: it’s an old problem, which has been fixed in 2017. That’s one of the fixes documented here.

So the best way to try it is to try with different runtimes.

To set up the benchmarking on multiple runtimes we need to do few additional steps: first of all, you need to install the C++ Desktop libraries using the Visual Studio installer.

This provides the underlying libraries used by BenchmarkDotNet to run the different frameworks.

Next step is to edit the csproj file of your project, transform the TargetFramework to TargetFrameworks (just the plural) and specify the list of libraries you want to target, separated by semicolon.

<PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>netcoreapp3.1;net461</TargetFrameworks>
</PropertyGroup>

Finally, decorate the benchmarked class with SimpleJobs attributes to indicate which runtimes must be used:

+[SimpleJob(RuntimeMoniker.Net461)]
+[SimpleJob(RuntimeMoniker.NetCoreApp31)]
public class EnumsBenchmark
{

That’s it! Now run the project and see how the performances dramatically changed.

This execution took about 7 minutes, because it run 12 benchmarks and, for each one, there was a warming up phase (which can be skipped, if you prefer).

The above table is a part of the result that you see in the previous image.

Notice the in .NET 4.6.1 the HasFlag method was more than twice slower than the bit comparison, and notice also how performance improved between .NET 4.6.1 and .NET Core 3.1. Amazing, isn’t it?

Wrapping up

If you want to easily create benchmarks, probably BenchmarkDotNet is the best choice. You can customize many aspects of the performance analysis, like the columns to be displayed and how to export the result (for instance, JSON, CSV and XML).

Talking about the performance about HasFlag now we can demonstrate that yes, in the past there was a HUGE difference about performance, but now the problem does not exist anymore.

If you want to try it on your own, here’s the project I set up for this article.

Happy coding!