Refining a Layer Mask with the Brush Tool

If you want to understand how to mask layers in Photoshop, then you need to become very friendly with your brush tool. It allows you to draw over the exact areas you want to apply an adjustment to…or rather, the areas where you want that particular layer to be visible or invisible.

It’s a VERY powerful tool, and a crucial part of layer masking…especially for photographers.

And since layer masks are part of a non-destructive workflow, you can always come back and alter your adjustment at any time by simply applying new brushwork to the layer mask.

So for this comprehensive Photoshop tutorial, I’ll be showing you exactly how to use your brush tool to refine a layer mask so that it compliments the unique content of your image. Your ability to dictate where a layer should (and should not) be visible is key to creating a professional, wall-worthy photograph.

Now if you’re a Lightroom user, the brush tool will be familiar as it works a lot like the adjustment brush…specifically how you can change the brush size, opacity, and feathering.

However, in Photoshop the brush tool is not loaded with an actual adjustment like it is in Lightroom. Instead, the layer itself will contain the adjustment, and we use the brush tool in combination with a layer mask to control where the adjustment is applied.

If you prefer to learn visually (which I highly recommend for anything Photoshop), you can download my free video course below. This five-part course also comes with practice files and a helpful PDF cheatsheet, so you can get hands-on with layers right away.

Table of Contents

• Video Lesson: How to Use the Brush Tool with Layer Masks
• Brush Tool Basics
• How to Brush Onto a Layer Mask
• Mask Reversal: How to Erase Your Masking with the Brush Tool
• Adjusting Your Brush Hardness
• Tailoring a Layer Mask
• Using a Gradient on a Layer Mask

Watch the Full Video Lesson Below

The brush tool is best learned when taught visually, so I’ve pulled a comprehensive video lesson from my membership program for you to watch. However, a video is not always the most convenient way to learn, so I’m also including the full written tutorial below.

Brush Tool Basics

To access your brush tool, simply press B or go over to your tools palette and select the brush tool from there. You can tell when the brush tool has been activated as your top menu will present options for customizing your brush tool.

To adjust the size of your brush, press the [ or ] bracket keys to increase and decrease your brush size. You’ll be doing this often so it’s good to familiarize yourself with these keyboard shortcuts.

The size of your brush is labeled by pixel count (i.e. 1200 px), so the area your brush affects is in direct relation to how many pixels your photo contains. This is why I like to just use my bracket keys to “eyeball” the brush size instead of trying to hit a specific number (i.e. 1200 px).

Brush opacity is something you’ll adjust often, so it’s helpful to know the keyboard shortcuts for this as well. At the top menu, you can adjust the opacity slider, but I find it easier to enter the opacity level with my keyboard. Much like in Lightroom, if you enter the number 1 your opacity will change to 10%, 2 for 20%, and so on. Pressing 0 will revert back to 100% opacity.

If you want a specific opacity level, press the numbers very quickly. For example, if you want to use a brush at 27% opacity, press 2 and 7 quickly…otherwise, it will change to 20% and then 70%.

The color of your brush is controlled by the foreground color swatch in your tools palette. Since we’re working with masks, your swatch should always be in greyscale…which means this swatch should be loaded with white, black, or a shade of grey.

Brushing onto a Layer Mask

First, press B to activate your brush tool and set your opacity to 100% by pressing 0. Then, make sure the layer mask is filled with 100% white and applied to the orange fill layer we created in the previous tutorial on layer masks.

Select the mask by clicking on the mask thumbnail, then make sure your foreground color is set to black by pressing D as in Default. This will set your foreground color to black and background color to white.

Click and drag your mouse across the top of your image once, which will brush a strip of black onto your layer mask. This will render the area transparent so you can see the photo underneath.

Since we are using a layer mask, this “hole” we created with the brush tool is temporary; we can always revisit this mask and add more black, white, or grey to it to change the transparency…or delete the layer mask altogether in order to bring back the orange fill layer in full.

Let’s do that right now. Change the opacity of your brush tool to 50% by pressing 5, and do the same thing again: click and drag your brush tool across the photo (making sure the layer mask is still selected).

The overall effect is still the same: we’re changing the transparency of the top layer so we can see more of the layer(s) underneath, but since we are using 50% opacity and not pure black, we are only removing 50% of the orange fill layer.

Let’s do this one more time, but with 25% opacity. Press “2” and “5” quickly to set your brush opacity to 25%, and click and drag across the bottom of the photo, making sure you are painting onto the layer mask.

Again, the effect is still apparent, but at a much lower opacity; we can now see more of the orange fill layer than the photo layer underneath since we are painting 25% grey on the layer mask. This shade of grey is 25% darker than pure white….which translates to only 25% of the underlying layer being visible.

Now let’s take a look at the layer mask so we can better visualize how this all works. The first line we drew is 100% black, the second line is 50% grey (or 50% transparent) and finally, the third line is 25% grey (which makes our layer only 25% transparent).

So in order to adjust the opacity of your layer, you need to adjust the opacity of your brush tool. Since it’s loaded with 100% black, changing the opacity of that black will control what shade of grey is painted onto your layer mask. Applying a brush stroke with 50% opacity is identical to choosing 50% grey in your color picker tool.

You could use your color picker tool to choose a shade of grey and apply that at full brush opacity, but that’s a convoluted way to paint onto a mask. It’s much easier to simply change the opacity of a brush loaded with pure black.

Reversing Parts of a Layer Mask

So far, we’ve only discussed how to add black and grey to a layer mask in order to change the transparency of a layer. We’ve also discussed how to remove a layer mask as a whole. But how do we reverse our masking for only specific parts of a layer mask? In other words, how do we shine a light back onto areas we’ve made transparent; to reverse what we just did and make that orange layer more visible again?

All you need to do is add white (or light) back to this layer mask with a white brush. This is one of the best traits of layer mask…being able to switch back and forth from adding black and white until you find the right balance between your layers. You’ll be using this a lot, especially with landscape photography.

First, let’s load the brush with white by pressing X, which will switch your foreground and background colors. You may need to press D first to load your foreground/background swatches with black and white, and then press X to switch them. Make sure your opacity is back to 100% by pressing 0.

Make sure your layer mask is selected, and then draw a white line straight down the canvas.

Since the opposite of black is white, adding white to an existing layer mask is just like using the eraser tool; it’s reversing any masking you applied, but only to the areas you are brushing…which makes it very easy to correct any sloppy brushwork.

If you look at the actual layer again, you can see that the orange has been brought back at 100% opacity by adding this strip of white.

Since we are working with 100% white, our previous masking was removed in full, as seen above. However, what happens if we use a white brush at a lower opacity to remove our masking?

Let’s demonstrate this so you can visualize how a lower opacity effects an existing mask. Bring up the layer mask by alt + clicking on the mask thumbnail in the layers palette, and adjust the opacity of our brush tool down to 50%.

Once again, brush downwards on the layer mask, making sure to cross over the black and grey horizontal lines that we created in the previous step.

The opacity of your brush directly correlates to strength. Since we are using a white brush set at 50% opacity, we are only working with 50% strength….which means we are only subtracting 50% of whatever we are brushing over.

The first horizontal black line was applied at full opacity. When we cross over this area with a white brush set at 50% opacity (or strength), it leaves us with 50%  transparency.

The second horizontal black line was applied with 50% opacity. When we cross over that with a white brush set at 50% strength, we are subtracting 50% from the existing 50%…and are left with only 25% transparency.

Finally, when we cross our third black horizontal line which was originally applied at 25% opacity, we are left with 12.5% transparency. In other words, we subtracted 50% from 25%.

If we turn off the layer mask and look at the actual orange fill layer, we can see that the same areas are now visible at 50%, 75%, and 87.5% respectively.

Adjusting your Brush Hardness

When working with layer masks, usually your brush will be very soft and feathered because you are working with 0% hardness…which gives you a nice tapered effect to your masking. It’s much like working with the adjustment brush over in Lightroom, where you adjust the feathering in order to soften your transitions.

However, there could be situations where you would want a harder transition line…for masking out very specific areas.

With the brush tool selected, right-click anywhere on your canvas. A submenu will come up allowing you to increase your brush hardness.

To demonstrate, I’ll adjust my brush hardness to 100% and go back to the mask we’ve been working with. With my foreground color set to black and opacity at 100%, I’m going to drag a line across the canvas. You can see that the edge of my brushwork is very hard and defined, with no tapering at all.

For good measure, I’ll also add a 50% grey line by adjusting my brush opacity down to 50% and drawing another line across my canvas.

If we turn off the layer mask and take a look at the layer itself, you can see how abrupt and sharp the transition is from the orange fill layer to the photo layer underneath.

Typically, you wouldn’t use a hardness set at 100% like this for tailoring an adjustment…but it can be useful when working with small and targeted areas.

I’m going to zoom into this turret here and start masking along this very sharp, precise area. You can see that my brushwork hugs the edge of this turret very closely. If I were to use a soft and feathered brush, the edges would spill over onto the sky.

With this precise targeting I can make adjustments just to this turret and not to the background sky or other parts of my image.

By adjusting your brush size, opacity, color (either black or white), and hardness…you have the ability to tailor a layer mask with a very high degree of accuracy and strength. 

Tailoring a Layer Mask

Now it’s time to move out of our practice area and do a “real world” example of how photographers use layer masks to tailor an adjustment.

First, let’s temporarily hide this orange fill layer that we’ve been working with my clicking once on the eye to make it invisible.

Next, let’s bring back the hue and saturation adjustment layer we added in the last tutorial. If you don’t have that, simply add your own and increase the saturation to 40%.

As I mentioned before, this saturation bump looks good for most of the image except for the red rock in the foreground; it’s a bit too heavy and distracting. To fix this, I’m going to mask out the rock by using my brush tool, which will ease up on the opacity – or the strength – of this saturation increase just around this area…while the rest of the image will retain the same saturation bump.

I could go in and brush over this area with a black brush set at 100% opacity, but that would remove the saturation completely. Instead, I just want to ease up on its effect.

To achieve this, I’m going to use a low opacity brush which will remove only some of the saturation. Not only will this look more natural than removing it completely, but it will give me more control over the saturation removal. If I’m not removing enough of the saturation with one brush stroke, I can simply brush over the area again and build up my brushwork until I get the result I want.

Press B to activate the brush tool, and make sure your foreground color is set to black and your hardness at 0%. Then, set your opacity to 20% by pressing 2, and simply brush over the red rock once (making sure you are brushing onto the layer mask for the saturation adjustment layer).

This is just one application of my brush, and the results already look better. I’ve scaled back on the saturation and brought back some detail in the rock that was blown out by the oversaturation.

However, the rock is still a bit too saturated. Select the layer mask again and brush over this area once more to build up your brushwork and remove more saturation.

To see a quick before and after of what this layer mask did to our image, simply shift + click the layer mask thumbnail to temporarily disable it.

Using a Gradient on a Layer Mask

Tailoring a layer mask is not just limited to your brush tool. You can use any Photoshop tool that creates a selection or allows you to paint a color.

A popular tool for layer masking is the gradient tool, which you can use to create a very soft and gradual transition between pure black and pure white.

The feathering effect of a gradient is softer than what can be achieved with your brush tool, so I will sometimes use this to blend exposures together. The results look much like a GND filter, but with more control.

First, let’s delete our current layer mask so we are looking at the orange fill layer at full opacity…and then add a fresh layer mask.

Next, select the gradient tool in your tools palette. It’s usually hidden underneath the paint bucket tool, so if it’s not visible you may need to right-click on the paint bucket to bring it up.

Once you select the gradient tool, you’ll notice that the top menu will change to gradient-specific options. These allow you to customize how your gradient will look…but for most situations, you’ll want to keep these settings to how they appear in the screenshot below.

To add a gradient to your layer mask, first make sure your mask is selected…and then click and drag to stretch out a line in any direction across your canvas. For this image, I dragged a horizontal line from the first point to the second point in the screenshot below.

The line you draw will dictate how wide the transition zone is between black and white, and will move in the direction you draw. The longer the line, the larger (and more feathered) your transition zone will be.

If you want to ensure that your gradient stays level, hold down the shift key which will keep your line completely horizontal or vertical. If you try to draw a diagonal gradient, it will restrict your line to a 45-degree angle.

For this image, I dragged my line horizontally from left to right…which means that everything before the first point in the screenshot above (where I clicked) will be pure black, everything after the second point (where I released the mouse) will be pure white, and the area between these two points will become a gradient of black slowly transitioning into white.

Since the gradient was applied directly to a layer mask, this made the orange fill layer slowly transition from being completely hidden to 100% visible.

You can see that the gradient is very smooth and evenly distributed, which would be very difficult to create with your brush tool. A gradient mask is perfect for any situation which requires very feathered and gradual masking, such as when you’re trying to mimic a GND filter.

If you make a mistake drawing your gradient, you don’t have to delete your layer mask to start again. Simply drag out a new gradient and Photoshop will automatically override and replace your existing gradient with the new one.

The settings used here will create a gradient that starts as black and ends as white, and moves in the direction you draw your line. If you want to reverse the direction of the gradient, simply tick the “Reverse” box located in your top menu which will invert the gradient to instead transition from white to black.

The brush tool and gradients barely scratch the surface as to what is possible with layer masks. Depending on your desired result, there are much more efficient tools for tailoring a layer mask…so make sure to explore alternative methods for drawing onto your mask (such as the marquee tools).

In landscape photography, luminosity masks are often used to instantly create layer masks that are completely tailored to the tonal value of your image…which allows for you to make adjustments to very targeted areas (for example, just the highlights in your clouds or reflections in the water). These are advanced selections which can be quite complicated to create, but there are free tools available that automate this process for you.

Don’t forget…download your free course on Photoshop Layers for Photographers!