Image masking is the process of separating an image from its background, either to cause the image to stand out on its own or to place the image over another background.
In the old days of film stripping, it was done by cutting a physical "mask"--a sheet of material such as rubylith--in the shape of the image, and then projecting the image through it.