How to manipulate pixels using Canvas

This article mainly introduces the relevant information on how to use Canvas to operate pixels. The content is quite good. I will share it with you now and give it as a reference.

Modern browsers support video playback via theelement. Most browsers can also access the camera through the MediaDevices.getUserMedia() API. But even if these two things are combined, we cannot directly access and manipulate these pixels.

Luckily, browsers have a Canvas API that allows us to draw graphics using JavaScript. We can actually draw images to thefrom the video itself, which allows us to manipulate and display those pixels.

What you learn here about how to manipulate pixels will provide you with the foundation for working with images and videos of any kind or source, not just canvas.

Add Image to Canvas

Before we start playing the video, let’s see how to add an image to canvas.

We create an image element to represent the image to be drawn on the canvas. Alternatively, we can use the Image object in JavaScript.

var canvas; var context; function init() { var image = document.getElementById('SourceImage'); canvas = document.getElementById('Canvas'); context = canvas.getContext('2d'); drawImage(image); // Or // var image = new Image(); // image.onload = function () { // drawImage(image); // } // image.src = 'image.jpg'; } function drawImage(image) { // Set the canvas the same width and height of the image canvas.width = image.width; canvas.height = image.height; context.drawImage(image, 0, 0); } window.addEventListener('load', init);

The above code draws the entire image to the canvas.

View Paint image on canvas via Welling Guzman (@wellingguzman) on CodePen.

Now we can start playing with those pixels!

Update image data

Image data on the canvas allows us to manipulate and change pixels.

The data property is an ImageData object, which has three properties - width, height and data/ All of these represent something based on the original image. All of these properties are read-only. What we care about is the data, a one-dimensional array represented by a Uint8ClampedArray object, containing data for each pixel in RGBA format.

Although the data property is read-only, it does not mean that we cannot change its value. It means that we cannot assign another array to this property.

// Get the canvas image data var imageData = context.getImageData(0, 0, canvas.width, canvas.height); image.data = new Uint8ClampedArray(); // WRONG image.data[1] = 0; // CORRECT

You may ask, what value does the Uint8ClampedArray object represent. The following is a description from MDN:

The Uint8ClampedArray type array represents an array of 8-bit unsigned integers, which is clamped to 0-255; if If you specify a value outside the range [0,255], 0 or 255 will be set; if you specify a non-integer, the nearest integer will be set. The contents are initialized to 0. Once established, you can use the object's methods to reference the element, or use standard array indexing syntax (i.e. using bracket notation)

In short, this array stores a value ranging from 0 to 255 at each position, which makes it a perfect solution for the RGBA format scheme because each part is represented by a value from 0 to 255.

RGBA Color

Color can be represented in RGBA format which is red, green and blue A combination of. A represents the alpha value of the color opacity.

Each position in the array represents a color (pixel) channel value.

• The first position is Red value

• The second position is the green value

• The third position is the blue value

• The fourth position is the Alpha value

• The fifth position is the next pixel red value

• The sixth position is the next The green value of a pixel

• The 7th position is the blue value of the next pixel

• The 8th position is the next pixel Alpha Values

• and so on...

If you have a 2x2 image, then we have a 16 bit array (2x2 pixels x 4 each value).

2x2 image scaled down

The array will look like this:

// RED GREEN BLUE WHITE [ 255, 0, 0, 255, 0, 255, 0, 255, 0, 0, 255, 255, 255, 255, 255, 255]

Change pixels Data

One of the quickest things we can do is set all pixels to white by changing all RGBA values to 255.

// Use a button to trigger the "effect" var button = document.getElementById('Button'); button.addEventListener('click', onClick); function changeToWhite(data) { for (var i = 0; i < data.length; i++) { data[i] = 255; } } function onClick() { var imageData = context.getImageData(0, 0, canvas.width, canvas.height); changeToWhite(imageData.data); // Update the canvas with the new data context.putImageData(imageData, 0, 0); }

The data will be passed as a reference, which means any modification we make to it, it will change the value of the passed parameter.

Invert Colors

A great effect that doesn’t require too much calculation is to invert the colors of an image.

Color values can be inverted using the XOR operator (^) or this formula 255 - value (value must be between 0-255).

function invertColors(data) { for (var i = 0; i < data.length; i+= 4) { data[i] = data[i] ^ 255; // Invert Red data[i+1] = data[i+1] ^ 255; // Invert Green data[i+2] = data[i+2] ^ 255; // Invert Blue } } function onClick() { var imageData = context.getImageData(0, 0, canvas.width, canvas.height); invertColors(imageData.data); // Update the canvas with the new data context.putImageData(imageData, 0, 0); }

We are incrementing the loop by 4 like before instead of 1, so we can go from pixel to pixel, filling the 4 elements in the array with each pixel .

The alpha value has no effect on inverting the color, so we skip it.

Brightness and Contrast

Use the following formula to adjust the brightness of an image: newValue = currentValue 255 * (brightness / 100).

1. Brightness must be between -100 and 100

2. currentValue is the current lighting value for red, green or blue.

3. newValue is the result of adding brightness to the current color light

4. Adjusting the contrast of the image can be done using this formula:

factor = (259 * (contrast + 255)) / (255 * (259 - contrast)) color = GetPixelColor(x, y) newRed = Truncate(factor * (Red(color) - 128) + 128) newGreen = Truncate(factor * (Green(color) - 128) + 128) newBlue = Truncate(factor * (Blue(color) - 128) + 128)

The main calculation is to get the contrast factor that will be applied to each color value. Truncation is a function that ensures the value remains between 0 and 255.

Let’s write these functions into JavaScript:

function applyBrightness(data, brightness) { for (var i = 0; i < data.length; i+= 4) { data[i] += 255 * (brightness / 100); data[i+1] += 255 * (brightness / 100); data[i+2] += 255 * (brightness / 100); } } function truncateColor(value) { if (value < 0) { value = 0; } else if (value > 255) { value = 255; } return value; } function applyContrast(data, contrast) { var factor = (259.0 * (contrast + 255.0)) / (255.0 * (259.0 - contrast)); for (var i = 0; i < data.length; i+= 4) { data[i] = truncateColor(factor * (data[i] - 128.0) + 128.0); data[i+1] = truncateColor(factor * (data[i+1] - 128.0) + 128.0); data[i+2] = truncateColor(factor * (data[i+2] - 128.0) + 128.0); } }

在这种情况下，您不需要truncateColor函数，因为Uint8ClampedArray会截断这些值，但为了翻译我们在其中添加的算法。

需要记住的一点是，如果应用亮度或对比度，则图像数据被覆盖后无法回到之前的状态。如果我们想要重置为原始状态，则原始图像数据必须单独存储以供参考。保持图像变量对其他函数可访问将会很有帮助，因为您可以使用该图像来重绘画布和原始图像。

var image = document.getElementById('SourceImage'); function redrawImage() { context.drawImage(image, 0, 0); }

使用视频

为了使它适用于视频，我们将采用我们的初始图像脚本和HTML代码并做一些小的修改。

HTML

通过替换以下行来更改视频元素的Image元素：

...with this:

JavaScript

替换这一行：

var image = document.getElementById('SourceImage');

...添加这行：

var video = document.getElementById('SourceVideo');

要开始处理视频，我们必须等到视频可以播放。

video.addEventListener('canplay', function () { // Set the canvas the same width and height of the video canvas.width = video.videoWidth; canvas.height = video.videoHeight; // Play the video video.play(); // start drawing the frames drawFrame(video); });

当有足够的数据可以播放媒体时，至少在几帧内播放事件播放。

我们无法看到画布上显示的任何视频，因为我们只显示第一帧。我们必须每n毫秒执行一次drawFrame以跟上视频帧速率。

在drawFrame内部，我们每10ms再次调用drawFrame。

function drawFrame(video) { context.drawImage(video, 0, 0); setTimeout(function () { drawFrame(video); }, 10); }

在执行drawFrame之后，我们创建一个循环，每10ms执行一次drawFrame - 足够的时间让视频在画布中保持同步。

将效果添加到视频

我们可以使用我们之前创建的相同函数来反转颜色：

function invertColors(data) { for (var i = 0; i < data.length; i+= 4) { data[i] = data[i] ^ 255; // Invert Red data[i+1] = data[i+1] ^ 255; // Invert Green data[i+2] = data[i+2] ^ 255; // Invert Blue } }

并将其添加到drawFrame函数中：

function drawFrame(video) { context.drawImage(video, 0, 0); var imageData = context.getImageData(0, 0, canvas.width, canvas.height); invertColors(imageData.data); context.putImageData(imageData, 0, 0); setTimeout(function () { drawFrame(video); }, 10); }

我们可以添加一个按钮并切换效果：

function drawFrame(video) { context.drawImage(video, 0, 0); if (applyEffect) { var imageData = context.getImageData(0, 0, canvas.width, canvas.height); invertColors(imageData.data); context.putImageData(imageData, 0, 0); } setTimeout(function () { drawFrame(video); }, 10); }

使用 camera

我们将保留我们用于视频的相同代码，唯一不同的是我们将使用MediaDevices.getUserMedia将视频流从文件更改为相机流。

MediaDevices.getUserMedia是弃用先前API MediaDevices.getUserMedia（）的新API。浏览器仍旧支持旧版本，并且某些浏览器不支持新版本，我们必须求助于polyfill以确保浏览器支持其中一种。

首先，从视频元素中删除src属性：

// Set the source of the video to the camera stream function initCamera(stream) { video.src = window.URL.createObjectURL(stream); } if (navigator.mediaDevices.getUserMedia) { navigator.mediaDevices.getUserMedia({video: true, audio: false}) .then(initCamera) .catch(console.error) ); }

Gray = 0.21R + 0.72G + 0.07B // Luminosity Gray = (R + G + B) ÷ 3 // Average Brightness Gray = 0.299R + 0.587G + 0.114B // rec601 standard Gray = 0.2126R + 0.7152G + 0.0722B // ITU-R BT.709 standard Gray = 0.2627R + 0.6780G + 0.0593B // ITU-R BT.2100 standard

function createGradient(colorA, colorB) { // Values of the gradient from colorA to colorB var gradient = []; // the maximum color value is 255 var maxValue = 255; // Convert the hex color values to RGB object var from = getRGBColor(colorA); var to = getRGBColor(colorB); // Creates 256 colors from Color A to Color B for (var i = 0; i <= maxValue; i++) { // IntensityB will go from 0 to 255 // IntensityA will go from 255 to 0 // IntensityA will decrease intensity while instensityB will increase // What this means is that ColorA will start solid and slowly transform into ColorB // If you look at it in other way the transparency of color A will increase and the transparency of color B will decrease var intensityB = i; var intensityA = maxValue - intensityB; // The formula below combines the two color based on their intensity // (IntensityA * ColorA + IntensityB * ColorB) / maxValue gradient[i] = { r: (intensityA*from.r + intensityB*to.r) / maxValue, g: (intensityA*from.g + intensityB*to.g) / maxValue, b: (intensityA*from.b + intensityB*to.b) / maxValue }; } return gradient; } // Helper function to convert 6digit hex values to a RGB color object function getRGBColor(hex) { var colorValue; if (hex[0] === '#') { hex = hex.substr(1); } colorValue = parseInt(hex, 16); return { r: colorValue >> 16, g: (colorValue >> 8) & 255, b: colorValue & 255 } }

var gradients = [ {r: 32, g: 144, b: 254}, {r: 41, g: 125, b: 253}, {r: 65, g: 112, b: 251}, {r: 91, g: 96, b: 250}, {r: 118, g: 81, b: 248}, {r: 145, g: 65, b: 246}, {r: 172, g: 49, b: 245}, {r: 197, g: 34, b: 244}, {r: 220, g: 21, b: 242}, {r: 241, g: 22, b: 242}, ];

var gradientColors = createGradient('#0096ff', '#ff00f0'); var imageData = context.getImageData(0, 0, canvas.width, canvas.height); applyGradient(imageData.data); for (var i = 0; i < data.length; i += 4) { // Get the each channel color value var redValue = data[i]; var greenValue = data[i+1]; var blueValue = data[i+2]; // Mapping the color values to the gradient index // Replacing the grayscale color value with a color for the duotone gradient data[i] = gradientColors[redValue].r; data[i+1] = gradientColors[greenValue].g; data[i+2] = gradientColors[blueValue].b; data[i+3] = 255; }