Pixelate Image Data Algorithm Very Slow

I had this pixelate algorithm in my tools, but when I came to apply it today to my drawing app, it's performance is seriously bad. I was wondering if you could help me with this. T

Solution 1:

Fast Hardware Pixilation

GPU can do it..

Why not just use the GPU to do it with the drawImage call, its much quicker. Draw the source canvas smaller at the number of pixels you want in the pixilation and then draw it to the source scaled back up. You just have to make sure that you turn off image smoothing before hand or the pixelation effect will not work.

No transparent pixels

For a canvas that has no transparent pixels it is a simplier solution the following function will do that. To pixelate the canvas where sctx is the source and dctx is the destination and pixelCount is the number of pixel blocks in the destination x axis. And you get filter option as a bonus because the GPU is doing the hard works for you.

Please Note that you should check vendor prefix for 2D context imageSmoothingEnabled and add them for the browsers you intend to support.

// pixelate where
// sctx is source context
// dctx is destination context
// pixelCount is the number of pixel blocks across in the destination
// filter boolean if true then pixel blocks as bilinear interpolation of all pixels involved
//                if false then pixel blocks are nearest neighbour of pixel in center of block


function pixelate(sctx, dctx, pixelCount, filter){
     var sw = sctx.canvas.width;
     var sh = sctx.canvas.height;
     var dw = dctx.canvas.width;
     var dh = dctx.canvas.height;
     var downScale = pixelCount / sw; // get the scale reduction
     var pixH = Math.floor(sh * downScale); // get pixel y axis count

     // clear destination
     dctx.clearRect(0, 0, dw, dh);  
     // set the filter mode
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = filter;
     // scale image down;
     dctx.drawImage(sctx.canvas, 0, 0, pixelCount, pixH);

     // scale image back up
     // IMPORTANT for this to work you must turn off smoothing
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = false;    
     dctx.drawImage(dctx.canvas, 0, 0, pixelCount, pixH, 0, 0, dw, dh);

     // restore smoothing assuming it was on.
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = true;
     //all done 
}

Transparent pixels

If you have transparent pixels you will need a work canvas to hold the scaled down image. To do this add a workCanvas argument and return that same argument. It will create a canvas for you and resize it if needed but you should also keep a copy of it so you don't needlessly create a new one each time you pixilate

function pixelate(sctx, dctx, pixelCount, filter, workCanvas){
     var sw = sctx.canvas.width;
     var sh = sctx.canvas.height;
     var dw = dctx.canvas.width;
     var dh = dctx.canvas.height;
     var downScale = pixelCount / sw; // get the scale reduction
     var pixH = Math.floor(sh * downScale); // get pixel y axis count

     // create a work canvas if needed
     if(workCanvas === undefined){
        workCanvas = document.createElement("canvas"); 
        workCanvas.width = pixelCount ;
        workCanvas.height = pixH; 
        workCanvas.ctx = workCanvas.getContext("2d");
     }else // resize if needed
     if(workCanvas.width !== pixelCount || workCanvas.height !== pixH){
        workCanvas.width = pixelCount ;
        workCanvas.height = pixH; 
        workCanvas.ctx = workCanvas.getContext("2d");
     }
     // clear the workcanvas
     workCanvas.ctx.clearRect(0, 0, pixelCount, pixH);  
     // set the filter mode Note the prefix, and I have been told same goes for IE
     workCanvas.ctx.mozImageSmoothingEnabled = workCanvas.ctx.imageSmoothingEnabled = filter;
     // scale image down;
     workCanvas.ctx.drawImage(sctx.canvas, 0, 0, pixelCount, pixH);

     // clear the destination
     dctx.clearRect(0,0,dw,dh);
     // scale image back up
     // IMPORTANT for this to work you must turn off smoothing
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = false;    
     dctx.drawImage(workCanvas, 0, 0, pixelCount, pixH, 0, 0, dw, dh);

     // restore smoothing assuming it was on.
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = true;
     //all done 
     return workCanvas; // Return the canvas reference so there is no need to recreate next time
}

To use the transparent version you need to hold the workCanvas referance or you will need to create a new one each time.

var pixelateWC; // create and leave undefined in the app global scope 
               // (does not have to be JS context global) just of your app 
               // so you dont loss it between renders.

Then in your main loop

pixelateWC = pixelate(sourceCtx,destContext,20,true, pixelateWC);

Thus the function will create it the first time, and then will use it over and over untill it needs to change its size or you delete it with

pixelateWC = undefined;

The demo

I have included a demo (as my original version had a bug) to make sure it all works. Shows the transparent version of the functio. Works well fullscreen 60fp I do the whole canvas not just the split part. Instructions in demo.

// adapted from QuickRunJS environment. 

// simple mouse
var mouse = (function(){
    function preventDefault(e) { e.preventDefault(); }
    var mouse = {
        x : 0, y : 0, buttonRaw : 0,
        bm : [1, 2, 4, 6, 5, 3], // masks for setting and clearing button raw bits;
        mouseEvents : "mousemove,mousedown,mouseup".split(",")
    };
    function mouseMove(e) {
        var t = e.type, m = mouse;
        m.x = e.offsetX; m.y = e.offsetY;
        if (m.x === undefined) { m.x = e.clientX; m.y = e.clientY; }
        if (t === "mousedown") { m.buttonRaw |= m.bm[e.which-1];
        } else if (t === "mouseup") { m.buttonRaw &= m.bm[e.which + 2];}
        e.preventDefault();
    }
    mouse.start = function(element, blockContextMenu){
        if(mouse.element !== undefined){ mouse.removeMouse();}
        mouse.element = element;
        mouse.mouseEvents.forEach(n => { element.addEventListener(n, mouseMove); } );
        if(blockContextMenu === true){
            element.addEventListener("contextmenu", preventDefault, false);
            mouse.contextMenuBlocked = true;
        }        
    }
    mouse.remove = function(){
        if(mouse.element !== undefined){
            mouse.mouseEvents.forEach(n => { mouse.element.removeEventListener(n, mouseMove); } );
            if(mouse.contextMenuBlocked === true){ mouse.element.removeEventListener("contextmenu", preventDefault);}
            mouse.contextMenuBlocked = undefined;            
            mouse.element = undefined;
        }
    }
    return mouse;
})();

// delete needed for my QuickRunJS environment
function removeCanvas(){
    if(canvas !== undefined){
        document.body.removeChild(canvas);
    }
    canvas = undefined;    
    canvasB = undefined;    
    canvasP = undefined;    
}
// create onscreen, background, and pixelate canvas
function createCanvas(){
    canvas = document.createElement("canvas"); 
    canvas.style.position = "absolute";
    canvas.style.left     = "0px";
    canvas.style.top      = "0px";
    canvas.style.zIndex   = 1000;
    document.body.appendChild(canvas);
    canvasP = document.createElement("canvas"); 
    canvasB = document.createElement("canvas"); 
}
function resizeCanvas(){
    if(canvas === undefined){ createCanvas(); }
    canvasB.width = canvasP.width = canvas.width = window.innerWidth;
    canvasB.height = canvasP.height = canvas.height = window.innerHeight; 
    canvasB.ctx  = canvasB.getContext("2d"); 
    canvasP.ctx  = canvasP.getContext("2d"); 
    canvas.ctx = canvas.getContext("2d"); 
    // lazy coder bug fix 
    joinPos = Math.floor(window.innerWidth/2);
}

function pixelate(sctx, dctx, pixelCount, filter, workCanvas){
     var sw = sctx.canvas.width;
     var sh = sctx.canvas.height;
     var dw = dctx.canvas.width;
     var dh = dctx.canvas.height;
     var downScale = pixelCount / sw; // get the scale reduction
     var pixH = Math.floor(sh * downScale); // get pixel y axis count

     // create a work canvas if needed
     if(workCanvas === undefined){
        workCanvas = document.createElement("canvas"); 
        workCanvas.width = pixelCount;
        workCanvas.height = pixH; 
        workCanvas.ctx = workCanvas.getContext("2d");
     }else // resize if needed
     if(workCanvas.width !== pixelCount || workCanvas.height !== pixH){
        workCanvas.width = pixelCount;
        workCanvas.height = pixH; 
        workCanvas.ctx = workCanvas.getContext("2d");
     }
     // clear the workcanvas
     workCanvas.ctx.clearRect(0, 0, pixelCount, pixH);  
     // set the filter mode
     workCanvas.ctx.mozImageSmoothingEnabled = workCanvas.ctx.imageSmoothingEnabled = filter;
     // scale image down;
     workCanvas.ctx.drawImage(sctx.canvas, 0, 0, pixelCount, pixH);
     
     // clear the destination
     dctx.clearRect(0,0,dw,dh);
     // scale image back up
     // IMPORTANT for this to work you must turn off smoothing
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = false;    
     dctx.drawImage(workCanvas, 0, 0, pixelCount, pixH, 0, 0, dw, dh);

     // restore smoothing assuming it was on.
     dctx.mozImageSmoothingEnabled = dctx.imageSmoothingEnabled = true;
     //all done 
     return workCanvas; // Return the canvas reference so there is no need to recreate next time
}



var canvas,canvaP, canvasB;
// create and size canvas
resizeCanvas();
// start mouse listening to canvas
mouse.start(canvas,true); // flag that context needs to be blocked
// listen to resize
window.addEventListener("resize",resizeCanvas);

// get some colours
const NUM_COLOURS = 10;
var currentCol = 0;
var cols = (function(count){
    for(var i = 0, a = []; i < count; i ++){
        a.push("hsl("+Math.floor((i/count)*360)+",100%,50%)");
    }
    return a;
})(NUM_COLOURS);
var holdExit = 0; // To stop in QuickRunJS environment
var workCanvas;
var joinPos = Math.floor(canvas.width / 2);
var mouseOverJoin = false;
var dragJoin = false;
var drawing = false;
var filterChange = 0;
var filter = true;
ctx = canvas.ctx;
function update(time){
    ctx.clearRect(0,0,canvas.width,canvas.height);
    ctx.drawImage(canvasB, 0, 0, joinPos, canvas.height, 0 , 0, joinPos, canvas.height); // draw background
    ctx.drawImage(canvasP,
        joinPos, 0,
        canvas.width - joinPos, canvas.height, 
        joinPos, 0, 
        canvas.width - joinPos, canvas.height
    ); // pixilation background
    
    if(dragJoin){
        if(!(mouse.buttonRaw & 1)){
            dragJoin = false;
            canvas.style.cursor = "default";
            if(filterChange < 20){
                filter = !filter;
            }
        }else{
            joinPos = mouse.x;
        }
        filterChange += 1;
        mouseOverJoin  = true;
    }else{
        if(Math.abs(mouse.x - joinPos) < 4 && ! drawing){
            if(mouse.buttonRaw & 1){
                canvas.style.cursor = "none";
                dragJoin = true;
                filterChange = 0;
            }else{
                canvas.style.cursor = "e-resize";
            }
            mouseOverJoin  = true;
        }else{
            canvas.style.cursor = "default";
            mouseOverJoin  = false;
            if(mouse.buttonRaw & 1){
                canvasB.ctx.fillStyle = cols[currentCol % NUM_COLOURS];
                canvasB.ctx.beginPath();
                canvasB.ctx.arc(mouse.x, mouse.y, 20, 0, Math.PI * 2);
                canvasB.ctx.fill();
                drawing = true
            }else{
                drawing = false;
            }
        }
        ctx.fillStyle = cols[currentCol % NUM_COLOURS];
        ctx.beginPath();
        ctx.arc(mouse.x, mouse.y, 20, 0, Math.PI * 2);
        ctx.fill();
    }
    if(mouse.buttonRaw & 4){ // setp cols on right button
        currentCol += 1;
    }
    if(mouse.buttonRaw & 2){ // setp cols on right button
        canvasB.ctx.clearRect(0, 0, canvas.width, canvas.height);
        holdExit += 1;
    }else{
        holdExit = 0;
    }
    workCanvas = pixelate(canvasB.ctx, canvasP.ctx, 30, filter, workCanvas);
    ctx.strokeStyle = "rgba(0,0,0,0.5)";
    if(mouseOverJoin){    
        ctx.fillStyle = "rgba(0,255,0,0.5)";
        ctx.fillRect(joinPos - 3, 0, 6, canvas.height);
        ctx.fillRect(joinPos - 2, 0, 4, canvas.height);
        ctx.fillRect(joinPos - 1, 0, 2, canvas.height);
    }
    ctx.strokeRect(joinPos - 3, 0, 6, canvas.height);
    
    ctx.font = "18px arial";
    ctx.textAlign = "left";
    ctx.fillStyle = "black"
    ctx.fillText("Normal canvas.", 10, 20)
    ctx.textAlign = "right";
    ctx.fillText("Pixelated canvas.", canvas.width - 10, 20);
    
    ctx.textAlign = "center";
    ctx.fillText("Click drag to move join.", joinPos, canvas.height - 62);
    ctx.fillText("Click to change Filter : " + (filter ? "Bilinear" : "Nearest"), joinPos, canvas.height - 40);

    ctx.fillText("Click drag to draw.", canvas.width / 2, 20);
    ctx.fillText("Right click change colour.", canvas.width / 2, 42);
    ctx.fillText("Middle click to clear.", canvas.width / 2, 64);

    if(holdExit < 60){
        requestAnimationFrame(update);
    }else{
        removeCanvas();
    }
}
requestAnimationFrame(update);

Solution 2:

You fetch an ImageData object for each pixel, then construct a colour string which has to be parsed again by the canvas object and then use a canvas fill routine, which has to go through all the canvas settings (transformation matrix, composition etc.)

You also seem to paint rectangles that are bigger than they need to be: The third and fourth parameters to fillRect are the width and height, not the x and y coordinated of the lower right point. An why do you start at pixel 1, not at zero?

It is usually much faster to operate on raw data for pixel manipulations. Fetch the whole image as image data, manipulate it and finally put it on the destination canvas:

var idata = sctx.getImageData(0, 0, w, h);
var data = idata.data;

var wmax = ((w / blockSize) | 0) * blockSize;
var wrest = w - wmax;

var hmax = ((h / blockSize) | 0) * blockSize;
var hrest = h - hmax;

var hh = blockSize;

for (var y = 0; y < h; y += blockSize) {
    var ww = blockSize;
    if (y == hmax) hh = hrest;

    for (var x = 0; x < w; x += blockSize) {
        var n = 4 * (w * y + x);
        var r = data[n];
        var g = data[n + 1];
        var b = data[n + 2];
        var a = data[n + 3];

        if (x == wmax) ww = wrest;

        for (var j = 0; j < hh; j++) {
            var m = n + 4 * (w * j);

            for (var i = 0; i < ww; i++) {
                data[m++] = r;
                data[m++] = g;
                data[m++] = b;
                data[m++] = a;
            }
        }
    }
}

dctx.putImageData(idata, 0, 0);

In my browser, that's faster even with the two additonal inner loops.