PxP Final – Progress – Apr 26, 2018

Todo List

  • [x] Find people in video
  • [x] If a pixel is belong to a person, substitute it with a reference pixel.
  • [x] Only process the first person in the video.
  • [] Refresh reference image every 5 minutes, if there’s no body in the video.


  • PC
  • Kinect V2.
Softwares & libraries:
  • Processing
  • KinectPV2 library
  • OpenCV

Schema & Codes:

Full Source Codes

Create needed PImages in Processing.

// Create 3 PImages for video, body and reference
PImage mainImg;
PImage bodyImg;
PImage refImg;

Since the library doesn’t provide mapped body image and color image, we need to map it by ourselves.

// Since we need to map those images by ourselves, create PImages for those mapped images.
// Since reference image is generated from main image, no need to map it.
PImage mainImgCropped;
PImage bodyImgCropped;

// Use depth image to separate users. 
PImage depthImgCropped;

In every frame, loop through all the video pixels and compare each video pixel with the body pixel.

// Main loop in draw()
  for (int y = 0; y < mainImgCropped.height; y++) {
    for (int x = 0; x < mainImgCropped.width; x++) {  

If at the same position, the body pixel is black, which means there’s person on the pixel

      // get the color of this pixel, see if this pixel belong to a body
      int xP = floor(map(x, 0, mainImgCropped.width, 0, bodyImgCropped.width));
      int yP = floor(map(y, 0, mainImgCropped.height, 0, bodyImgCropped.height));
      PxPGetPixel(xP, yP, bodyImgCropped.pixels, bodyImgCropped.width);               // get the RGB of the image (Bart)
      boolean isBody = false;
      if (R < 240) {  
        isBody = true;

Only remove the nearest person (by depth).

      // get the depth of this pixel, see if it's in range
      PxPGetPixel(xP, yP, depthImgCropped.pixels, depthImgCropped.width); 
      int depth = R;

      //If the body image in the range, do the removing.
      // if (is body && in range), substitute the pixel.
      if (isBody == true && ((nearestP < depth && depth < nearestP + range) || depth < 10)) {
        int i = x + y* mainImgCropped.width;
        mainImgCropped.pixels[i] = refImg.pixels[i];

Leave a Reply

Your email address will not be published. Required fields are marked *