Point Cloud rendering in Unreal Engine (4.13 preview)

UPDATE : Tutorial available here.

I have been wondering for some time what would be a “good way” to render PointCloud data in Virtual Reality. At least it seems to be useful to get a better idea of the volume of the area and for data quality assessment.

From my understanding, the only specialized software with VR compatibility is CloudCompare from danielgm. But the sdk integration is not up to date and the performance hit with the previous DK2 sdk makes it almost unusable at the moment.

Then, my first idea was to look into WebVR. I tried to push the excellent WebGL/Three.js program Potree into StereoRendering, but it did not work out very well. WebVR is not “that” good at this point, even with thinks going on like A-Frame.

Next, I looked into Unity, since there is a dedicated plugin hooking-up the PCL library with the engine. It works, but not as good as intended and anyway I’m more of Unreal Engine guy…

Not so long ago, another thread about getting PointCloud data inside Unreal Engine started. But unlike any other, this time someone came forward with a solution: xnihil0zer0. Kudos  to him. To be honest, it’s more like “hacking” our way inside the engine than anything.

As I pointed out in the topic, the general idea is:

1. You take your point cloud (pcl)
2. You sample it to fit the pixel count of a square texture (in this case 128×128 so 16384 points as you pointed out)
3. You convert each x,y,z position to a 0-255 scale
4. You convert your “xyz” csv into a texture for which each raw equal to a pixel with its rgb values
5. You “extract” those rgb values inside UE4 so that it reads it as a coordinate system for a 256x256x256 cube
6. You can “beef up” this cube density with a greater image depth which allow you to save rgb values with extended decimals

The next step was to try it with a bigger cloud and colors. We picked-up my 3D production from Besancon and as you can see it works.

I will record a step by step tutorial by the end of the month for those willing to try it out. The Besancon project is also available in the original topic.

editor

Download the VR Build (Beware, the performance are pretty clunky. Between 40 and 60 fps with a dual xeon workstation and a stock 290x)

As you may understand in the original topic, there are lots of limitation with this way of getting PCL inside Unreal Engine.

The next step seems to think of a better solution/integration, because at this point the performance hit of rendering PCL is not really compatible with a good VR experience.

10 Responses to “Point Cloud rendering in Unreal Engine (4.13 preview)

  • Hey,
    I’d love to get that tutorial if you have the time to put it online :-).

  • Hello !! I was very excited that you have imported point clound to UE4, but I was very confused about how to import into the engine!!

    thanks!!!

    http://digitime.nazg.org/index.php/2016/09/23/point-
    cloud-rendering-in-unreal-engine-part-2/

    • Did you find your way with UE4?
      I can answer your questions if you need and sorry for the late reply, I was not aware of the comment because WordPress did not send me an alert…

      • I would really love to know how to import it into the engine, have the png now but don’t know how to load them into the engine. please help!!!

        thank!

  • Hi there.
    I was also toying with the idea to add webVR support for potree.

    Would you mind sharing the result of your work on this? It’d be great not to reinvent the wheel 🙂
    Thanks in advance!

    • For the moment, the project is in stand-by.
      Markus from Potree has been doing a lot of work lately to clean the code and make it easier to work with.
      I will wait a few weeks more for the “final” Potree 1.5 release. Then, I will try to make a component for A-Frame.
      I think it would be the best way to add more flexibility for point cloud display and interactivity.

  • Hello sir, could you please share a sample code for rendering point cloud in unreal engine. I am in need of that.

  • Hello, im very impressed by the work you’ve been doing! Im an artist on not that much of an expert when it comes to technical stuff like this but still I want to work with point clouds for a VR experience. I did your tutorial and got till the end, but when I run your script in mathlab it comes up with this error:

    >> pcl_conversion_script
    Error: File: pcl_conversion_script.m Line: 7 Column: 16
    Unexpected MATLAB expression.

    Here I post your script again with my edits for my project:

    filename = ‘fincatree2cloudfinal.csv’; %read csv file from CloudCompare
    A = csvread(filename); %read csv file from CloudCompare

    B = A(:,4:6); %New matrix B keep color information
    A = A(:,1:3); %Old maxtrix A keep spatial information

    A(:,1) = A(:,1)30.28; %Get ride of big coordinates from .las file
    A(:,2) = A(:,2)32.48; %Get ride of big coordinates from .las file
    A(:,3) = A(:,3)72.60; %Get ride of big coordinates from .las file

    A = A – min(A(:));
    A = A/max(A(:)); %Get the 0-1 scale
    A = round(A, 6); %round 6 decimals

    Ai = reshape(A,2048,2048,3);
    Bi = reshape(B,2048,2048,3);
    %image(Ai);
    %image(Bi);

    imwrite((Ai), ‘/Users/simonspeiser/Documents/Sparkling forest/pcl for unreal\spatial.png’, ‘png’, ‘BitDepth’,16)
    imwrite((Bi), ‘/Users/simonspeiser/Documents/Sparkling forest/pcl for unreal\color.png’, ‘png’, ‘BitDepth’,16)

    It seams to be a problem with the coordinate number but I can’t see what the problem is. I hope you can help me out!

    Thank you very much in advance!!!

Leave a Reply

Your email address will not be published. Required fields are marked *