LiDAR To PLY Conversion - The Retired Engineer

This post is a follow up on some earlier investigations on using freely available LiDAR data. For that post I had written some software to check the feasibility of converting the LiDAR data to a 3D model format. While the software worked OK it was lacking in certain areas and the whole process needed a lot of manual intervention. Rather than continue down that route I re-wrote the software in C++ in a more modular format and added some more automation for creating these models.

PLY & LiDAR Libraries

The new libraries can be found in my GitHub respository. There are more details on building and using the software in GitHub but the salient points are:

• There’s a class called “ply” which will import/export PLY files in both ascii and binary formats and allows the user to modify the PLY data.
• The “ply” class is only really designed to read/write files rather than creating from scratch1. To create the LiDAR PLY files I created a new class called “lidarply” ( derived from “ply” ) which makes it much easier to manually create and modify the file.
• The “lidar” class will read in a single LiDAR file without any assumptions about the structure of the file that were made by the previous software.
• The “lidarimage” class reads and stores Imagemagic “.txt” format images which are used for the overlay data.

There are two example executables:

plymenu

This is a menu driven command line program that allows the user to read in and perform certain edits to a PLY file. Its main features are:

• Simple edits, e.g. rescale the model, change all vertex colours, change PLY file type
• Simple hole filling algorthim mainly for use with 3D models created by photgrammetry and their preparation for 3D printing.

lidar2ply

This is a command line program to convert the raw LiDAR data files to PLY files. There are options to add an image overlay and to create mesh from the imported points. Single or multiple files can be processed.

Current missing functionality:

• The “autofill missing data” ( -a ) option has not been implemented.
• The meshing option ( -m ) will not mesh multiple files correctly.

1Despite being fairly simple to describe PLY files contain a lot of complexity. The main issues are that arbitrary data can be stored with each vertex and the data can have a number of different types which makes it difficult to handle in a generic fashion. The “lidarply” class assumes a simple set of data of fixed type when creating each vertex.

Overlay Data

Converting the LiDAR data to PLY was only part of the process. The 3D models became a lot more interesting if more data was overlaid on top of them. My approach was to colour the model verticies using an overlaid image, sourced for example from an OS map or a Google satellite photo. There were two problems with this approach:

1. This sort of image data is usually only available under a “paid for” licence which goes against what I was trying to achieve.
2. The process that I was using was very manual and required a lot of cutting and pasting of screenshots. I really wanted something that was more automated.

After a lot of research I decided that there were two possible sources of data to overcome these problems:

Ordnance Survey

The Ordnance Survey website has an API which ” … enables you to embed and use our maps in business and non-business website or mobile applications.” However it’s not immediately obvious which OS products are available for use but, after signing up, I couldn’t find any way of accessing the Landranger ( 1:50000 ) or Explorer ( 1:25000 ) maps data. This is unfortunate because these are the interesting ones with footpaths and contour data etc. Also it seems that the API is designed just for embedding in a website rather than downloading and using elsewhere.

Because of these problems and the fact that the licence is a bit restrictive I decided to abandon the OS route.

Open Street Map

The data from Open Street Map is licenced under the Open Database License (ODbL) and is much friendlier for this sort of experimentation. It’s not as detailed as the OS Landranger/Explorer maps but as these are not available anyway then that’s not really an issue 🙂

Downloading Data

Downloading OSM raw data can be done from the web page at https://www.openstreetmap.org. The annotated screenshot shows the steps needed:

The Overpass API format is as follows:

https://overpass-api.de/api/map?bbox=-4.1954,51.5558,-3.8596,51.7134

where the four values specified by “bbox” are the bounding lat/long coordinates. Once this format is known then it’s easy to generate the Overpass API call without resorting to the web page.

There are a couple of points to note:

1. The LiDAR files use the OS Map Grid Reference whereas OSM uses Lat/Long. I use this site to convert between the two. ( The maths for doing the conversion is actually given on that web page and I should really include it in my software )
2. When tiling multiple files it’s better to download and render the whole area then split the resulting image rather than trying to download each file separately. I had a lot of alignment problems when downloading separately because of the loss of precision in converting from OS Grid Ref to Lat/Long.

Rendering

Once the OSM raw data has been downloaded then it needs to be rendered into an image file so that it can be overlaid onto the 3D model. Selecting a suitable renderer proved more difficult than I expected. The wiki page has a long list of possibilities and, from my reading, Mapnik seems to be quite popular. But the wiki does also say that ” … it isn’t particularly easy to set up.” Hmm 😦

However one renderer that isn’t mentioned on the list was Ceyx ( although it does have its own wiki page ). It’s possibly a little out of date but as it’s written in Python with a few small dependencies then it was a simple matter of downloading and running it. To my slight surprise it worked very well. The only problem that I found was that I couldn’t get it to run unless I was in the directory that it was downloaded to. I suspect that I need to set up some extra path information but that’s a small point.

Height data

So far I’ve not managed to find an easy way of adding height information to the OSM data. It’s certainly possible to do it as the OsmAnd Android app does it very well. OpenCycleMap also has contour lines, and there’s a blog post about it as well. Finally the OSM wiki has a page on how use some publicly available NASA height data and render it on a map. This is certainly high on the “to do” list.

Tiling Multiple Files

Script

The “lidar2ply” executable described above will handle a single LiDAR file or a number of files defined in a separate list. To make it easier to create the files for this model I’ve put together a simple BASH script. The configuration file defines:

• Locations of the various files and what sort of image overlay is required
• Structure of the tiling – X/Y dimensions, resolution etc.
• Lat/long for downloading OSM data ( this could probably be calculated from the LiDAR data in future. )

Examples

A couple of examples of tiled files created with the script, both are screenshots from Meshlab. ( In each example the blue lines across the image are where the meshing algorithm doesn’t yet work with tiled files. )

This is a 3km x 3km square with the North Dock in Llanelli in the foreground. It’s interesting that the breakwater on the left doesn’t quite line up with LiDAR data and some of the roads are not quite where they should be. It’s possible that this is due to some alignment problems in the OS Grid Ref to Lat/Long conversion. Also some of the railway lines around the dock are long gone and are not shown on the Open Street Map web version so I think that I need to tweak the rendering settings a bit. However it’s interesting to see where they originally ran. The various buildings in the town show up quite well.

This is a 4km x 3km area around Llyn-y-Fan Fawr in the Breacon Beacons ( similar to the image in the previous post ). It’s OK but would be so much better with some height data / contour lines. Also I’ve done a bit of walking around here and it would be interesting to overlay a GPS track onto the model.

Future Work

There are two main avenues to be investigated next in the software libraries:

1. Missing functionality
   • Improve the meshing code so that it handles tiled files
   • Add the “auto-fill” for missing data
2. New features
   • Add height data information, e.g. contour lines
   • Display GPS track data

Share this:

• X
• Facebook

Like Loading...

Related