To achieve autonomous driving in complex environments, both cars and rail vehicles must be capable of perceiving their surroundings to ensure safe operation. Sensors must identify objects at a certain distance and quickly calculate their speed and direction. For rail vehicles to avoid collisions and react in time, they must meet higher standards than those required for automobiles, which primarily rely on cameras.

To develop a sufficiently safe system for rail, we need to determine the minimum performance requirements for these sensors, both individually and when fusing them. We must identify the distance at which objects can be detected and the accuracy of these estimates. We need to assess how quickly the system can react and whether it can exceed human reaction times. While modern cameras provide very accurate imaging, they struggle under specific conditions and at certain ranges. To achieve autonomy in rail, we need to look further. LiDAR and its unique advantages may offer a robust solution to solving perception in rail.

‍

What is a LiDAR

LiDAR (Light Detection and Ranging) is a sensor that measures distances to objects using infrared light. It generally operates by measuring the time it takes for the light to reflect off a target and return to the sensor, a method known as "Time of Flight."  The data generated by a LiDAR are discrete collections of points, each with a specific position in space. When combined into a single image, these points are collectively referred to as a point cloud.

‍

‍

Different types of LiDAR’s

There are many sub types and designs of LiDAR, as well as ways to categorize them. We will begin by distinguishing between solid-state, quasi solid-state and non solid-state (”mechanical”):

• Solid-state LiDAR’s: do not contain moving parts
• Quasi solid-state LiDAR’s: contain moving parts, but of very small size and only to steer the laser beam in free space, without moving any optical components.
• Non solid-state/Mechanical: contains moving optical components, usually a rotating head to scan the laser across the environment.

‍

Using these broad categories, LiDAR’s can be subdivided further:

• Flash LiDARs are, by design, solid state. They emit a flash covering the full area to be scanned. They have a limited range as a result of spreading their laser power over the whole scanning area at once, and are more commonly used for indoor or shorter range applications. Due to their receivers needing to capture light from the whole aperture at once, they are prone to noise from solar radiation and other sources. This closely resembles how your camera takes a photo at night, except that by measuring the time light takes from leaving your flash to reaching each pixel, a distance can be calculated.
  ‍
• Scanning LiDARs come in both mechanical and solid forms:
  • Mechanical scanners the oldest and most well known type, uses a laser emitter and receiver are mounted on a rotating head, which spins multiple times a second. Because of this, they suffer from short lifespans, poor shock and vibration performance, and have given LiDAR the reputation of a fragile sensor package.
  • MEMS (micro electronic mirror systems) is a newer piece of technology, has emerged in the past decade, where the moving parts, the mirrors, are less than a millimeter across, reducing the susceptibility to lower frequency vibrations and shock.
  • Solid-state scanning LiDARs are cutting edge and a very recent innovation, brought to life in the form of Optical Phased Array (OPA) scanners. This type of solid-state scanner contains no moving parts and is built on a semiconductor base. They are in the final stages of being productized, and are considered a significant step forward for future sensor setups.

‍

How are LiDAR’s being used

LiDAR’s are already employed in many different industries. The most common is in robotics, for both indoor and outdoor use cases.

• 3D LiDAR’s are used to scan industrial sites, allowing robots to navigate complex environments usually reserved for humans (SPOT with lidar), as well as to detect abnormalities in environments
• 3D LiDAR are used for mapping complex urban and farming environments, where 3D maps ranging from the size of buildings to entire cities are made
• 2D safety LiDAR are used
  • as a safety barrier in factories, by detecting the presence of a foreign object within a given zone (SIL rated LiDARS)
  • by autonomous vehicles in controlled environments where their high reliability coupled with operational adjustments ensure vehicles have a clear path ahead

OTIV brings the usage LiDAR into even more complex environments, applying it to advanced use cases, fusing it with camera and other sensor modalities to provide advanced perception for rail operations

‍

What makes LiDAR a good fit for rail?

The most critical difference in sensor requirements between automotive and rail is the need for greater range and higher spatial accuracy. Due to the heavier weight of trams and trains, their braking distances are significantly longer than those of cars. Considering reaction time and braking distance at different speeds, trams require longer range perception systems that can calculate collisions with obstacles beyond the usual 30-60m cameras can offer, while also maintaining the spacial accuracy radar lacks.

Another inherent aspect of rail operations is the fixed path of the vehicles. Rail systems can accurately know where the vehicle will be at a given time, thanks to the lateral constraint of the rails. This allows engineers to use LiDAR's higher accuracy to make precise vehicle space measurements. This is especially important as rail vehicles do not have the luxury of being able to drive further away from obstacles that have uncertain shapes or sizes. Additionally, newer scanning MEMS LiDARs are more reliable thanks to fewer, smaller moving elements, and allow for horizon adjustment and the use of regions-of-interest to focus the beam along the rail path.

Additionally, LiDARs resolution advantage over RADAR due to the wavelengths being used, extends into other advantages. For example being less affected by reflective metal surfaces, such as those of wagons or around other vehicles.

LiDAR also performs better than cameras when dealing with the visual similarity of long, continuous surfaces like wagons, or when there is strong visual similarity between items placed in front of each other. However, LiDAR can struggle with low reflectance on shallow-angled metal surfaces like rails, or objects like tires that are dark and curved inward.

‍

What does LiDAR not do well?

Naturally every sensor has shortcomings, and so does LiDAR:

• Certain LiDAR sensors experience a phenomenon called ghosting, where some points that are detected are false positives. This effect is generally caused by highly reflective surfaces that contain retroreflector features. As a result the LiDAR generates points in places where there aren’t, which could lead to unexpected braking situations.
• LiDAR sensors can also present issues with blooming on highly reflective surfaces. Lidar blooming is similar to that experienced by your camera or eyes when a bright light shining obscures what is around it. This happens when the laser beam from a lidar system is reflected or scattered back to the sensor by a particularly bright or reflective object. This can cause the lidar to report an object as larger than its true size.
• LiDAR struggles with returning points on low reflectance, shallow-angled metal surfaces like rails, or concave objects like tires.These sorts of false negatives indicate the absence of a target when there actually is one, but do not generally hinder performance as inferences can still be made based on context.
• Reduced performance is observed during periods of rain or snow, especially when there is rain, ice or snow on the LiDAR body itself. These manifest themselves as false positives due to rain or dust in the air, and reduced range.
• Resolution: LiDAR, even though it struggles less with small objects such as poles and bikes, it still can not match cameras, with a minimum detection width in the order of multiple centimeters

‍

Many of these issues can be solved by fusing LiDAR with other sensors. However other considerations when integrating LiDAR, such as high cost and potentially limited operational lifetime, are to be weighed at the same time. Depending on their design, they can have very delicate mechanical elements, and contain sensitive electrical components.

New sensor developments however promise improvements on multiple fronts: Optical Phased Array LiDAR are solid state and offer longer lifespans, which can reduce lifetime costs. They also offer new advanced features such as dynamic scanning patterns to focus on areas of interest, doppler shift measurement to read intrinsic point speeds, and sometimes even extreme ranges of 250+ meters.

‍

How does LiDAR compare to other sensors

To make it easier for you to understand all the different nuances when it comes to comparing sensors, we’ve made a comparative table. This does not encapsulate all the different nuances between different sensors, but offers a better idea of each modality strengths and flaws, as well as how they compliment each other. The section marked with a yellow circle do require some more context, which you can find below.

‍

Video camera: distance to object
‍A video camera can intrinsically detect distance when two cameras are used, replicating the way our eyes perceive depth through a process called convergence. When two eyes view an object from slightly different angles, the brain compares and processes this information to create a single image. With two camera sensors we use software to achieve the same result.

Radar: object size detection
‍A radar provides the “return” of an object, which can be imagined as being the size of an object, but within the wavelengths of radar. This means that a metallic object will be “bigger” than an identically (physically) sized plastic one. Additionally for 2D radar, it detects the size of an object approximately within a plane. If the radar is placed too high or it has to deal with low objects, it might give inaccurate object size readings.

LiDAR: Speed detection
‍Not all LiDAR’s are able of accurately detecting speed of objects at this time.

LiDAR: Colour detection
‍A LiDAR is capable of reading out reflectivity, but not in the same manner as we do. Similarly to Radar and its limitations due to wavelength, LiDAR can read out the brightness or reflectivity of a target at the same time as its range. This is like having vision in black and white, allowing LiDAR to distinguish targets such as reflective yellow vests and emergency vehicles.

‍

Combining LiDAR with other sensors

A LiDAR complements other perception sensors such as cameras and radar in various ways:

• It does not require external light, so it works both during the day and at night.
• It allows engineers to obtain shapes of arbitrary objects without the need for visual cues like camera or object material and shape as much as radar.
• LiDAR provides reliable estimation of short and long-range distances, and changes in distance and direction over time. This helps identify collisions earlier than pure vision would,  and allows engineers to distinguish targets of interest from noise due to the ability to distinguish height, a key distinction from classical 2D radars.
• It offers high-definition 3D data, at a rate of millions of points per second, with detection ranges of up to and beyond 100 meters, and a position error of less than two centimeters. This enables the creation of a real time, precise, 3D depiction of the surrounding environment.

‍

How does OTIV apply LiDAR?

Different sensors offer a wide range of possibilities. At OTIV, we are continuously testing new sensors as well as machine learning techniques. We maintain ongoing relationships with multiple manufacturers to experiment with their latest radar, camera, lidar, ultrasonic, microwave and other product ranges.

Our multiple years of working with different sensors has provided us with ample amounts of data that we are now able to leverage. We are able to offer camera-only solutions based on deep learning, expertise in stereo and multi camera setups, as well as fused camera-LiDAR systems based on state-of-the-art robotics techniques, developed in house. We have applied this knowledge to numerous projects, both for learning and in long-term deployment. For us, the advantages of LiDAR come down to:

• Reductions in false positive rates
• Longer range detections thanks to efficient fusing of camera and LiDAR data
• LiDAR allows us to effectively manage edge cases:
  • There's no need to see the same obstacle multiple times to learn its shape and size.
  • Less data is required to continuously retrain internal models.
  • Reduced need for annotation labor.

We also collaborate closely with several leading manufacturers to test new and experimental LiDAR sensors and work on the certification of next-generation sensors.

For each of our clients we make a thorough assessment of what sensor setups are required for their needs and environmental factors. If you are interested in exploring what added value assistance and autonomous technology can bring to your rail operations, please contact us!