Figure 4: Edge hardware architecture.
Field of View (FoV). As shown in figure 4, LiDAR,
Radar, and camera units are connected to the switch
using Ethernet 1G connection. Further, the switch is
connected to the central application computer which
does all the software development of the edge. All
the sensors are powered using a 12 or 24 V DC power
supply. The switch is powered using a 48V DC power
supply. The application unit is a high computing
desktop computer that consists of multi-core CPUs
and dedicated GPU to develop and deploy AI algo-
rithms for sensor fusion.
The goal of this research in the direction of en-
vironment perception is to detect, classify and track
vehicles and VRUs in different weather and light con-
ditions. As each sensor has its pros and cons, to com-
plement them, sensor fusion is developed in the ap-
plication unit. Further, this work specifically aims to
develop and implement AI-based raw level sensor fu-
sion algorithms to fuse the raw data from two or more
sensors for finding optimum solutions and parame-
ters for different light and weather situations across
the junction. The raw level sensor fusion is selected
over object-level fusion to explore the benefits of us-
ing complete data available from sensors.
As described in figure 5, raw data of LiDAR, i.e.
point cloud, raw data from cameras, i.e., RGB images,
and raw data from Radar, i.e., radar detection points
are acquired from sensors and sent to the application
unit at a pre-defined data rate. At present, this data
rate is defined as 10 Hz for each sensor. For the de-
velopment of a software framework, Robot Operating
System (ROS) is used.
To apply AI-based algorithms, labeled data is re-
quired. For this purpose, the sensors are mounted
on the lab mast using customized 3D mountings, and
then they are calibrated with each other and also with
edge to get the data in edge coordinate frame. Further,
all the sensors are synchronized together using com-
mon time reference before collecting the data in the
application unit. To label the sensor data, methods
involving both manual labeling and semi-automatic
labeling are used. Further specific scenarios with
pedestrians and vehicles equipped with GPS and IMU
systems are also designed to collect ground truth data.
In order to use the raw data directly as fusion, two
approaches are finalized after doing a literature survey
(Chadwick et al., 2019) (Chang et al., 2020). In the
first approach, the radar point cloud is transformed
into an RGB plane, and values of radar, i.e. spatial
information (X, Y, Z) and measurement information,
i.e. range, doppler velocity, and RCS are encoded to
RGB values. Similarly, the lidar dense point cloud is
transformed and encoded into a separate RGB plane.
These results in 3 independent RGB planes, each from
radar, lidar, and camera for the same instance. These
are then at first fed into a few separate CNN layers
to extract high-level features, then added together and
further passed through more CNN layers to finally get
the object position and class information.
In the second approach of the raw sensor fusion,
the radar and Lidar 3d point cloud data is encoded
into a separate 3D voxel grid. Then the 3d voxel grid
input is fed to 3D CNN layers to extract upper layer
features separately for radar and Lidar. Camera RGB
images are fed into 2D CNN layers. After extrac-
tion of high-level features, an intermediate later is de-
signed to transform the features in a common plane,
then added together and further trained using more
layers to finally extract object position and class.
The sensor fusion algorithm provides position,
speed, and class of objects which are further tracked
using filters. The final track objects’ information is
sent to the backend. As per the perception informa-
tion processed and sent by edge, the backend takes
appropriate decisions to optimize the traffic flow and
also to assist passing vehicles by providing informa-
tion and/or warnings in real-time.
Figure 5: Edge software architecture.
3.2 Communication
An appropriate communication design enables the
promising information exchange among all intelligent
components in the entire system, which guarantees
the performance of the intelligent infrastructure-based
traffic services and applications. In this paper, the
challenges of efficient data sharing in communication
networks is specifically tackled, i.e., 1) highly hetero-
geneous networks for dissemination of various mes-
sages using V2V, V2C, V2I, etc. 2) variety of QoS
requirements in miscellaneous traffic services.
Concept of Smart Infrastructure for Connected Vehicle Assist and Traffic Flow Optimization
363