Creation of 3D Maps for Satellite Communications to Support Ambulatory Rescue Operations

Abstract

A communications profile is a system that acquires information from communication links to an ambulance or other vehicle moving on a road and compiles a database based on this information. The equipment (six sets of HDTVs, fish-eye camera, satellite antenna with tracking system, and receiving power from the satellite beacon of the N-star) mounted on the roof of the vehicle, image data were obtained at Yokohama Japan. From these data, the polygon of the building was actually produced and has arranged on the map of the Geographical Survey Institute of a 50 m-mesh. The optical study (relationship between visibility rate and elevation angle) were performed on actual data taken by fish-eye lens, and simulated data by 3D-Map with polygons. There was no big difference. This 3D map system then predicts the communication links that will be available at a given location. For line-of-sight communication, optical analysis allows approximation if the frequency is sufficiently high. For non-line-of-sight communication, previously obtained electric power data can be used as reference information for approximation in certain cases when combined with predicted values calculated based on a 3D map. 3D maps are more effective than 2D maps for landing emergency medical helicopters on public roadways in the event of a disaster. Using advanced imaging technologies, we have produced a semi-automatic creation of a high-precision 3D map at Yokohama Yamashita Park and vicinity and assessed its effectiveness on telecommunications and ambulatory merits.

I. PURPOSE

High-precision 3D maps based on advanced imaging technologies can be useful in establishing communication profiles for ambulance vehicles, visualizing transport routes (e.g., for landing emergency helicopters on public roadwaysor for rescues from high-rise buildings), and predictingtsunami damage. To assess the effectiveness of 3D maps for such applications, we produced a prototype 3D map of Yokohama Yamashita Park and vicinity. This paper presents an analysis and assessments of 3D maps and theirapplications in emergency medical services.

 

II. BACKGROUND

Polygon means the form where picture information was processed with the characteristic outline by computergraphics. Operation processing is reduced by this and amotion of a three-dimensional picture can respond in shorttime. The line concretely obtained from the outline of the building here is pointed out.

By the way, three-dimensional spatial data is the application to a car-navigation system such as Google Map(however -- fixing the viewpoint 1m in the height by the camera attached to the car) or widely used in various fields of guidance service. It is used for the space design of the simulation and environmental design of a townscape, astore, disaster refuge guidance, flood damage, a firesimulation, etc.

In a two-dimensional map, since the complicatedgeographic information of an unclear townscape can bevisually grasped intelligibly with 3D(dimension) map, operators (include fireman, helicopter pilot) can understand a city atmosphere in the more nearly actually state. At the manufacturing step of 3 map of a city area, it is froman aerial photograph.

There are many processes to create polygons from actualimage, such as object removal of the outside for restoration, roadside tree for acquisition of building positioninformation, the acquisition of height information using alaser range finder, and building appearance creation are included.

In this research, texture mapping of 3D polygon map wascut out from two or more continuous pictures from an in-vehicle camera with changing photography directions. It is able to create side part of the buildings from land mobilecar. We can treat image distortion with geometricalgorithm. Furthermore, the roof portion which cannot betaken from land mobile vehicle was created from imagedata from helicopter. These two approaches (land and air) can make the building polygon by 3D.

Related these digital work, there are some paper reported to create building polygon automatically by graphiccomputers, however, under the present circumstances in a Japanese-styled house, it is very difficult to design and perform the full-automation of 3D polygon. Eventually, the building of each needs a check by human eyes.

 

III. METHODS

 

3.1. Mathematical Proof of Polygon

The principle which extracts points (points of Polygon) with the feature from an input picture [1-14], and its mathematical proof are as following. It is called the Scale Invariant Feature Transform: SIFT introduced by Dr. David G. Lowe who has the patent issue these formations [15].

 

3.1.1Scale space

The gauss function G is defined as follows,

\(G(x, y, \sigma)=\frac{1}{2 \pi \sigma^{2}} e^{-\left(x^{2}+y^{2}\right) / 2 \sigma^{2}}\)       (1)

where I (x, y) is a function of an input picture and the function L (x, y) is defined,

\(L(x, y, \sigma)=G(x, y, \sigma) * I(x, y)\)       (2)

where D (x, y) which is a difference (Difference-of-Gaussian: DoG) of the smoothing picture which collapsed the input picture I (x, y) is defined from the following formula.

\(\begin{aligned} &D(x, y, \sigma)=(G(x, y, k \sigma)-G(x, y, \sigma)) * I(x, y)\\ &=L(x, y, k \sigma)-L(x, y, \sigma) \end{aligned}\)       (3)

Consider the relationship of D(x,y) to . Therelationship between D(x,y) and can be understood from the heat diffusion equation:

\(\frac{\partial G}{\partial \sigma}=\sigma \nabla^{2} G\)       (4)

From this, we see V2G can be computed from the finitedifference approximation to,

\(\frac{\partial G}{\partial \sigma} \approx \frac{G(x, y, k \sigma)-G(x, y, \sigma)}{k \sigma-\sigma}\)       (5)

and therefore

\(\frac{G(x, y, k \sigma)-G(x, y, \sigma)}{k \sigma-\sigma} \approx \sigma \nabla^{2} G\)       (6)

\(G(x, y, k \sigma)-G(x, y, \sigma) \approx(k-1) \sigma^{2} \nabla^{2} G\)       (7)

 

When D(x,y) has scales differing by a constant factor ital ready incorporates the σ2 scale normalization required forscale-invariance.

 

3.1.2 Localization

The 3D quadratic function is fit to the local sample points, and will start with Taylor expansion with sample point as the origin:

\(D(\mathrm{X})=D+\frac{\partial D^{T}}{\partial \mathrm{X}} \mathrm{X}+\frac{1}{2} \mathrm{X}^{\tau} \frac{\partial^{2} D}{\partial \mathrm{X}^{2}} \mathrm{X}\)       (8)

where

\(\mathrm{X}=(x, y, \sigma)^{T}\)(9)  

Take the derivative with respect to X, and set it to 0, giving

\(0=\frac{\partial D}{\partial X}+\frac{\partial^{2} D}{\partial X^{2}} \hat{X}\)       (10)

the location of the keypoint

\(\hat{\mathrm{X}}=-\frac{\partial^{2} D^{-1}}{\partial \mathrm{X}^{2}} \frac{\partial D}{\partial \mathrm{X}}\)       (11)

This is a 3x3 linear system.as:

\(\left[\begin{array}{lll} {\frac{\partial^{2} D}{\partial \sigma^{2}}} & {\frac{\partial^{2} D}{\partial \sigma y}} & {\frac{\partial^{2} D}{\partial \alpha x}} \\ {\frac{\partial^{2} D}{\partial \sigma y}} & {\frac{\partial^{2} D}{\partial y^{2}}} & {\frac{\partial^{2} D}{\partial y x}} \\ {\frac{\partial^{2} D}{\partial \alpha x}} & {\frac{\partial^{2} D}{\partial y x}} & {\frac{\partial^{2} D}{\partial x^{2}}} \end{array}\right]\left[\begin{array}{l} {\sigma} \\ {\frac{\partial D}{\partial \sigma}} \\ {\frac{\partial D}{\partial y}} \\ {\frac{\partial D}{\partial x}} \end{array}\right]\)       (12)

Derivatives approximated by finite differences, example:

\(\begin{aligned} &\frac{\partial D}{\partial \sigma}=\frac{D_{k+1}^{i, j}-D_{k-1}^{i, j}}{2}\\ &\frac{\partial^{2} D}{\partial \sigma^{2}}=\frac{D_{k-1}^{i, j}-2 D_{k}^{i, j}+D_{k+1}^{i j}}{1}\\ &\frac{\partial^{2} D}{\partial \sigma y}=\frac{\left(D_{k+1}^{i+1, j}-D_{k-1}^{i+1, j}\right)-\left(D_{k+1}^{i-1, j}-D_{k-1}^{i-1, j}\right)}{4} \end{aligned}\)       (13)

If X is > 0.5 in any dimension, process repeated.

 

3.1.3. Filtering

Contrast can be described as:

\(D(\hat{\mathrm{X}})=D+\frac{1}{2} \frac{\partial D^{T}}{\partial \mathrm{X}} \hat{X}\)       (14)

If | D(X) | < 0.03, throw it out. To find Edgeiness, we usethe ratio of principal curvatures to throw out poorly defined peaks.

Curvatures come from Hessian in the following:

\(H=\left[\begin{array}{ll} {D_{x x}} & {D_{x y}} \\ {D_{x y}} & {D_{y y}} \end{array}\right]\)       (15)

where Ratio of Trace(H)2, sum of the diagonal components Tr(H), and Determinant(H)

If ratio > (r+1)2/(r), throw it out. 

\(\begin{aligned} &\operatorname{Tr}(H)=D_{x x}+D_{y y}\\ &\operatorname{Det}(H)=D_{x x} D_{y y}-\left(D_{x y}\right)^{2} \end{aligned}\)       (16)

It can be understood as simple expressions:

\(T r(H)=\alpha+\beta, \quad \operatorname{Det}(H)=\alpha \cdot \beta\)       (17)

 

3.1.4. Orientation assignment and descriptor

Based on these steps, descriptor computed relative tokey point's orientation achieves rotation invariance.

Precomputed along for all levels and it is the most stableresults, gradient magnitude: m(x,y) and orientation is precomputed using pixel differences,

\(\begin{aligned} &m(x, y)=\sqrt{(L(x+1, y)-L(x-1, y))^{2}+(L(x, y+1)-L(x, y-1))^{2}}\\ &\theta(x, y)=a \tan 2((L(x, y+1)-L(x, y-1)) /(L(x+1, y)-L(x-1, y))) \end{aligned}\)       (18)

The descriptor has three dimensions and multiple orientations has assigned to keypoints from an orientation histogram of gradient magnitudes.

 

3.2. Optical data collection

We used special vehicles, each mounted with a fish-eyecamera lens facing straight up into the sky and as well assix high-definition cameras to obtain images of the surrounding area. We used this system to record images of the urban environment. The images and aerial photosobtained were used to produce a 3D map. Rather than placing the images of buildings and other structures on asimple flat map surface, we superimposed the images on a 50-m mesh elevation map published by the Geospatial Information Authority of Japan. Showing equally spaced grids to indicate area units of the same shape or size, meshmaps are generally used for quantitative analyses of varioustopographical information or to record a series of numerical data for each area unit. We used the vehicle's GPSpositional data to obtain positional information corresponding to each image and obtained data on direction of travel from a GPS gyro. Our methods for producing the map information and converting location information tocoordinates are based on the Numerical Map User's Guide.1

Using these methods and software that was created by the authors, a 3D map of an area in Yokohama City was completed (Figure 1, 2). Still pictures of two or more sheets are continuously obtained from the moving vehicle (Figure 3). Each picture has spatial correlation and it becomes aspatial noise reduction by adding pictures.

 

Fig. 1. Area map.

 

Fig. 2. Aerial photo.

 

Fig. 3. Still picture of two or more sheets continuously are obtained from moving vehicle.

 

Fig. 4. Mounted equipment on the roof of the vehicle, HDTV camera, GPS gyro and satellite tracking antenna.

 

Fig. 5. Motion pictures from six sets of HDTVs and fish-eyecamera carried on the roof of the vehicle and NTSC output from the spectrum analyzer showing the received signal of the satellite beacon.

 

Table 1. Procedures for acquiring and processing images.

Figure 4 shows the mounted equipment on the roof of thevehicle, HDTV camera, GPS gyro and satellite tracking antenna, and Figure 5 displayed the motion pictures fromeach camera and NTSC image from spectrum analyzer. Every motion pictures were recorded by Panasonic HDCV video recorders. Table 1 lists the procedures for obtaining and processing images of an urban environment. For the purposes of our research, we prepared two special vehicles. Three dedicated staff members worked for two full years to acquire and process the images.

Finally, we can create the 3D map of the Yokohama Yamashita area with over 450 or more polygons. Thesepolygons were put on 50m mesh map of the Geographical Survey Institute, and 3D map was completed. Figure 6 shows the resulting prototype of the 3D map of Yokohama Yamashita Park and its vicinity.

 

Fig. 6. Created 3D Map of Yokohama.

 

3.3. Mobile satellite signal

Geostationary satellite is the term for a communication/broadcasting satellite that remains at acertain orbital altitude above a specific point on the Earth atall times. They orbit in synchronization with the surface of the Earth at approximately 36,000 km above the equator. They are called geostationary because they appear fixed in the sky when viewed from the ground. One geostationarysatellite can cover the whole nation. However, there is amajor technological issue posed by the limited transmission power of a moving vehicle such as ambulance at urban area. This is the Blocking by buildings occurs because Japan is located at mid-latitude, not at the equator.

We have intended to monitor the beacon from N-Star of NTT DoCoMo on S-band. The block diagram is show in the Figure 7. This antenna system can track the satellite based on the GPS gyroscope (GPS gyro). The GPS gyro is made by Furuno Ltd. in order to provide Azimuth direction of moving vessel on the sea surface. It has three points antennas of GPS with 1m-triangle and has calculated the azimuth direction by having obtained the time lag(delay) of the GPS signals. Based on the azimuth data of the GPS gyroand the location data of the GPS, this tracking system canautomatically beam 12dBi antenna to the satellite. Under the environment of line-of-sight propagation at the urbanarea, we can record microwave blockings generated between the vehicles and GEO satellite [16-18].

 

Fig. 7. Block diagram of the tracking system.

 

3.4. Comparison of actual optical blocking with fish-eye camera and simulation by polygons

With a fish-eye lens, the vehicle runs a comparativelylarge way in Yokohama, assumption direction of the satellite will be south (Az = 180 degrees), the actual optical relation of elevation angle were shown in Figure 8: Top. In actual blocking, the building form considered to be the first floor and the second floor is observed in a curve.

Mr. Karasawa’s formula (approximate expression) of the elevation angle and visibility is provided in Table 2. The curve currently recorded has correlation in the curve of the Karasawa's sub urban formula comparatively. On the other hand, in 3D map created with polygons, the form difference of the building of the first floor and the second floor has disappeared (Figure 8: Bottom). It is because that polygons of buildings are expressed as a simple cube (matchbox) regardless of the portion which has pushed out to the roadside of the first floor.

 

Fig. 8. Relationship between optical elevation angle and visibility (azimuth 180 degree) by fish-eye camera and by 3D map with polygons

 

Table 2. Karasawa’s approximation by radio propagation.

 

IV. DISCUSSIONS

 

4.1 Discussion

 

4.1.1 Communication profile

During a disaster, there are two types of communication profiles recommended to perform smooth contact: ground wave communications and satellite communication. The propagation of line-of-sight is indispensable for the latter.

The latter will be predict beforehand with 3D-map with polygons. Based on this study, we can provide a satellitecommunications profile in Yokohama Yamashita area infuture. However, the ground wave communications aresynthesized by multipath waves reflected from buildingsand artificial things. Composition of these reflective wavesmake frequency-selective fading channels, i.e., nonlinear propagation environment. On the other hand, assumingreflected waves of relatively high frequency (e.g., S band frequencies) from all directions, frequency-selective f ading becomes flat fading. This allows relatively accuratepredictions. On the spot, if received power could berecorded beforehand, the reliability of flat fading will beincreased (not phase level but power level).

A communications profile is a system that acquiresinformation from communications links to a moving ground vehicle and compiles a database based on thisinformation. This system then predicts the communicationlinks that will be available at a given location. For line-of-sight communication, optical analysis allows approximation if the frequency is sufficiently high. For non-line-of-sight communication, previously obtainedelectric power data can be used as reference information forapproximations in certain cases when combined with predicted values calculated based on a 3D map.

In case of the terrestrial wave with elevation angle 10 degrees or less, it does not become real use. On the other hand, if the elevation angle for a satellite communicationabout 45 angles, 3D-Map made from the polygon may be able to predict communication as a communication profile.

 

4.1.2. Metadata

The planar map search devices currently used connect to a firefighting/emergency control board and automatically display a map of the target area when the location of the emergency caller or a landmark object is entered. It cannotrender an image that depicts the environment more fully. In contrast, a 3D map shows a visually recognizable three-dimensional image of an emergency helicopter landinglocation or emergency transport route. A 3D map also allows predictions of areas likely to be affected by atsunami immediately after an earthquake. Personal data, including telephone numbers, can be displayed on the mapas metadata (Figure 9) to help facilitate communications and to help issue instructions for rescues from high-rise buildings.

 

Fig. 9. Metadata will be useful to confirm who he is at the building of the two or more floor.

 

4.1.3. On-road landing of emergency medical helicopters

The first instance of an emergency medical helicopterlanding on a road took place on March 10, 1972. Anemergency medical helicopter was dispatched to the site of a head-on collision between a truck and a sightseeing buson the Kanagawa Prefecture side of the Kobotoke Tunnelon the Chuo Expressway. The accident killed two andinjured 41. The helicopter transported a physician from the Japan Ground Self-Defense Force Camp Ichigaya heliport.

As part of rescue, emergency, and firefighting training for fires that might result from automobile accidents, on-roadhelicopter landing training was performed on August 2, 1985, before the Hanshin Expressway Route 7 Kita-Kobeexpress way entered service. As part of this training, ahelicopter landed near the Zenkai Interchange in NishiWard, Kobe City, for emergency patient transport. Thistraining confirmed that a helicopter could land safely without the main rotor blades extending into oncoming traffic if it landed with its center positioned over the traffic median. In response to an accident on the Higashi-Kanto Express way in the Kanto area in April 2002, an emergency medical helicopter on standby at the Nippon Medical School Chiba Hokusoh Hospital transported a physician to the accident site to treat the injured and then return with them to the hospital.

According to Aviation Law Enforcement Regulations of Japan revised in February 2000, if an emergency medical helicopter from a helicopter transport service company is dispatched to an emergency site in response to a request or notification by a firefighting or other such agency, take of fand landing must proceed in the same way as with helicopters involved in search and rescue missions. This means emergency medical helicopters are permitted to land on public roadways in the event of emergencies. Landing ahelicopter on a road requires an understanding of the three-dimensional positional relationships between nearby obstacles, including buildings and overhead power lines. Overhead power lines are clearly visible when an observerlooks up into the sky; they are much more difficult to identify when an observer peers down against a background consisting of structures and the ground. Given the pressure to complete these missions as quickly as possible, it can beespecially hazardous to make emergency landing simmediately before rain or before sunset. All the secircumstances make research on and the application of methods that provide support for on-road helicopterlandings (and reduce risks in the event of emergencylandings) quite valuable (Figure 10).

 

Fig. 10. Doctor Heli at Tokai university hospital.

 

4.1.4 Predicting tsunamis

Our 3D map is based on a 50-m mesh map published by the Geospatial Information Authority of Japan. The heightinformation provided on this map is accurate, and we believe the map can be used to predict the range of areasaffected in the event of a tsunami.

 

V. CONCLUSIONS

The equipment (six sets of HDTVs, fish-eye camera, satellite antenna with tracking system, and receiving powerfrom the satellite beacon of the N star) mounted on the roof of the vehicle, image data were obtained at Yokohama Japan. From these data, the polygon of the building was actually produced and has arranged on the map of the Geographical Survey Institute of a 50 m mesh. The optical study (relationship between visibility rate and elevationangle) were performed on actual data taken by fish-eye lens, and simulated data by 3D map with polygons. There was nobig difference. The 3D map system with polygons will bevery useful to support ambulatory rescue operations during disaster.