Table of Contents
Section data_source Data sources
(KML) is an XML file format for geographic visualization in two-dimensional maps such as Google Maps and three-dimensional earth browsers such as Google Earth or Marble.
Fldigi can generate data with geographical locations, which can be used to generate KML data. This list might expand in the future
- The emitting station of a Navtex message.
- The origin of a SYNOP weather report.
- The Maidenhead locator of the user, as entered in fldigi user's profile.
KML generation from Navtex messages
Each Navtex message comes with the code of the sending station, also called origin.
These messages are displayed, in KML files, at the coordinates of the sender. That is: KML placemarks are created or updated with these coordinates. Fldigi parses the Navtex reports, uses the station identifier to make a lookup in the Navtex stations file which contains geographical coordinates. These coordinates are used to create KML placemarks.
More explanation about how station coordinates are used, are given at the Navtex page.
Navtex messages are quite often sent with embedded coordinates of the event they describe (Ship wreck, oil exploration etc...). For example: "LIGHT BUOY MARKING DANGEROUS WRECK 58-01.2 NORTH 005-27.1 WEST" or "THREE MEN OVERBOARD IN PSN 39-07,7N 026-39,2E" . A future version will parse the content of the message, extracting raw coordinates, and will display a graphic entity at the location of the described event.
KML generation from SYNOP reports
SYNOP is a code used for reporting weather information and as such, is used to broadcast meteorological data by radio. One of the most important emitter is Deutsche Wetterdienst which transmits them in RTTY, and fldigi is able to decode them and generate KML placemarks at the location of the weather information.
KML files structure
The KML data are made of different files
fldigi.kml | Entry point. Only this one has to be loaded. It never changes. |
styles.kml | KML style sheet. Freely changeable by the user, for example to customize the icons. |
User.kml | Location of the user based on his/her Maidenhead locator. |
Synop.kml | Synop weather reports displayed at the location of the WMO station, or ship, or buoy. |
Navtex.kml | Navtex reports, displayed at the place of the emitting station. A future version will plot the position of the coordinates indicated in the Navtex messages themselves. |
Extended data
When creating a new placemark, written in of the KML data files (Synop.kml, Navtex.kml etc... ) data are sent to the KML module in the form of key-value pairs and are written into two forms:
-
HTML content, in the
<description>tag, surrounded byCDATAdirectives. The HTML format is chosen exclusively for display purpose and might change at any new version. -
Regular
<ExtendedData>XML tags: These data are internally used by Fldigi to reload the previous session. The format is stable and can be used by external applications. All useful data are saved.
Parameters
Fldigi maintains in a internal container, a set of placemarks which are data associated to geographical coordinates, an unique name, a set of key-value pairs and a timestamp. At regular intervals, a thread is woken up to save these geographical data to a KML file, in a specific directory. At this moment, a process can be started, running an external command. Depending on the type of data, a given file name will be used.
All KML files are accessible from an unique KML filename. Placemarks are identified with an unique name, for example a vessel name, or their WMO identifier. Placemark with a moving position such as ships, can have their path visualized because they still cen be identified in two different reports. These reports can be kept as separate, or they can be merged into a single placemark: This depends on the distance between two placemarks with the same name, compared to the merging distance parameter.
Data can be kept for a given retention time, after which delay they are purged. At startup, former KML data can be reloaded, or cleaned up. Data as key-value pairs associated to a given placemarks can be displayed several ways.
All these parameters are controlled by the KML configuration tab.
Destination directory
The default destination directory where KML files are saved is a subdirectory called /kml in the fldigi users directory. For example on Linux: $HOME/.fldigi/kml/ and <defaultpath>/fldigi.files/kml on Windows™. This destination can be freely changed.
The file fldigi.css is created at installation, and is not changed later. Therefore it is possible to customize it by adding specific icons.
The file fldigi.kml is created by fldigi when it is not there, or when the refresh interval is changed.
If this destination directory is accessible from the internet, then it can be published to Google Maps.
Note:
Files updates are atomic. This means that a file is not accessible by a reader until it is completely written and closed. This is achieved by writing into temporary files, which are atomically renamed (POSIX function rename() ) at the end of operation.
Therefore, the KML destination directory can safely be accessed by one writer and multiple readers. Several sessions of fldigi might also updates different KML files, as long as the main fldigi.kml file is not changed.
KML root file
This is the default name of the entry file of the generated KML document, which by default is fldigi.kml. If it does not exist, it is generated with the list of possible source of KML data (Synop, Navtex etc...). If Google Earth or Marble are installed on your machine, then they are associated to the file extension .kml and you just need to click on fldigi.kml to visualize it. It is automatically refreshed when fldigi adds new Synop weather reports or Navtex messages to it.
KML refresh interval
This delay, in seconds, is used at two places:
- This is the frequency at which new KML files are created, if new data is available
- This is the refresh interval specified in the KML file with tag <refreshInterval>.
This should not be too small, especially if the data files are big, otherwise fldigi will spend most of its time refreshing KML data, and accordingly Google Earth or Marble, reloading them.
Cleanup on startup
By default, at startup, fldigi reloads the existing KML files, extracting the key-value pairs contained in the "ExtendedData" tags. However, it is possible to force fldigi to restart from scratch.
Merging distance
Different reports with the same placemark name can be merged into a single report if their distance is below a given threshold which is the merging distance. Otherwise, separate placemarks are created and joined by a red line, visible in the KML document.
KML balloon display type
Reports are inserted in the KML document one after the other. These description data are visible as KML balloons, or when getting placemark properties. If they have the same name and are within the merging distance, they will form a single placemark. The descriptions of each report will be displayed and merged by three possible ways.
Plain text
Description are inserted without any HTML formatting. Only special HTML entities such as ampersands are reformatted. This is especially useful if the KML document is later converted to GPX, because many GPS devices are not able to display HTML data.
HTML tables
Each description of placemark is transformed into a HTML table labelled with the time stamp of the insertion. Here is an example of two Navtex messages from the same station at different times:
| 2013-02-14 23:18 | |
| Callsign | OST |
| Country | Belgium |
| Locator | JO11JE |
| Message number | 35 |
| Frequency | 0 |
| Mode | TOR |
| Message | 191533 UTC NOV ; WZ 1196 SELF CANCELING. CANCEL WZ 1192 (GA92) (MA33). WALKER LIGHTBUOY NORMAL CONDITIONS RESTORED." |
| 2013-02-14 23:13 | |
| Callsign | OST |
| Country | Belgium |
| Locator | JO11JE |
| Message number | 35 |
| Frequency | 0 |
| Mode | TOR |
| Message | ... etc ... |
Distinct HTML matrix
For the same KML placemark, the key will typically the same for all reports. More, some data are numeric. This is therefore convenient to group them in matrices:
Here is an example for SYNOP weather data, made of three reports:
| 2012-12-16 00:00 | 2012-12-17 06:00 | 2012-12-18 00:00 | |
| Dewpoint temperature | Undefined | Undefined | |
| Figure | 11 | ||
| Humidity | Unspecified | ||
| Precipitations | Omitted, no observation | Omitted, no observation | |
| Pressure change | Not specified | Not specified | Not specified |
| Sea level pressure | 994 hPa | 1000 hPa | 1013 hPa |
| Ship average speed | 0 knots | 0 knots | |
| Ship direction | Calm | Calm | |
| Station type | Automated station. No observation (No 7WW) | Automated station. No observation (No 7WW) | |
| Temperature | 9.5 deg C | 9.3 deg C | 10.3 deg C |
| Value | 37 | ||
| Visibility | 4 km | 4 km | |
| Wave height | 3.6 meters | 4.7 meters | |
| Waves height | 3.5 meters | 4.5 meters | |
| Waves period | 8 seconds | 8 seconds | |
| Wind direction | 265 degrees | 275 degrees | |
| Wind speed | 33 knots (Estimated) | 15 knots (Estimated) |
Data Retention Time
Data may be automatically purged based on their time-stamp and a maximum retention time in hours. If the retention time is zero, then data are kept for ever.
Command run on KML creation
This command is executed at regular times, by default 180 seconds, and only if new data was written to any KML files. The first time this command is run, its process id is stored. Next time this command must be run, we check if this process is still running. If yes, no new process is created.
The intention is to handle the same way, programs which should always be running, for example KML visualizers, and on the other hand, one-shot scripts or converters. Typical situations are:
- Starting a program such as Google Earth or Marble, only once per session.
- They will be automatically restarted if they crash, because their process identifier is not present anymore.
- Run as needed conversion programs such as GpsBabel, to another format (GPX). Or a FTP transfer to a remote platform, for inclusion of KML files in Google Maps.
- Accordingly, do not restart this conversion process as long it is not finished (FTP transfers might take long)
Example of commands
FTP Transfer
A new transfer - and a new process - must be initiated at each KML file save. A script is created for this purpose, and the command can be:
fldigi/scripts/ftp_kml_files.sh ftpperso.free.fr MyFtpUserName MyPassword kml
An obvious use is to save these file to a remote machine where they can be accessed with a public URL. This URL can then be given as CGI parameter to Google Maps which will display the placemarks on a map. There are limitations on the maximum size of KML files which have to be smaller than 10 megabytes.
Note that KML files are for the moment not compressed into KMZ files.
An FTP copy is not necessary if the destination directory for KML files storage is public (That is, accessible from the Internet).
Launch google-earth
The program will only be launched once, because its process id is still present. The command can be:
googleearth $HOME/.fldigi/kml/fldigi.kml
It is possible to change the icons by customizing the file styles.kml.
GPS Babel conversion
The command GpsBabel, for example, will selectively convert the KML file of Synop reports. It is generally advised to generate plain text description tags in the KML files, because GPS devices might not be able to correctly display HTML data. The command can be:
gpsbabel -i kml -f $HOME/.fldigi/kml/Synop.kml -o gpx -F out.gpx
The generated files can for example be fed into Xastir.
