Re: [Paparazzi-devel] Support for multiple communication links

On Thu, Mar 14, 2013 at 11:31 PM, Felix Ruess <address@hidden> wrote:

Hi Cameron,

while I haven't thought about this thoroughly, it sounds like a good way to support multiple links to the same vehicle to me (at least I can't think of a better solution on the spot right now).

Gautier might have something to say about this though...?

Cheers, Felix

On Tue, Mar 12, 2013 at 5:14 AM, Cameron Lee <address@hidden> wrote:

Hello again everyone,

I've started working on some of the coding. I've changed my plan slightly though, to make it a simpler change. The new plan is this:

Just like before, to use multiple links you'll run multiple link processes, so that minimal changes to link.ml are necessary. I'll get each link process to send ivy messages that will be received by a new process (and only by this process), a so-called "link combiner". The link combiner process listens to the ivy messages from each link instance and outputs ivy messages to the server, etc... as if it was a link. This way, the Server, logging, and other processes don't need to be changed at all. They only see one link, the combined link, as output by the "link combiner".

Here's a simple diagram:

Old method: serial port → link → server → GCS

New method: serial port 1 → link →→→↓

serial port 2 → link → →link combiner → server → GCS

:
:
serial port n → link →→→↑

Then, in order to get the GCS to display data regarding the status of each link, I'll be changing the message with this info (TELEMETRY_STATUS I think) to support multiple links. So this will involve some changes to the GCS.

Some details:
The link agent will have an additional option, in order to set it's unique name. If this option is set (ex. link -d /dev/ttyUSB0 -s 57600 -name link_1) then the link will transmit over the ivy bus in the slightly different format, which is listed to by the link combiner only, not the server. If it's not, then link will act like it does right now.

The old ivy message format is (for example):
1 DOWNLINK 0 626 1212 (where "1" is the aircraft id)
The new ivy message format for going between the link and the link combiner will be (for example):

1 link_1 DOWNLINK 0 626 1212 (where "1" is the aircraft id and "link_1" is the link name).
This new message format isn't matched by the regex used by the GCS or messages agent. I think it's safe to use, but this is a week point of this plan.

I'll write the link combiner in Python (what I'm familiar with, although I'm starting to be able to read OCaml).
The link combiner will perform it's best guess at removing repeat messages, but it might not be perfect. I plan on filtering out messages that are identical to messages that came in through another link within the past x seconds. A typical x could probably be 2. For the ground to plan redundancy on the other hand, I'll be making sure it eliminates repeat messages 100% reliably. This will necessitate adding a serial number or time-stamp to outgoing messages.

The main advantages are:
1. Keep the power, flexibility, and features of the link agent intact (for example, you'll be able to have a link using UDP and another using a serial port at the same time).

2. Backwards compatible with the current single link system (I.e. you could run the slightly modified link, server, and GCS agents just like you used to without any issues).
3. Limited agents need changing, so it won't break everything.

4. You don't have to send the same messages over each link. You could send some data over just one link and not the other.
5. Most programming is in Python, so I'll actually be able to do it.

The main disadvantages are:
1. Lots more data going over the ivy bus. It could accidentally be listened to by processes that shouldn't listen to it.
2. When the link agents are talking to the redundant link agent, it'll be OCaml code talking to Python code. Not a big deal, but to make changes to how this works, you'll need to change two sets of code. So it's not following DRY (Don't Repeat Yourself), and requires knowledge of both languages.

Please let me know if you have any questions, suggestions, or concerns.

Cameron Lee

On Wed, Feb 27, 2013 at 9:12 PM, Cameron Lee <address@hidden> wrote:

So I've started my project to add support for multiple links in order to provide redundancy. So far I've mostly been looking at the code and trying to understand the system. I think I've got a good grasp of things and am going to start implementing my changes. I have a few questions though.

So my plan is to do pretty much what Chris suggested. The Link process will have a command line argument to specify it's id/name and so one Link process can be run for each link. The messages sent over the ivy bus to the server will contain this link id and the other ground segment processes will all need to be changed to support this change. This way, the GCS can display the status of each link, the messages process can display messages of both links, and many other advantages.

My questions for right now are:
1. The format of the messages sent over the ivy bus by the Link process seems to be (I'm using ivyprobe to look at them):
<Process_name> <Message>

where Message is:
<aircraft_id> <message_name> <value1> <value2> <value3> ...
for example:
Link sent '1 DOWNLINK_STATUS 98 96 6 610 0.0 0.0 -1362023310933.28'

What I'm wondering is, where in here should I include the link id? Should I change the process name? ex:
Link_primary sent '1 DOWNLINK_STATUS 98 96 6 610 0.0 0.0 -1362023310933.28'

or add a new field before the message name? ex:
Link sent '1 primary DOWNLINK_STATUS 98 96 6 610 0.0 0.0 -1362023310933.28'

I'm going to start looking into how the server reads these messages to try and answer my own question, but any advice would be great.

2. The processes that I know I need to change are: Link, GCS, Server, Messages, Messages (Python), Real Time Plotter. Are there others? I figure that the logging (in the Server process, right?) and Log File Player probably need to be changed as well. Anything else? Also, the above is the order I would start changing things.

Thanks for any help,

Cameron

On Mon, Oct 29, 2012 at 11:05 PM, Cameron Lee <address@hidden> wrote:

Thanks for your comments Chris. I haven't looked into this much yet so I'm not sure exactly what will need to be done. There will be a significant learning curve for me, which is partly why I'm asking for advice and suggestions.

I heard that there is a new messaging system being developed. Would this change things at all? Should I look at adding this functionality to the existing system, or the new system?

Cameron

On Sun, Oct 28, 2012 at 9:32 PM, Chris Gough <address@hidden> wrote:

Hi Cameron,

On Mon, Oct 29, 2012 at 11:52 AM, Cameron Lee <> wrote:
> I mean two links from the autopilot to the GCS but only one link from the
> GCS to the autopilot. New software at the GCS would listen to both incoming
> links and pass the data on to the GCS itself if either incoming link has
> valid data. The new software I would write for the autopilot wouldn't be
> used in this setup.

Ah, I was getting things mixed up.

Rather than making a "virtual link" that aggregates the modems behind
another process, have you looked into modifying the existing server,
GCS and link programs to support multiple links natively? Maybe there
are already features there I don't know about - the comments in
link.ml tantalisingly suggest that multiple concurrent links are
supported, but reading through the code it's not clear to me how
(link_id is always 0?).

server.ml sends TELEMETRY_STATUS and TELEMETRY_ERROR messages with an
aircraft_id but not any kind of link_id. Either these messages would
need to have a link_id added, or some new messages would be required.

The GCS would need to be modified to do something with the multiple
links described in TELEMETRY_* messages, and I think link.ml would
need to be modified to get/use a link_id, maybe assigned by a new
command line option

Maybe some new TELEMETRY_INFO messages ("link 2 down" etc). It would
be nice to know how many links there are to a given aircraft, and/or
how many aircraft are visible on a certain link.

Is this a sensible approach (anyone)?

> What I was wondering most is if the Lisa M could easily be set up to allow
> multiple links.

I suspect it could be, but am not the best person to help with that.

Chris Gough

> On Oct 28, 2012 6:34 PM, "Chris Gough" <address@hidden>
> wrote:
>>
>> > Whatever the case, for us, the primary use would be to have two links
>> > from
>> > the autopilot the the groups station but only one from the ground
>> > station to
>> > the autopilot. We can split the Tx of the TWOG in two and send one over
>> > 900
>> > MHz and the other over WiFi.
>>
>> Do you mean only one link (from GCS->autoplot) at a time, but with the
>> ability to change? Otherwise, like you said, you can just splice Tx
>> from the autopilot to both modems (and connect autopilot Rx from the
>> one you chose).
>>
>> > Also, future hardware could have more UARTs for this, couldn't it?
>>
>> I guess so.
>>
>> Chris Gough
>>
>> > Cameron
>> >
>> > On Oct 28, 2012 5:20 PM, "Chris Gough" <address@hidden>
>> > wrote:
>> >>
>> >> Hi Cameron,
>> >>
>> >> If you going to add multiple serial modems directly to an existing
>> >> autopilot, you might need to find a way to free up some serial IO (i2c
>> >> GPS?).
>> >>
>> >> We did this by cheating; inserting a linux computer between the
>> >> autopilot and modems. This was required because our ubiquity link
>> >> (bullet/rocket) is only accessible via ethernet, but it would allow an
>> >> arbitrary number of serial modems to be added (we use one or two).
>> >>
>> >> It's not a very elegant solution though (or cheap, or light, or
>> >> compact), ethernet/usb connectors don't tolerate vibrations very well
>> >> so the whole thing requires a lot of hot glue to make it reliable.
>> >>
>> >> Chris Gough
>> >>
>> >>
>> >> On Mon, Oct 29, 2012 at 6:58 AM, Cameron Lee <address@hidden>
>> >> wrote:
>> >> > Hello everyone,
>> >> >
>> >> > I'm a fourth year Electrical Engineering student interested in
>> >> > working
>> >> > on an
>> >> > aspect of Paparazzi for a class of mine. I'm planning on adding
>> >> > support
>> >> > for
>> >> > multiple communication links at both the GCS and on the autopilot.
>> >> > Similar
>> >> > to item 6 in the wishlist
>> >> > (http://paparazzi.enac.fr/wiki/Software_Wish_List):
>> >> >
>> >> >
>> >> > The possibility to use multiple ground modem connected to a single
>> >> > ground
>> >> > station. The RSSI could be use to dynamically choose which currently
>> >> > has
>> >> > the
>> >> > best signal. This would allow the use of different antennas on each
>> >> > of
>> >> > the
>> >> > modems or have antenna pointing in different directions(?Possibly
>> >> > more
>> >> > hardware related)
>> >> >
>> >> >
>> >> >
>> >> > Here's a description I've written up:
>> >> >
>> >> > The goal is to improve the open source Paparazzi autopilot by adding
>> >> > support
>> >> > for multiple communication links to provide redundancy and increased
>> >> > flexibility. Currently, a single bi-directional serial data link
>> >> > enables
>> >> > the
>> >> > autopilot to provide telemetry to the ground station and the ground
>> >> > station
>> >> > to provide commands to the autopilot. This serial data link is
>> >> > usually
>> >> > created using RF radios and if it’s lost for any reason, all
>> >> > communication
>> >> > with the autopilot is lost. If two data links can be created, then
>> >> > communication can be maintained even if one of the links is lost.
>> >> > Typically,
>> >> > the two links would be of different varieties: two different
>> >> > frequencies, or
>> >> > a short-range high-throughput link and a long-range low-throughput
>> >> > link,
>> >> > or
>> >> > the same type of radio, but with different types of antennas, other
>> >> > combination.
>> >> > This project will involve enabling the autopilot and the ground
>> >> > station
>> >> > to
>> >> > each transmit their data through multiple links as well as receive
>> >> > data
>> >> > through multiple links. Receiving data through multiple links will
>> >> > involve
>> >> > deciding which link to take as correct based on the data coming in
>> >> > through
>> >> > all available links.
>> >> >
>> >> >
>> >> >
>> >> > Before I start working on this, I figured that I should ask for
>> >> > feedback
>> >> > -
>> >> > is this functionality indeed useful? Is this the best way to go about
>> >> > it? Is
>> >> > this project achievable (in particular on the autopilot side - will
>> >> > it
>> >> > take
>> >> > too much system resources)? Any tips or advice would be appreciated.
>> >> > Also,
>> >> > how difficult would it be to have different telemetry sent out over
>> >> > each
>> >> > link?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Cameron
>> >> >
>> >> > _______________________________________________
>> >> > Paparazzi-devel mailing list
>> >> > address@hidden
>> >> > https://lists.nongnu.org/mailman/listinfo/paparazzi-devel
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> .
>> >>
>> >> _______________________________________________
>> >> Paparazzi-devel mailing list
>> >> address@hidden
>> >> https://lists.nongnu.org/mailman/listinfo/paparazzi-devel
>> >>
>> >
>> > _______________________________________________
>> > Paparazzi-devel mailing list
>> > address@hidden
>> > https://lists.nongnu.org/mailman/listinfo/paparazzi-devel
>> >
>>
>>
>>
>> --
>> .
>>
>> _______________________________________________
>> Paparazzi-devel mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/paparazzi-devel
>>
>
> _______________________________________________
> Paparazzi-devel mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/paparazzi-devel
>

--
.

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

From:	Felix Ruess
Subject:	Re: [Paparazzi-devel] Support for multiple communication links
Date:	Thu, 14 Mar 2013 23:32:32 +0100