Re: [HamWAN PSDR] Mesh backbone
Hi Craig, First, let's speak the same language. There's a lot of confusion around the word "MESH". I'm also not sure why it's always capitalized. It's not an acronym as far as I know. The term "mesh network" describes nothing more than the logical topology of a network. There's a really good write-up on this at the wikipedia page <http://en.wikipedia.org/wiki/Mesh_networking>. I'm pretty sure that when you (and others on this email) use the phrase "mesh network" it is not the wikipedia definition you're intending. It means something different, including: 1) Using a common RF channel 2) Promiscuous neighbor discovery + association 3) Automatic IP configuration based on MAC 4) Nearly open node authentication 5) Omnidirectional operation 6) WDS <http://en.wikipedia.org/wiki/Wireless_distribution_system>-style operation These are the typical traits of a SeattleWireless <http://seattlewireless.net/> style mesh network (which, BTW, has been attempting to bootstrap itself for the last 13 years). This type of definition of "mesh network" is quite a different animal from the canonical mesh network definition (wikipedia's, derived from network theory). The reason I want to make the distinction clear is that nearly all large networks in the world are indeed mesh networks, but nearly none of them possess the qualities of 1-6. So when I say something like "HamWAN <https://www.hamwan.org/t/tiki-index.php> is a mesh network", I want it to be clear that I'm referring to the network topology only (the wikipedia definition). I'm not sure what is the right word to describe the other type of network; perhaps capitalizing all the letters is a good differentiator after all. :) So, with the definitions out of the way, let me address your actual email now that I can speak to it unambiguously. I concur that we all want to implement a mesh network, but I don't think everyone wants to implement a MESH network. Phrasing the problem of "how do we provide modern digital communications to the ham community" in terms of "how do we implement a MESH network" is putting the cart before the horse. Over the last 6 months, I've been leading the HamWAN effort to create solutions to the first question. We've made respectable progress on both the RF engineering and networking fronts. We had a fully functional cell site setup @ last weekend's Flea Market. This site design is about to start rolling out to the real world. This transition in the project's status has allowed me to start thinking about how HamWAN might integrate with other ham networking efforts. We've had a good relationship with BCWARN.net <http://www.bcwarn.net/intermapper/rf-map.html> for the last few months, and integration with that network will be simple. The physical links are already planned in fact! We're both excited to make it happen and start exchanging traffic internationally! The integration with NW-MESH efforts is far more challenging. In fact, it may be outright impossible unless changes are made in the NW-MESH design. From what I've seen, this difficulty of peering will be present in all MESH networks. The problems range from simple route exchange, to address space conflicts, to policy propagation (access, QoS, filtering, etc). I don't even wanna think about DNS. :P Having said all that, there is some value to HamWAN using a MeSh (hybrid of mesh and MESH) layer. Traffic between nodes may flow more optimally on the ground than through the mountain sites. A nearby ham who doesn't want to invest in a dish might get on HamWAN by MeShing with his neighbor. I'd be good for the health of HamWAN to make use of these optimizations. But like I said, in order for these routing decision to be made correctly and automatically, NW-MESH designs will need to change. I'd like to invite you (in fact, all of you) to join the HamWAN weekly meeting today @ 7PM. I'll re-send the connectivity details on the mailing list (email: psdr-join@hamwan.org) an hour before the meeting, but basically install Mumble <http://mumble.sourceforge.net/> 1.2.4 <http://mumble.info/snapshot/mumble-1.2.4-rc1-8-gb115a29.msi>+ (currently beta), and connect to BartK.us. Please use a headset to avoid generating echoes. Craig, can you give me an idea of your skills? Perhaps you would enjoy solving these types of problems as part of the HamWAN development team? We run a tight ship with specific assignments and weekly reporting. I believe this is the "small group of experts" approach you were proposing. :) --Bart On 03/12/2013 12:09 PM, Craig B wrote:
When I first heard about the MESH project from Daniel Stevens (KL7WM) back in late fall 2012, the first question I had for him was "what will it connect to?" Since then, as I have become more involved I have started to formulate what I think it could be connected to and how it could be used.
Based on what I have learned and seen to date, I see 3 tiers of network involved here. The backbone, which I see as a long-haul that would be based on a region that is defined by terrain and distance. The middle tier would be smaller and could be between HAM towers or other "secondary" sites. The 3rd tier would be for the "neighborhood" or "home" MESHing with WRT's and other low-power devices. In this type of configuration, I see the backbone as being the one common piece across regions while the secondary and tertiary tiers could be specific to the 'regional' implementation. Each tier would have to bridge from itself to the next level, which seems to be reasonable where a given site could choose to bridge by adding necessary hardware or remain remote.
What I would like to do is see if we can't get a written network plan for a regional backbone and then any additional tiers that need to be included in a good network design document. I am a firm believer that it should be hardware agnostic for the most part, although could provide a list of acceptable components that have been shown or believe to be the best hardware based on application. It would also dictate how traffic might be handled moving up/down through the tiers, possibly allowing for QoS or other transport methods.
As I am sure we all want to see a MESH network available to all HAMs; given the area NW-MESH has been getting feedback on, I think we need to start looking at how we connect them all together. As such, in talking with Bob Rutherford, it seems like the first step is to build out a plan that could be presented to FWARC, Tukwila Radio Club, and EMCOMM (and other clubs/groups).
Since this is all great in theory, it seems the next step is to secure funding and move it from paper to reality. Given the real application of this MESH network for EMCOMM, and their generally deep pockets, it seems like a great way to get a backbone built.
If we are going to design this, I believe it needs to be initially designed by a small group of network and radio experts. Once an initial plan is cobbled together it could be released to the larger MESH community for comments and additions/subtractions, etc.
I am willing to take on leading this charge; however, I will need a team of experts behind me helping lay the ground work. My initial thought is to have something ready by mid or late summer, given that we all have other priorities as well, not adverse to taking longer if it means having a complete and well planned out design.
Any thoughts on this?
Thanks, Craig KF7LLA
Quick note before I get dragged back into work.... Bart is correct that MESH has been overloaded as both a name for a topology and a proper noun for a specific implementation of a MANET. As long as we know what we're talking about, we can use either term. We can call it MANET which is technically accurate, or MESH since that is commonly used and understood. I view the NW-MESH and HamWAN projects as complimentary. HamWAN in general and the PSDR component in particular makes an excellent backbone transport network to move data around the wider area. NW-MESH works well to cover the last mile. Full bidirectional interconnection of the networks has some significant challenges as Bart states. They are surmountable, but there are some issues to address. On the other hand, using the PSDR as a transport means for interconnecting pockets of NW-MESH is not terribly difficult and can be done rather simply. I greatly prefer to keep the projects separate and have a collaboration on the interface between the two efforts outside the mainline work of either. The projects have differing goals, differing timelines, and differing concepts. I'd like to see some more detail from Bart on what he sees as the large issues with bidirectional interconnection (I think I know most of them) or simply using the PSDR as transport via tunneling between access nodes. 73-KY9K/Brian On Tue, Mar 12, 2013 at 2:16 PM, Bart Kus <me@bartk.us> wrote:
Hi Craig,
First, let's speak the same language.
There's a lot of confusion around the word "MESH". I'm also not sure why it's always capitalized. It's not an acronym as far as I know. The term "mesh network" describes nothing more than the logical topology of a network. There's a really good write-up on this at the wikipedia page. I'm pretty sure that when you (and others on this email) use the phrase "mesh network" it is not the wikipedia definition you're intending. It means something different, including:
1) Using a common RF channel 2) Promiscuous neighbor discovery + association 3) Automatic IP configuration based on MAC 4) Nearly open node authentication 5) Omnidirectional operation 6) WDS-style operation
These are the typical traits of a SeattleWireless style mesh network (which, BTW, has been attempting to bootstrap itself for the last 13 years). This type of definition of "mesh network" is quite a different animal from the canonical mesh network definition (wikipedia's, derived from network theory).
The reason I want to make the distinction clear is that nearly all large networks in the world are indeed mesh networks, but nearly none of them possess the qualities of 1-6. So when I say something like "HamWAN is a mesh network", I want it to be clear that I'm referring to the network topology only (the wikipedia definition). I'm not sure what is the right word to describe the other type of network; perhaps capitalizing all the letters is a good differentiator after all. :)
So, with the definitions out of the way, let me address your actual email now that I can speak to it unambiguously.
I concur that we all want to implement a mesh network, but I don't think everyone wants to implement a MESH network. Phrasing the problem of "how do we provide modern digital communications to the ham community" in terms of "how do we implement a MESH network" is putting the cart before the horse. Over the last 6 months, I've been leading the HamWAN effort to create solutions to the first question. We've made respectable progress on both the RF engineering and networking fronts. We had a fully functional cell site setup @ last weekend's Flea Market. This site design is about to start rolling out to the real world.
This transition in the project's status has allowed me to start thinking about how HamWAN might integrate with other ham networking efforts. We've had a good relationship with BCWARN.net for the last few months, and integration with that network will be simple. The physical links are already planned in fact! We're both excited to make it happen and start exchanging traffic internationally!
The integration with NW-MESH efforts is far more challenging. In fact, it may be outright impossible unless changes are made in the NW-MESH design. From what I've seen, this difficulty of peering will be present in all MESH networks. The problems range from simple route exchange, to address space conflicts, to policy propagation (access, QoS, filtering, etc). I don't even wanna think about DNS. :P
Having said all that, there is some value to HamWAN using a MeSh (hybrid of mesh and MESH) layer. Traffic between nodes may flow more optimally on the ground than through the mountain sites. A nearby ham who doesn't want to invest in a dish might get on HamWAN by MeShing with his neighbor. I'd be good for the health of HamWAN to make use of these optimizations. But like I said, in order for these routing decision to be made correctly and automatically, NW-MESH designs will need to change.
I'd like to invite you (in fact, all of you) to join the HamWAN weekly meeting today @ 7PM. I'll re-send the connectivity details on the mailing list (email: psdr-join@hamwan.org) an hour before the meeting, but basically install Mumble 1.2.4+ (currently beta), and connect to BartK.us. Please use a headset to avoid generating echoes.
Craig, can you give me an idea of your skills? Perhaps you would enjoy solving these types of problems as part of the HamWAN development team? We run a tight ship with specific assignments and weekly reporting. I believe this is the "small group of experts" approach you were proposing. :)
--Bart
On 03/12/2013 12:09 PM, Craig B wrote:
When I first heard about the MESH project from Daniel Stevens (KL7WM) back in late fall 2012, the first question I had for him was "what will it connect to?" Since then, as I have become more involved I have started to formulate what I think it could be connected to and how it could be used.
Based on what I have learned and seen to date, I see 3 tiers of network involved here. The backbone, which I see as a long-haul that would be based on a region that is defined by terrain and distance. The middle tier would be smaller and could be between HAM towers or other "secondary" sites. The 3rd tier would be for the "neighborhood" or "home" MESHing with WRT's and other low-power devices. In this type of configuration, I see the backbone as being the one common piece across regions while the secondary and tertiary tiers could be specific to the 'regional' implementation. Each tier would have to bridge from itself to the next level, which seems to be reasonable where a given site could choose to bridge by adding necessary hardware or remain remote.
What I would like to do is see if we can't get a written network plan for a regional backbone and then any additional tiers that need to be included in a good network design document. I am a firm believer that it should be hardware agnostic for the most part, although could provide a list of acceptable components that have been shown or believe to be the best hardware based on application. It would also dictate how traffic might be handled moving up/down through the tiers, possibly allowing for QoS or other transport methods.
As I am sure we all want to see a MESH network available to all HAMs; given the area NW-MESH has been getting feedback on, I think we need to start looking at how we connect them all together. As such, in talking with Bob Rutherford, it seems like the first step is to build out a plan that could be presented to FWARC, Tukwila Radio Club, and EMCOMM (and other clubs/groups).
Since this is all great in theory, it seems the next step is to secure funding and move it from paper to reality. Given the real application of this MESH network for EMCOMM, and their generally deep pockets, it seems like a great way to get a backbone built.
If we are going to design this, I believe it needs to be initially designed by a small group of network and radio experts. Once an initial plan is cobbled together it could be released to the larger MESH community for comments and additions/subtractions, etc.
I am willing to take on leading this charge; however, I will need a team of experts behind me helping lay the ground work. My initial thought is to have something ready by mid or late summer, given that we all have other priorities as well, not adverse to taking longer if it means having a complete and well planned out design.
Any thoughts on this?
Thanks, Craig KF7LLA
-- ---------- Society is, always has been, and always will be a structure for the exploitation and oppression of the majority through systems of political force dictated by an elite, enforced by thugs, uniformed or not, and upheld by a willful ignorance and stupidity on the part of the very majority whom the system oppresses. - Richard K. Morgan ----------
Everyone take a deep breath before reading the thread. There is conflict ahead, but we knew that. Read it only if you're willing to take the view that what is written is done so under the best possible intentions. On 03/12/2013 03:04 PM, ZPO wrote:
Quick note before I get dragged back into work....
Bart is correct that MESH has been overloaded as both a name for a topology and a proper noun for a specific implementation of a MANET. As long as we know what we're talking about, we can use either term. We can call it MANET which is technically accurate, or MESH since that is commonly used and understood.
I'm glad we agree on at least that. :) Let's stick with MESH in all caps when referring to NW-MESH type stuff, since MANET can also mean a network like HamWAN and we'd once again be ambiguous.
I view the NW-MESH and HamWAN projects as complimentary. HamWAN in general and the PSDR component in particular makes an excellent backbone transport network to move data around the wider area. NW-MESH works well to cover the last mile.
This is where I think HamWAN is being sold short, since it also works great on the last mile. Reason being is HamWAN sites are all planned to be non-ground-level, so LoS probability increases which makes last mile happy. Your trip may be physically longer by a few miles, but the latency and speed will still be good, if not better when compared to multiple ground hops for the same distance. In close-proximity situations, the end-user-site to end-user-site direct connectivity of NW-MESH I believe provides an optimization. A COUPLE hops of MeSh might indeed be superior to traversing a busy sector.
Full bidirectional interconnection of the networks has some significant challenges as Bart states. They are surmountable, but there are some issues to address.
I'm glad we both recognize this fact.
On the other hand, using the PSDR as a transport means for interconnecting pockets of NW-MESH is not terribly difficult and can be done rather simply.
It sure can! But read on to the last point.
I greatly prefer to keep the projects separate and have a collaboration on the interface between the two efforts outside the mainline work of either. The projects have differing goals, differing timelines, and differing concepts.
And I'd greatly prefer to merge the projects, because I'm not convinced that we do have differing goals. If our goals can be communally stated as "provide the amateur radio community with a fast wide-area multi-megabit IP-based digital network that can scale and out-last either of us", then I think we have a chance at merging, because that's pretty much the HamWAN mission statement. For the more detailed version, see the official HamWAN mission statement <https://www.hamwan.org/t/tiki-index.php?page=Constitution&structure=HamWAN>, which is the first article of our constitution. This is where I think you, Rob, and the rest of the NW-MESH team need to get together with the HamWAN people to spell out exactly what it is that each of us is trying to accomplish. Do you have a mission statement you can share? Do you have a favorite low-noise public meeting location with power + wifi? Starbucks is sounding likely.
I'd like to see some more detail from Bart on what he sees as the large issues with bidirectional interconnection (I think I know most of them) or simply using the PSDR as transport via tunneling between access nodes.
So even though it's possible for HamWAN to act as a dumb pipe for NW-MESH, this is not a fair arrangement for HamWAN. If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of. A fair peering agreement is where each network moves equal amounts of data to and from a peer, where said traffic is between clients of each network. This means not via a peer network, and the traffic is from/to clients of the same originating network. With BCWARN, it's easy. We each cover different regions, we each come to the game with our own backbone transport, so we are indeed peers and can exchange traffic as such with no ill feelings. BCWARN also uses OSPF/BGP and public IP addresses, which simplifies matters even more. They also have restrictions in place as to who can participate so we're not carrying questionable traffic on HamWAN. In short, all the bases are covered and we're happy to peer. I'd rather not spell out the NW-MESH incompatibilities with networks like BCWARN and HamWAN in this email, simply because I don't feel qualified. I'll need to ask questions in order to fully explore the subject. This is where I think it makes sense to have a meeting between our groups and hash these technical things out. Off the top of my head, the attributes of using 10/8 space is a problem since it can conflict with people's home networks (and other reasons), as is the access policy since as far as I know any non-ham can just get the key and join a NW-MESH node. Correct me if I'm wrong. Hell, I don't even know if NW-MESH is running part 97 or part 15 rules. What say you? :) --Bart
73-KY9K/Brian
On Tue, Mar 12, 2013 at 2:16 PM, Bart Kus <me@bartk.us> wrote:
Hi Craig,
First, let's speak the same language.
There's a lot of confusion around the word "MESH". I'm also not sure why it's always capitalized. It's not an acronym as far as I know. The term "mesh network" describes nothing more than the logical topology of a network. There's a really good write-up on this at the wikipedia page. I'm pretty sure that when you (and others on this email) use the phrase "mesh network" it is not the wikipedia definition you're intending. It means something different, including:
1) Using a common RF channel 2) Promiscuous neighbor discovery + association 3) Automatic IP configuration based on MAC 4) Nearly open node authentication 5) Omnidirectional operation 6) WDS-style operation
These are the typical traits of a SeattleWireless style mesh network (which, BTW, has been attempting to bootstrap itself for the last 13 years). This type of definition of "mesh network" is quite a different animal from the canonical mesh network definition (wikipedia's, derived from network theory).
The reason I want to make the distinction clear is that nearly all large networks in the world are indeed mesh networks, but nearly none of them possess the qualities of 1-6. So when I say something like "HamWAN is a mesh network", I want it to be clear that I'm referring to the network topology only (the wikipedia definition). I'm not sure what is the right word to describe the other type of network; perhaps capitalizing all the letters is a good differentiator after all. :)
So, with the definitions out of the way, let me address your actual email now that I can speak to it unambiguously.
I concur that we all want to implement a mesh network, but I don't think everyone wants to implement a MESH network. Phrasing the problem of "how do we provide modern digital communications to the ham community" in terms of "how do we implement a MESH network" is putting the cart before the horse. Over the last 6 months, I've been leading the HamWAN effort to create solutions to the first question. We've made respectable progress on both the RF engineering and networking fronts. We had a fully functional cell site setup @ last weekend's Flea Market. This site design is about to start rolling out to the real world.
This transition in the project's status has allowed me to start thinking about how HamWAN might integrate with other ham networking efforts. We've had a good relationship with BCWARN.net for the last few months, and integration with that network will be simple. The physical links are already planned in fact! We're both excited to make it happen and start exchanging traffic internationally!
The integration with NW-MESH efforts is far more challenging. In fact, it may be outright impossible unless changes are made in the NW-MESH design. From what I've seen, this difficulty of peering will be present in all MESH networks. The problems range from simple route exchange, to address space conflicts, to policy propagation (access, QoS, filtering, etc). I don't even wanna think about DNS. :P
Having said all that, there is some value to HamWAN using a MeSh (hybrid of mesh and MESH) layer. Traffic between nodes may flow more optimally on the ground than through the mountain sites. A nearby ham who doesn't want to invest in a dish might get on HamWAN by MeShing with his neighbor. I'd be good for the health of HamWAN to make use of these optimizations. But like I said, in order for these routing decision to be made correctly and automatically, NW-MESH designs will need to change.
I'd like to invite you (in fact, all of you) to join the HamWAN weekly meeting today @ 7PM. I'll re-send the connectivity details on the mailing list (email: psdr-join@hamwan.org) an hour before the meeting, but basically install Mumble 1.2.4+ (currently beta), and connect to BartK.us. Please use a headset to avoid generating echoes.
Craig, can you give me an idea of your skills? Perhaps you would enjoy solving these types of problems as part of the HamWAN development team? We run a tight ship with specific assignments and weekly reporting. I believe this is the "small group of experts" approach you were proposing. :)
--Bart
On 03/12/2013 12:09 PM, Craig B wrote:
When I first heard about the MESH project from Daniel Stevens (KL7WM) back in late fall 2012, the first question I had for him was "what will it connect to?" Since then, as I have become more involved I have started to formulate what I think it could be connected to and how it could be used.
Based on what I have learned and seen to date, I see 3 tiers of network involved here. The backbone, which I see as a long-haul that would be based on a region that is defined by terrain and distance. The middle tier would be smaller and could be between HAM towers or other "secondary" sites. The 3rd tier would be for the "neighborhood" or "home" MESHing with WRT's and other low-power devices. In this type of configuration, I see the backbone as being the one common piece across regions while the secondary and tertiary tiers could be specific to the 'regional' implementation. Each tier would have to bridge from itself to the next level, which seems to be reasonable where a given site could choose to bridge by adding necessary hardware or remain remote.
What I would like to do is see if we can't get a written network plan for a regional backbone and then any additional tiers that need to be included in a good network design document. I am a firm believer that it should be hardware agnostic for the most part, although could provide a list of acceptable components that have been shown or believe to be the best hardware based on application. It would also dictate how traffic might be handled moving up/down through the tiers, possibly allowing for QoS or other transport methods.
As I am sure we all want to see a MESH network available to all HAMs; given the area NW-MESH has been getting feedback on, I think we need to start looking at how we connect them all together. As such, in talking with Bob Rutherford, it seems like the first step is to build out a plan that could be presented to FWARC, Tukwila Radio Club, and EMCOMM (and other clubs/groups).
Since this is all great in theory, it seems the next step is to secure funding and move it from paper to reality. Given the real application of this MESH network for EMCOMM, and their generally deep pockets, it seems like a great way to get a backbone built.
If we are going to design this, I believe it needs to be initially designed by a small group of network and radio experts. Once an initial plan is cobbled together it could be released to the larger MESH community for comments and additions/subtractions, etc.
I am willing to take on leading this charge; however, I will need a team of experts behind me helping lay the ground work. My initial thought is to have something ready by mid or late summer, given that we all have other priorities as well, not adverse to taking longer if it means having a complete and well planned out design.
Any thoughts on this?
Thanks, Craig KF7LLA
Replying to the earlier message (fixed Steve's email) to keep things in-line.... On Tue, Mar 12, 2013 at 4:29 PM, Bart Kus <me@bartk.us> wrote:
Everyone take a deep breath before reading the thread. There is conflict ahead, but we knew that. Read it only if you're willing to take the view that what is written is done so under the best possible intentions.
On 03/12/2013 03:04 PM, ZPO wrote:
Quick note before I get dragged back into work....
Bart is correct that MESH has been overloaded as both a name for a topology and a proper noun for a specific implementation of a MANET. As long as we know what we're talking about, we can use either term. We can call it MANET which is technically accurate, or MESH since that is commonly used and understood.
I'm glad we agree on at least that. :) Let's stick with MESH in all caps when referring to NW-MESH type stuff, since MANET can also mean a network like HamWAN and we'd once again be ambiguous.
MESH it is.
I view the NW-MESH and HamWAN projects as complimentary. HamWAN in general and the PSDR component in particular makes an excellent backbone transport network to move data around the wider area. NW-MESH works well to cover the last mile.
This is where I think HamWAN is being sold short, since it also works great on the last mile. Reason being is HamWAN sites are all planned to be non-ground-level, so LoS probability increases which makes last mile happy. Your trip may be physically longer by a few miles, but the latency and speed will still be good, if not better when compared to multiple ground hops for the same distance. In close-proximity situations, the end-user-site to end-user-site direct connectivity of NW-MESH I believe provides an optimization. A COUPLE hops of MeSh might indeed be superior to traversing a busy sector.
I don't think I'm selling HamWAN short, just looking at the likely timelines to have those cell-type sectors densely deployed and seeing it stretch into years if ever. It is good to have multiple things in the network gene pool. Let the use cases shake themselves out. My thought process is that a few clubs with advantaged sites can form the initial connections from MESHland to the PSDR. I'm being brief, this makes a lot more sense with a whiteboard.
Full bidirectional interconnection of the networks has some significant challenges as Bart states. They are surmountable, but there are some issues to address.
I'm glad we both recognize this fact.
Maybe we can grab a cold beer at SEAPAC and I'll tell you the joys of trying to tie together US and NATO networks in Astan. All kinds of messiness.
On the other hand, using the PSDR as a transport means for interconnecting pockets of NW-MESH is not terribly difficult and can be done rather simply.
It sure can! But read on to the last point.
I greatly prefer to keep the projects separate and have a collaboration on the interface between the two efforts outside the mainline work of either. The projects have differing goals, differing timelines, and differing concepts.
And I'd greatly prefer to merge the projects, because I'm not convinced that we do have differing goals. If our goals can be communally stated as "provide the amateur radio community with a fast wide-area multi-megabit IP-based digital network that can scale and out-last either of us", then I think we have a chance at merging, because that's pretty much the HamWAN mission statement. For the more detailed version, see the official HamWAN mission statement, which is the first article of our constitution.
Breaking between these two paragraphs as this is one of the first spots where the ethos of HamWAN and the MESH programs (the one here and others) tends to differ. There is no "constitution" in MESHland. That is by design. There are sets of interoperability recommendations that can be thought of as barebones versions of the RFCs underpinning the global Internet. I don't particularly care what hardware somebody brings to the party as long as it has the right interfaces and can speak the right protocols. This allows the network to grow organically with each node extending the network a bit more. It also tends to make things very resilient.
This is where I think you, Rob, and the rest of the NW-MESH team need to get together with the HamWAN people to spell out exactly what it is that each of us is trying to accomplish. Do you have a mission statement you can share? Do you have a favorite low-noise public meeting location with power + wifi? Starbucks is sounding likely.
I'd like to see some more detail from Bart on what he sees as the large issues with bidirectional interconnection (I think I know most of them) or simply using the PSDR as transport via tunneling between access nodes.
So even though it's possible for HamWAN to act as a dumb pipe for NW-MESH, this is not a fair arrangement for HamWAN. If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of. A fair peering agreement is where each network moves equal amounts of data to and from a peer, where said traffic is between clients of each network. This means not via a peer network, and the traffic is from/to clients of the same originating network.
This concept probably needs to be fleshed out a bit. I didn't adequately define it in my initial post. Better with a dry erase board.. My goal would be to have every node in MESHland offering services to the wider network do so via a 44-net address which will be advertised as an HNA4 on the network. This is easy to link into HamWAN at interconnection points via BGP. That completely isolates the internal 10.x.y.z/8 2.4GHz addressing (which isn't a viable long-term solution for lots of reasons) from HamWAN. This is a good thing. On the other hand, there are some functions and features which add value to MESHland that require direct communication with OLSR ETX values intact, and routing staying on 10.0.0.0/8. Some folks may choose to not go with 44-net addresses and advertise their services to the wider world. Someone traveling out of area may want their MESH devices to transparently link up with other networks. The statement " If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of" reflects a commercial service provider mindset. This is not such an environment. If the ham community as a whole contributes to fund, build, and support HamWAN then HamWAN needs to accept uses that may not fit its model. I don't expect there will be huge amounts of traffic flowing via tunneled connections, but it is an important feature. What I don't want to see happen is money, site availability, and goodwill expended needlessly operating two parallel networks. There will be compromise and adaptation on both sides. The most important thing in such an interconnection is harmonizing DSCP marking for QoS. If we want to get very granular and complex, DISA UCR 2013 ( http://www.disa.mil/Services/Network-Services/UCCO/UCR-2013-Draft ) provides a detailed setup (section6 - http://www.disa.mil/Services/Network-Services/UCCO/~/media/Files/DISA/Servic... ). In the long run, as long as both domains use the same QoS, things are manageable. I've played the game where DSCP tags are remarked on traffic ingress/egress. It isn't fun and generally ends up breaking. --snipping the BCWARN stuff since it isn't germane --
Hell, I don't even know if NW-MESH is running part 97 or part 15 rules. What say you? :)
In most cases, the difference between Part97 and Part15 is a state of mind. It is only when you start using frequencies outside the ISM/UNII bands or running higher EIRPs on point-to-multipoint links that things become definitely Part97. For example, a 5.8GHz PtP link between two sites running 500mW radios and 26dBi antennas could be considered Part15 or Part97. There may be value in considering it Part15 as we may want to run the link with encryption or perhaps run an encrypted tunnel for a served agency through the link. In such an arrangement, we might get the served agency to fund the link, we install/maintain it, and they get a portion of the available bandwidth as a backup communications channel. That isn't a bad deal for any of the involved parties. Such a setup is being mulled now for one group/area in particular. Probably the best idea is to continue discussions at an exploratory level and then spend some time with a couple dry-erase boards at SEAPAC. I will be there Friday-Monday (driving back Monday). We can enjoy some frosty beverages and dry-erase our way through most of the challenges. 73-KY9K/Brian
I think you're glossing over some pretty significant issues, and hyper-focusing on specific technical ideas. The proposal to kick back, have some beer, and hash everything out in the future is appreciated, but I wonder if you're assuming that HamWAN can sit around waiting for this. We already have hardware specified, timelines for initial deployments set, a great deal of groundwork laid out, and some rather pressing organizational and regulatory issues we're contending with. Please don't take this the wrong way, but we've got quite a bit of personal investment in what's going on. Certainly you can see how another group coming in and saying "Oh yeah, hey, that work you're doing? Yeah, postpone for a while so we can have beer and throw in our own two cents" might be contentious. The peering model is less about commercialism, and more about fairness. It would be terribly unfair for the members of HamWAN to fund and build a solid network only to later have other networks come in and simply use it as a network pipe to serve their own needs without adding any value. Are you suggesting that members of NW-MESH are interested in becoming members of or otherwise funding HamWAN? On Tue, Mar 12, 2013 at 9:35 PM, ZPO <geekdownrange@gmail.com> wrote:
Replying to the earlier message (fixed Steve's email) to keep things in-line....
On Tue, Mar 12, 2013 at 4:29 PM, Bart Kus <me@bartk.us> wrote:
Everyone take a deep breath before reading the thread. There is conflict ahead, but we knew that. Read it only if you're willing to take the view that what is written is done so under the best possible intentions.
On 03/12/2013 03:04 PM, ZPO wrote:
Quick note before I get dragged back into work....
Bart is correct that MESH has been overloaded as both a name for a topology and a proper noun for a specific implementation of a MANET. As long as we know what we're talking about, we can use either term. We can call it MANET which is technically accurate, or MESH since that is commonly used and understood.
I'm glad we agree on at least that. :) Let's stick with MESH in all caps when referring to NW-MESH type stuff, since MANET can also mean a network like HamWAN and we'd once again be ambiguous.
MESH it is.
I view the NW-MESH and HamWAN projects as complimentary. HamWAN in general and the PSDR component in particular makes an excellent backbone transport network to move data around the wider area. NW-MESH works well to cover the last mile.
This is where I think HamWAN is being sold short, since it also works
great
on the last mile. Reason being is HamWAN sites are all planned to be non-ground-level, so LoS probability increases which makes last mile happy. Your trip may be physically longer by a few miles, but the latency and speed will still be good, if not better when compared to multiple ground hops for the same distance. In close-proximity situations, the end-user-site to end-user-site direct connectivity of NW-MESH I believe provides an optimization. A COUPLE hops of MeSh might indeed be superior to traversing a busy sector.
I don't think I'm selling HamWAN short, just looking at the likely timelines to have those cell-type sectors densely deployed and seeing it stretch into years if ever. It is good to have multiple things in the network gene pool. Let the use cases shake themselves out. My thought process is that a few clubs with advantaged sites can form the initial connections from MESHland to the PSDR. I'm being brief, this makes a lot more sense with a whiteboard.
Full bidirectional interconnection of the networks has some significant challenges as Bart states. They are surmountable, but there are some issues to address.
I'm glad we both recognize this fact.
Maybe we can grab a cold beer at SEAPAC and I'll tell you the joys of trying to tie together US and NATO networks in Astan. All kinds of messiness.
On the other hand, using the PSDR as a transport means for interconnecting pockets of NW-MESH is not terribly difficult and can be done rather simply.
It sure can! But read on to the last point.
I greatly prefer to keep the projects separate and have a collaboration on the interface between the two efforts outside the mainline work of either. The projects have differing goals, differing timelines, and differing concepts.
And I'd greatly prefer to merge the projects, because I'm not convinced
that
we do have differing goals. If our goals can be communally stated as "provide the amateur radio community with a fast wide-area multi-megabit IP-based digital network that can scale and out-last either of us", then I think we have a chance at merging, because that's pretty much the HamWAN mission statement. For the more detailed version, see the official HamWAN mission statement, which is the first article of our constitution.
Breaking between these two paragraphs as this is one of the first spots where the ethos of HamWAN and the MESH programs (the one here and others) tends to differ. There is no "constitution" in MESHland. That is by design. There are sets of interoperability recommendations that can be thought of as barebones versions of the RFCs underpinning the global Internet. I don't particularly care what hardware somebody brings to the party as long as it has the right interfaces and can speak the right protocols. This allows the network to grow organically with each node extending the network a bit more. It also tends to make things very resilient.
This is where I think you, Rob, and the rest of the NW-MESH team need to get together with the HamWAN people to spell out exactly what it is that each of us is trying to accomplish. Do you have a mission statement you can share? Do you have a favorite low-noise public meeting location with power + wifi? Starbucks is sounding likely.
I'd like to see some more detail from Bart on what he sees as the large issues with bidirectional interconnection (I think I know most of them) or simply using the PSDR as transport via tunneling between access nodes.
So even though it's possible for HamWAN to act as a dumb pipe for NW-MESH, this is not a fair arrangement for HamWAN. If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of. A fair peering agreement is where each network moves equal amounts of data to and from a peer, where said traffic is between clients of each network. This means not via a peer network, and the traffic is from/to clients of the same originating network.
This concept probably needs to be fleshed out a bit. I didn't adequately define it in my initial post. Better with a dry erase board.. My goal would be to have every node in MESHland offering services to the wider network do so via a 44-net address which will be advertised as an HNA4 on the network. This is easy to link into HamWAN at interconnection points via BGP. That completely isolates the internal 10.x.y.z/8 2.4GHz addressing (which isn't a viable long-term solution for lots of reasons) from HamWAN. This is a good thing. On the other hand, there are some functions and features which add value to MESHland that require direct communication with OLSR ETX values intact, and routing staying on 10.0.0.0/8. Some folks may choose to not go with 44-net addresses and advertise their services to the wider world. Someone traveling out of area may want their MESH devices to transparently link up with other networks.
The statement " If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of" reflects a commercial service provider mindset. This is not such an environment. If the ham community as a whole contributes to fund, build, and support HamWAN then HamWAN needs to accept uses that may not fit its model. I don't expect there will be huge amounts of traffic flowing via tunneled connections, but it is an important feature. What I don't want to see happen is money, site availability, and goodwill expended needlessly operating two parallel networks. There will be compromise and adaptation on both sides.
The most important thing in such an interconnection is harmonizing DSCP marking for QoS. If we want to get very granular and complex, DISA UCR 2013 ( http://www.disa.mil/Services/Network-Services/UCCO/UCR-2013-Draft ) provides a detailed setup (section6 -
http://www.disa.mil/Services/Network-Services/UCCO/~/media/Files/DISA/Servic... ). In the long run, as long as both domains use the same QoS, things are manageable. I've played the game where DSCP tags are remarked on traffic ingress/egress. It isn't fun and generally ends up breaking.
--snipping the BCWARN stuff since it isn't germane --
Hell, I don't even know if NW-MESH is running part 97 or part 15 rules. What say you? :)
In most cases, the difference between Part97 and Part15 is a state of mind. It is only when you start using frequencies outside the ISM/UNII bands or running higher EIRPs on point-to-multipoint links that things become definitely Part97. For example, a 5.8GHz PtP link between two sites running 500mW radios and 26dBi antennas could be considered Part15 or Part97. There may be value in considering it Part15 as we may want to run the link with encryption or perhaps run an encrypted tunnel for a served agency through the link. In such an arrangement, we might get the served agency to fund the link, we install/maintain it, and they get a portion of the available bandwidth as a backup communications channel. That isn't a bad deal for any of the involved parties. Such a setup is being mulled now for one group/area in particular.
Probably the best idea is to continue discussions at an exploratory level and then spend some time with a couple dry-erase boards at SEAPAC. I will be there Friday-Monday (driving back Monday). We can enjoy some frosty beverages and dry-erase our way through most of the challenges.
73-KY9K/Brian
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.org/mailman/listinfo/psdr_hamwan.org
-- Benjamin
On 3/12/2013 9:35 PM, ZPO wrote:
I view the NW-MESH and HamWAN projects as complimentary. HamWAN in general and the PSDR component in particular makes an excellent backbone transport network to move data around the wider area. NW-MESH works well to cover the last mile.
This is where I think HamWAN is being sold short, since it also works great on the last mile. Reason being is HamWAN sites are all planned to be non-ground-level, so LoS probability increases which makes last mile happy. Your trip may be physically longer by a few miles, but the latency and speed will still be good, if not better when compared to multiple ground hops for the same distance. In close-proximity situations, the end-user-site to end-user-site direct connectivity of NW-MESH I believe provides an optimization. A COUPLE hops of MeSh might indeed be superior to traversing a busy sector.
I don't think I'm selling HamWAN short, just looking at the likely timelines to have those cell-type sectors densely deployed and seeing it stretch into years if ever. It is good to have multiple things in the network gene pool. Let the use cases shake themselves out. My thought process is that a few clubs with advantaged sites can form the initial connections from MESHland to the PSDR. I'm being brief, this makes a lot more sense with a whiteboard.
That's a common misconception, that the cell sites need to be densely populated. We're not like the cellular telcos you are familiar with who have to cater to tiny antennas in people's pockets that run barely any power, and shoot through buildings. We're targeting fixed installs which can provide 30dBi gain and run 1W of power, via LoS links. The projected coverage radius of a cell site in point-to-multipoint mode is 25 miles. Just a handful of those cover much of the Puget Sound. We're not talking years. Improving coverage would be an ongoing effort, sure, but to initially blanket the whole Puget Sound is not years.
Maybe we can grab a cold beer at SEAPAC and I'll tell you the joys of trying to tie together US and NATO networks in Astan. All kinds of messiness.
Sounds like a good time.
Breaking between these two paragraphs as this is one of the first spots where the ethos of HamWAN and the MESH programs (the one here and others) tends to differ. There is no "constitution" in MESHland. That is by design. There are sets of interoperability recommendations that can be thought of as barebones versions of the RFCs underpinning the global Internet. I don't particularly care what hardware somebody brings to the party as long as it has the right interfaces and can speak the right protocols. This allows the network to grow organically with each node extending the network a bit more. It also tends to make things very resilient.
Yeah, all the constitution/organizational business in HamWAN land is so that we can channel money in such a way that noone gets offended. It would be good if HamWAN were hardware independent, but WiMAX/LTE gear is just too expensive to make this a reality. So we've taken a compromise on the hardware front. Technically HamWAN doesn't care what hardware you use either as long as you speak the protocols, but the protocols happen to be offered by one vendor (right now). More deeply though, I believe it is the lack of centralized funding that keeps projects like SeattleWireless from taking off. The investment is made either on the ground by individuals buying nodes, which have a high attenuation path between each other, or by a handful of enlightened individuals who seek height, but for a lack of structured broad funding, run out of personal money to really build and sustain something large. HamWAN pools the financial resources of the larger community and invests where the biggest bang for the buck exists: at high, strategic, locations. Such a communal investment naturally gives rise to expectations (on part of the donors) to have it protected, and therefore the need for the HamWAN organization which maintains the investment and keeps it in good health. The raw MESH approach does have theoretical merit, but the window of optimal behavior is quite small. Too low a user density, and you can't see each other to MESH. Too high a user density, and you've got a slow or unusable network. There exists a small window of just the right user density that allows a MESH network to work properly, but the dependency on such a small window is what gives me the impression of it being a fragile design. In the HamWAN design, a low user density is best served by sparse, tall sites. Increased density (which comes with increased funding) is met with more numerous site deployments, each having a smaller coverage area. This model scales continually as the network evolves in either direction (bigger or smaller) through time. In terms of the resiliency, what's stopping a NW-MESH node from announcing a whole bunch of nonsense routes and hijacking traffic?
This is where I think you, Rob, and the rest of the NW-MESH team need to get together with the HamWAN people to spell out exactly what it is that each of us is trying to accomplish. Do you have a mission statement you can share? Do you have a favorite low-noise public meeting location with power + wifi? Starbucks is sounding likely.
I'd like to see some more detail from Bart on what he sees as the large issues with bidirectional interconnection (I think I know most of them) or simply using the PSDR as transport via tunneling between access nodes.
So even though it's possible for HamWAN to act as a dumb pipe for NW-MESH, this is not a fair arrangement for HamWAN. If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of. A fair peering agreement is where each network moves equal amounts of data to and from a peer, where said traffic is between clients of each network. This means not via a peer network, and the traffic is from/to clients of the same originating network.
This concept probably needs to be fleshed out a bit. I didn't adequately define it in my initial post. Better with a dry erase board.. My goal would be to have every node in MESHland offering services to the wider network do so via a 44-net address which will be advertised as an HNA4 on the network. This is easy to link into HamWAN at interconnection points via BGP. That completely isolates the internal 10.x.y.z/8 2.4GHz addressing (which isn't a viable long-term solution for lots of reasons) from HamWAN.
OK, but this doesn't allow for the MESH nodes to talk to each other via HamWAN should their ground links fail, right? Seems kinda limited in utility. Will these 44-net addresses be advertised to the Internet? Individuals getting allocations of 1-16 IPs directly from AMPR won't be allowed to announce. You need to aggregate somehow up to at least a /24 to qualify. This in most cases implies suballocating from an upstream carrier who is announcing for you. That's the HamWAN plan for its users. What's the plan for NW-MESH users? What IS the viable long-term solution for the NW-MESH internal addressing if 10/8 is not it?
This is a good thing. On the other hand, there are some functions and features which add value to MESHland that require direct communication with OLSR ETX values intact, and routing staying on 10.0.0.0/8. Some folks may choose to not go with 44-net addresses and advertise their services to the wider world. Someone traveling out of area may want their MESH devices to transparently link up with other networks.
For the HamWAN mobility case, I'd like to have other networks with their own 44-blocks all dialed into a VPN to support the exchange of mobile traffic. That means if a user leaves their HamWANOregon network and carries a sub-allocation of his 44.x.y.z/28 or whatever, and then joins the HamWANWashington network, his personal subnet is still fully routable on the Internet and via RF by way of HamWANOregon and HamWANWashington exchanging that route via VPN, even though the /28 announcement would be too small for the Internet and would probably violate BGP route filters with our respective ISPs. This way noone needs to compromise. Permanent re-location of networks might cause some excessive traffic though, so re-addressing of the user's subnet might have to be done. But that's where DNS makes things nice.
The statement " If this were a peering agreement at an Internet exchange, it would be considered a violation of the terms of service set out in every Internet exchange I'm aware of" reflects a commercial service provider mindset. This is not such an environment.
Well, the commercial rules arose out of general principles of fairness. I'm not suggesting money change hands to offset traffic imbalances, but I would like to make the traffic exchange fair. What "fair" means exactly is a complicated thing, and we need to hold a few discussions about it I'm sure.
If the ham community as a whole contributes to fund, build, and support HamWAN then HamWAN needs to accept uses that may not fit its model.
In reality, HamWAN donors will be driving the creation of the network. It is to them that I feel primarily beholden, not the general ham community. For example, if sector users are finding their speeds going to zero due to inter-MESH traffic, HamWAN will likely react to make the service work for the people who made it happen in the first place. I consider this a fair approach, although it is certainly not an everyone-is-equal approach. You don't face this problem in NW-MESH since you're not centralizing your funding, so everyone can be equal. But on the flip side of the NW-MESH approach, any joker can come online and deny the network for others, no? How do you deal with that kind of situation? In the HamWAN case once abusive behavior is identified, the offending client can be booted from the network.
I don't expect there will be huge amounts of traffic flowing via tunneled connections, but it is an important feature.
I'd like to see detailed models on this. What sticks in my memory is the Auburn meeting I attended, and just the one room of routers (20-30 of them maybe?) produced so much traffic that you could barely get a ping through the MESH there. I expect they were all running 54Mbit to boot, due to their close proximity. Sadly noone was able to do an over-the-air protocol analysis to see what was going on.
What I don't want to see happen is money, site availability, and goodwill expended needlessly operating two parallel networks. There will be compromise and adaptation on both sides.
Yup!
The most important thing in such an interconnection is harmonizing DSCP marking for QoS. If we want to get very granular and complex, DISA UCR 2013 ( http://www.disa.mil/Services/Network-Services/UCCO/UCR-2013-Draft ) provides a detailed setup (section6 - http://www.disa.mil/Services/Network-Services/UCCO/~/media/Files/DISA/Servic... ). In the long run, as long as both domains use the same QoS, things are manageable. I've played the game where DSCP tags are remarked on traffic ingress/egress. It isn't fun and generally ends up breaking.
Yup! And QoS is an open engineering problem on the HamWAN side right now. I don't think the IPv4 DS field will be the answer either. We may actually be looking at rolling VLANs to solve this without compromise. The problem is we actually want to extend QoS down to the clients' internal networks so that your VoIP phone for example will have a higher QoS than your PC. At the same time, it'd be handy for the same PC to normally ride a "normal" QoS tier, but when you know you're gonna move 100GB of data and you're not in a hurry, that same PC ought to declare to the network "hey, use a best-effort QoS". This may not be implementable given current technology, but we'll shoot for the stars and hopefully reach the moon. Have not seen that mil spec. Looks daunting. :)
--snipping the BCWARN stuff since it isn't germane --
*gasp!*
In most cases, the difference between Part97 and Part15 is a state of mind. It is only when you start using frequencies outside the ISM/UNII bands or running higher EIRPs on point-to-multipoint links that things become definitely Part97. For example, a 5.8GHz PtP link between two sites running 500mW radios and 26dBi antennas could be considered Part15 or Part97.
This is not the case in 5GHz land. EIRP is limited to 1W under U-NII. Anything beyond that and you gotta claim part 97 (or possibly ISM?). I have not looked at 2.4GHz, but you may want to. I suspect the rules will be similar. See our Spectrum Allocation <https://www.hamwan.org/t/tiki-index.php?page=Spectrum+Allocation&structure=HamWAN> page for a detailed to-the-point analysis of the official publications with sources cited. Apologies in advance for the wide format.
There may be value in considering it Part15 as we may want to run the link with encryption or perhaps run an encrypted tunnel for a served agency through the link. In such an arrangement, we might get the served agency to fund the link, we install/maintain it, and they get a portion of the available bandwidth as a backup communications channel. That isn't a bad deal for any of the involved parties. Such a setup is being mulled now for one group/area in particular.
Probably the best idea is to continue discussions at an exploratory level and then spend some time with a couple dry-erase boards at SEAPAC. I will be there Friday-Monday (driving back Monday). We can enjoy some frosty beverages and dry-erase our way through most of the challenges.
I agree on the white boarding, but the timeline you propose is really far away. It's going to become harder to make any necessary changes on HamWAN/NW-MESH as the networks grow over the next few months. I'd like to see us reach some agreements well before SeaPac. --Bart
participants (3)
-
Bart Kus -
Benjamin Krueger -
ZPO