Hey Luke -

Thanks for the response and the great points! What you say about a load balancer is very true - single point of failure, source of metadata, etc. I guess I suggested it because it was the only approach I've heard of technically that might be able to do the distribution. But other, more torrent based approaches are more interesting and probably more robust.

On Mon, 27 May 2019 at 11:30, Luke Murphy <lukewm@riseup.net> wrote:
I must ask: is the idea of load balancing itself an "always online" hangover!?

Yes it is! :) I suspected I might get that feedback. Having been in charge of the Scratch online community, infrastructure, and development team for about 4 years I'll admit I got a little obsessed with the idea of staying up. But my own reasoning for wanting a way to share the load has less to do with power, and more with the human energy required to keep a local server secure and up and running. I'm thinking of hosting my personal website (currently here, but I'm planning a redesign: http://amosamos.net). It might go down for lack of power, and it might go down for lack of proper configuration, updates, my router, my ISP - there are plenty of potential failure points besides power. I think it would be really cool if I could federate with a bunch of other folks with some shared interests / needs / and values so that if my server went down, my site would stay up for a while (a week or so?) while I sort out the problem. It's not that I mind a day or two of downtime, but potentially it could be weeks. I'd like to be able to be flakey, and to support others when they are flakey for a while, and still mostly be up most of the time. It's not just the weather that is unpredictable, it's also the server admin and their time and circumstances.

Of course this leads to social challenges: For example, I won't agree to cohosting content for a Nazi's website, or one that's disrespectful to women. But I would welcome a conversation about general values and guidelines with a group of people I was federating my solar powered hosting with - say 2 to 10 or so. In many ways, I think the current terrible state of FB and Twitter is because the people in charge pretended that the technical challenges were the most important to spend time on while trying to ignore the (now festering) social questions. Lets agree on what is good and reasonable, and support one another.
 

RE: bittorrent, that's a nice idea too. I also thought about the DAT
protocol and it has a nice section on the considerations between:

  https://docs.datproject.org/docs/faq#how-is-dat-different-than-academic-torrents-or-bittorrent

Where it points to the drawback of "BitTorrent is for sharing static
files, that is, files that do not change over time." for website
content.

Searching on this topic yesterday I found this thing called Planktos which claims to work with plain existing (recent  / updated) browsers to serve static assets via torrenting. It appears as though they have a strategy for cache invalidation.
https://github.com/xuset/planktos#how-it-works
It appears to be using other viewers to serve the files though - not sharing across multiple hosts? Or maybe you can setup seeds that way - I dunno.


So, with DAT (or something similar), we would be downloading copies of
each others websites to our own computers and serving them locally. That
kind of makes sense to me as a nice approach.

Me too!


If I had to guess at the considerations ...

  * Content is changing, so needs to handle syncing / updating
  * Needs to be redundant if one server goes down ("offline first")

Above seem possible.
 
  * Needs to be discovered without central authority (DNS)

I have no idea how this could work. I mean DNS has to be involved if you are using a domain name, but what happens after DNS to route the request to wherever it needs to go next? The only thing I can imagine is some kind of load balancer. :/
 
Finally, I'm not sure if solving this well would kind of invalidate one of the points of lowtechmagazine's solar server: To show that constant uptime is not a necessity, and that if we change our standards a bit we might, you know, be able to maintain a livable biosphere. ;) I agree with that, but it seems like it should be possible to have cheap solar servers and pretty decent uptime (and manage traffic spikes) too.

Thanks for the fun conversation! I'm interested in any further thoughts you or others have.

Amos



  * Should be performant for lower bandwidths
  * ???

Then to map those considerations onto a protocol ...

Interested to hear further thoughts!

Luke

On  0, Amos Blanton <amos.blanton@gmail.com> wrote:
>   Greetings list -
>   Inspired by the great low powered server described on the lowtechmagazine
>   website, I'm interested in setting up a home solar powered server for my
>   personal website.  I set up a webserver on a raspberry pi and made [1]an
>   instructable about it long ago.
>   While I appreciate the need to accept some downtime, I wonder if folks on
>   this list have thought about strategies for federating solar powered
>   servers to share load? Up here in Denmark I have a great deal of daylight
>   at the moment - maybe 18 hours or so. If I could mirror other small sites,
>   and serve them when their hosts are low on batteries, I'd be happy to do
>   it. And I could use the same sort of help when winter comes.
>   I wonder if anyone here has considered creating a solar-energy load
>   balancing service. Members would get mirrorred in other member's
>   locations, and report their remaining battery capacity / charging status
>   to the balancer. The load balancer could then route requests to whoever
>   has the most charge / active sunlight at the moment. Or perhaps there is a
>   strategy that would use something like the bittorrent protocol to server
>   websites?
>   Interesting in any thoughts you have. And thanks for making and sharing
>   such excellent documentation!
>   Amos
>   --
>   [2]http://amosamos.net
>
>References
>
>   Visible links
>   1. https://www.instructables.com/id/Host-your-own-blog-from-a-25-Raspberry-Pi-compute/
>   2. http://amosamos.net/

>_______________________________________________
>HBSC mailing list -- hbsc@we.lurk.org
>To unsubscribe send an email to hbsc-leave@we.lurk.org
_______________________________________________
HBSC mailing list -- hbsc@we.lurk.org
To unsubscribe send an email to hbsc-leave@we.lurk.org


--
http://amosamos.net