Just a short note on mixing Linux utilities with Windows. I wanted to set up a single button deployment of updates to a static web site. I had been using an rsync script, but it required me to manually enter authentication credentials every time it was used to send updates to a remote Ubuntu server.
I set up public key authentication, but it would not work with the permissions of the key files on the Windows machine where the updates were coming from. After struggling with this for a bit, I finally used Cygwin running on the Windows machine to set the permissions just as I would on a Linux system. After that, things worked fine — it’s a single click script with no intervention required.
Still don’t know how to get this done using Windows only. Too lazy to figure it out.
As I’ve mentioned before, I use several cloud storage services. However, being a cheap bastard, I have only signed up for free accounts up to now. That means I have lots of little bits of storage scattered here and there (although recently Box gave me 50GB free and my Copy account has risen to 25GB). But that means having to keep track of where I put certain things. It sure would be nice to have a program that would aggregate all these little pieces into a single thing.
Aggregation into something like the storage pools of the ZFS file system would be ideal. Having the ability to configure redundancy and RAID-like capabilities would be icing. Additional requirements would include cross platform operation and strong encryption on the client side.
Of course, using multiple cloud services would introduce more possible sources of failure. What should the system do if one of the services goes down? What should the system do if one of the services decides to shut down? More complications.
Such a program would have to understand the API of each service. That would be another great project — developing a library that abstracts away the differences between services — basically a ZFS-like file system for the cloud.
Why have I not written such a thing myself? As I have whined about before, I spent the last 30 – 40 years writing desktop and embedded systems. Not too much work on web-based applications. Although I’m learning, I would rather have such an application before I have the skill to write it. Any takers?
Way back in my career there was a need to calculate binomial confidence intervals on experiments with very large numbers of trials (thousands to tens of thousands.) The statistics packages of the time couldn’t seem to handle such large numbers of trials.