Joeri Sebrechts
12-17-2001, 11:00 AM
The problem: my sister is making her own website. The files are on a fileshare on my local home server. They need to go to her account on another machine. I don't want to give her ssh login ability, so what I did was add myself to the group she is in, so I could add files to and delete files from her home dir, while still disallowing logins as user britt.
Now, about the copying of the actual files. I could just scp the files over, but that's a lot of traffic (2 megs each time the site gets updated). So, in comes the hero of the day: rsync. Rsync is a program that you use to sync collections of files across servers or across directories. It minimizes traffic and can be used across an ssh link with a single option. Pretty cool program in other words. If you ever wondered how they manage to keep those huge mirrors in sync with eachother, rsync is the answer.
Anyway, what I did was make a script that contained this simple command:
rsync -rzv -e ssh --delete ~/world/britt/site/ destinationserver:~/friends/britt/public_html
~/friends/britt is a link pointing to /home/britt. (and yes, my sister is called britt, and no, you can't have her email address, she's too young for you)
This rsync command copies the contents of the local html dir to the public dir on the remote server recursively and securely, gzipping it while in transit and deleting files on the remote server that don't exist on the local fileshare.
That's pretty kewl. A lot of ability in that single command there. The only thing lacking though were the permissions. The local permissions (due to how the fileshare is structured to allow both local and remote access) are rw-rw----. The remote permissions should be rw-r--r-- for files and rwx-r-xr-x for dirs. Luckily, the answer is simple, find can do this for us (in combination with chmod).
So I placed a script on the remote server that contains this:
chmod -R 744 ~/friends/britt/public_html/*
find ~/friends/britt/public_html/* -type f -exec chmod u-x {} \;
find ~/friends/britt/public_html/* -type d -exec chmod go+x {} \;
Ok, that's a lot of action there. The first chmod command makes the permissions of all files rwxr--r--. The find command after that removes the x of the user for all files (but not the directories) and the last find adds x for both the group and world (producing rwxr-xr-x) for all dirs. Ofcourse, this all is fully recursive.
Now, I can hear you thinking: "why didn't he combine the first two commands into chmod 644, so that he didn't have to remove the execute ability for the user afterwards?". Well, the problem (and this is rather stupid) is the recursive nature. If you make it 644 it will first come across a directory, and remove the x, even for the local user. So that means the local user can't get into that directory any more. Next it wants to look inside the directory, which was just made impossible, and it barfs on it. Stupid right? Hence the workaround. There is most likely a better way, but this way works, and that's all that matters.
Next I needed a way to make sure this permissions script (called brittperms, due to lack of inspiration) ran on the remote server, but initiated locally. Because I have ssh access, the solution was simple:
ssh destinationserver ~/bin/brittperms
I added that to the script with the rsync in it, and it worked. Now, the only nuisance is that every time I execute this script, it asks me for the password for ssh twice, once for the rsync and once for the ssh command. The solution is, again, relatively simple.
Both servers are running debian stable, with ssh 1.x, so I don't know how well this converts to newer servers. But what I did was take ~/.ssh/identity.pub and copied it to destinationserver:~/.ssh/authorized_keys. That way you don't have to type a password when logging into the remote with ssh ever again. The way it works is that every time you login, the remote takes your public key, encodes some challenge with it, sends it to you, and only if you have to private key matching that public key can you decode it and send the correct response. Pretty nice application of public/private key encryption. Ofcourse, the weak point is the private key, which you can never, ever, give out to someone else. But then, why would you do that?
Anyway, as usual, all that was needed listening on the remote server was ssh. And as usual, there was a relatively simple solution to a complex problem. All I need to do to automate the whole process is add the local script to my crontab, and I can make my sister's site update automatically every hour, on the hour. And both she and I won't have to execute a single command, the way everything ought to be.
[ 17 December 2001: Message edited by: Joeri Sebrechts ]
Now, about the copying of the actual files. I could just scp the files over, but that's a lot of traffic (2 megs each time the site gets updated). So, in comes the hero of the day: rsync. Rsync is a program that you use to sync collections of files across servers or across directories. It minimizes traffic and can be used across an ssh link with a single option. Pretty cool program in other words. If you ever wondered how they manage to keep those huge mirrors in sync with eachother, rsync is the answer.
Anyway, what I did was make a script that contained this simple command:
rsync -rzv -e ssh --delete ~/world/britt/site/ destinationserver:~/friends/britt/public_html
~/friends/britt is a link pointing to /home/britt. (and yes, my sister is called britt, and no, you can't have her email address, she's too young for you)
This rsync command copies the contents of the local html dir to the public dir on the remote server recursively and securely, gzipping it while in transit and deleting files on the remote server that don't exist on the local fileshare.
That's pretty kewl. A lot of ability in that single command there. The only thing lacking though were the permissions. The local permissions (due to how the fileshare is structured to allow both local and remote access) are rw-rw----. The remote permissions should be rw-r--r-- for files and rwx-r-xr-x for dirs. Luckily, the answer is simple, find can do this for us (in combination with chmod).
So I placed a script on the remote server that contains this:
chmod -R 744 ~/friends/britt/public_html/*
find ~/friends/britt/public_html/* -type f -exec chmod u-x {} \;
find ~/friends/britt/public_html/* -type d -exec chmod go+x {} \;
Ok, that's a lot of action there. The first chmod command makes the permissions of all files rwxr--r--. The find command after that removes the x of the user for all files (but not the directories) and the last find adds x for both the group and world (producing rwxr-xr-x) for all dirs. Ofcourse, this all is fully recursive.
Now, I can hear you thinking: "why didn't he combine the first two commands into chmod 644, so that he didn't have to remove the execute ability for the user afterwards?". Well, the problem (and this is rather stupid) is the recursive nature. If you make it 644 it will first come across a directory, and remove the x, even for the local user. So that means the local user can't get into that directory any more. Next it wants to look inside the directory, which was just made impossible, and it barfs on it. Stupid right? Hence the workaround. There is most likely a better way, but this way works, and that's all that matters.
Next I needed a way to make sure this permissions script (called brittperms, due to lack of inspiration) ran on the remote server, but initiated locally. Because I have ssh access, the solution was simple:
ssh destinationserver ~/bin/brittperms
I added that to the script with the rsync in it, and it worked. Now, the only nuisance is that every time I execute this script, it asks me for the password for ssh twice, once for the rsync and once for the ssh command. The solution is, again, relatively simple.
Both servers are running debian stable, with ssh 1.x, so I don't know how well this converts to newer servers. But what I did was take ~/.ssh/identity.pub and copied it to destinationserver:~/.ssh/authorized_keys. That way you don't have to type a password when logging into the remote with ssh ever again. The way it works is that every time you login, the remote takes your public key, encodes some challenge with it, sends it to you, and only if you have to private key matching that public key can you decode it and send the correct response. Pretty nice application of public/private key encryption. Ofcourse, the weak point is the private key, which you can never, ever, give out to someone else. But then, why would you do that?
Anyway, as usual, all that was needed listening on the remote server was ssh. And as usual, there was a relatively simple solution to a complex problem. All I need to do to automate the whole process is add the local script to my crontab, and I can make my sister's site update automatically every hour, on the hour. And both she and I won't have to execute a single command, the way everything ought to be.
[ 17 December 2001: Message edited by: Joeri Sebrechts ]