Shell script in bash to download a file from an ftp server

I need to write a shell script for the bash shell to transfer a file from an ftp server given
ftp server - fileserver@example.com user user1 password pass1

now in / dir 1 / dir2 on the ftp server I have a folder in the following forms
0.7.1.70
0.7.1.71
0.7.1.72

I need to copy the file "file1.iso" from the last folder ie 0.7.1.72 in this case. I also have to check file integrity while copying. Suppose the file is uploaded to the server, and at this time, if I start copying, in this case the copying will not be completed.

I need to do this every 4 hours. this can be done by doing cron work. please, help

I did this. I installed the ftp server folder on my local machine. to check whether the file was fully downloaded or not, I check the size every 50 seconds 5 times, if it is the same then I copy it, otherwise I run the script after 4 hours ... I saved the text file "foldernames.txt ", which has the name of all the folders from which I copied the required file. Therefore, I check if a new folder has been added on the server by checking its name in the foldername.text .. **

every thing works fine just the problem now .. Suppose the file was uploaded at that time there was some kind of network failure .. as I will be sure that I completely downloaded the file .... I tried using md5sum and chksum, but to calculate on The mounted folder took a long time. please, help

here is my script ..

#!/bin/bash
#
# changing the directory to source location 
echo " ########### " >> /tempdir/pvmscript/scriptlog.log
echo `date`>> /tempdir/pvmscript/scriptlog.log
echo " script is strting " >> /tempdir/pvmscript/scriptlog.log
cd /var/mountpt/pvm-vmware
#
# array to hold the name of last five folders of the source location
declare -a arr
i=0
for folder in `ls -1 | tail -5 `; do
arr[i]=$folder
#echo $folder
i=$((i+1))
done
echo " array initialised " >> /tempdir/pvmscript/scriptlog.log
#
#now for these 5 folders we will check if their name is present in the list of copied         
#  folder names
#
echo " checking for the folder name in list " >> /tempdir/pvmscript/scriptlog.log
## $(seq $((i-1)) -1 0 
for j in $(seq $((i-1)) -1 0  ) ; do
var3=${arr[$j]}
#var4=${var3//./}
echo " ----------------------------------------" >>  /tempdir/pvmscript/scriptlog.log
echo " the folder name is $var3" >> /tempdir/pvmscript/scriptlog.log
#
# checking if the folder name is present in the stored list of folder names or not
#
#
foldercheck=$(grep $var3 /tempdir/pvmscript/foldernames.txt | wc -l)
#
if test $foldercheck -eq 1
then 
echo " the folder $var3 is present in the list so will not copy it " >>  /tempdir/pvmscript/scriptlog.log
foldercheck=" "
continue
else
#
echo " folder $var3 is not present in the list so checking if it has the debug.iso file ">> /tempdir/pvmscript/scriptlog.log
#enter inside  the new folder in source
#
cd  /var/mountpt/pvm-vmware/$var3
#
# writing the names of content of folder to a temporary text file
#
ls -1 > /var/temporary.txt
#checking if the debug.iso is present in the given folder
var5=$(grep debug.iso /var/temporary.txt | wc -l)
var6=$(grep debug.iso //var/temporary.txt)
#
check1="true"
#
# if the file is present then checking if it is completely uploaded or not  
#
rm -f /var/temporary.txt
if test $var5 -eq 1 
then 
echo " it has the debug.iso checking if upload is complete   ">>/tempdir/pvmscript/scriptlog.log
#
# getting the size of the file we are checking if size of the file is constant or     changing    # after regular interval
#
var7=$(du -s ./$var6 |cut -f 1 -d '.')
#echo " size of the file is $var7"
sleep 50s
#
# checking for 5 times at a regular interval of 50 sec if size changing or not 
#
#
for x in 1 2 3 4 5 ;do
var8=$(du -s ./$var6 |cut -f 1 -d '.')
#
#if size is changing exit and check it after 4 hrs when the script will rerun
#echo " size of the file $x is $var7"
if test $var7 -ne $var8
then
check1="false"
echo " file is still in the prossess of being uploadig so exiting will check after 4 hr  " >> /tempdir/pvmscript/scriptlog.log
break
fi
sleep 50s
done
#
#if the size was constant copy the file to destination
#
if test $check1 = "true" 
then
echo " upload was complete so copying the debug.iso file  " >>  /tempdir/pvmscript/scriptlog.log
cp $var6 /tempdir/PVM_Builds/ 
echo " writing the folder name to the list of folders which we have copied " >>  /tempdir/pvmscript/scriptlog.log
echo $var3 >> /tempdir/pvmscript/foldernames.txt
echo " copying is complete  " >> /tempdir/pvmscript/scriptlog.log
fi
#else 
#echo $foldercheck >> /vmfs/volumes/Storage1/PVM_Builds/foldernames.txt
else
echo " it do not have the debug.iso file so leaving the directory "  >>/tempdir/pvmscript/scriptlog.log
echo $var3 >> /tempdir/pvmscript/foldernames.txt
echo 
fi
#rm -f /var/temporary.txt
fi
done
+3
3

, . .

( .)

?

(, ).

, , MB, GB, TB, PB? , , db-backup ???.

, , , ?

SLA / ? , , , X (, , ..), .


, , , "", .

,

  filename : TradeData_2012-04-13.dat
  recCount : 777777
  fileSize : 37604730291
  workOfDate: 2012-04-12
  md5sum    : ....

, , , , , , . , script , .

, , ( ), , , ... :

  • X
  • N
  • , .
  • ..

" , X, Y, Z, , ".


rsync , , , , , , rsync .


script, , . Glenn Jackman . scriptFile 'getRemotedata.sh' , while, , "getRemotedata.sh" . , , - 3 * normalTime. , . , , , , .

, .


P.S. StackOverflow (SO) , , FAQ, http://tinyurl.com/2vycnvr, Q/A, , http://i.imgur.com/kygEP.png , , , , <2 >

+2

FTP . , , . , rsync .

+1
#!/bin/sh
if mkdir /tmp/download_in_process 2>/dev/null; then
    echo "cannot start, download in process"
    exit 1
fi

latest=$(ftp hostname << END1 | tail -1
user user1 pass1
cd /dir1/dir2
ls
END1
)

ftp hostname << END2
user user1 pass1
cd /dir1/dir2/$latest
get file1.iso
END2

rmdir /tmp/download_in_process
+1
source

All Articles