List of New Files Shell Script

This one goes out to the nerds.

This is a pretty sweet bash script that I wrote today…

One of our Windows machines needed a to-do list of files to be processed based on files that had been created on several Linux boxes. The solution was to write a script on the Linux boxes that did the following:

1. Get the current to-do list from the Windows box,
2. Get a list of any files that had changed in the last few minutes,
3. Strip the path and extension
4. Append any new items to the to-do list,
5. Copy the list back to the Windows box.

We’d be using this script to do some verification on new containers created in WebSphere, but you can use the basic idea for whatever you may need.

A couple of great things about this:

• it will continue compiling the list even if the connection between the two machines is lost
• each item appears on the list only once, so if a file is changed multiple times before the list is processed, the item will only be processed once
• multiple machines can run the same script and keep appending to the same to-do list on a remote system

So here’s the script:

#Copy remaining "to_process" from Windows
cat /mnt/282787svnsrv/to_process.txt > /home/wasadmin/scripts/newContainers/to_process.txt
cat /mnt/282787svnsrv/to_process.txt > /home/wasadmin/scripts/newContainers/todo.txt

#Get any new containers
/usr/bin/find /opt/ibm/loganalysis/tmp -type f -cmin -5 -name *.ear | /bin/sed 's/.*\///' | /bin/sed 's/\(...*\)\..*/\1/' > /home/wasadmin/scripts/newContainers/newCont.txt

#Append "new Containers" not in "todo" to end of "todo"

#/usr/bin/comm -23 /home/wasadmin/scripts/newContainers/newCont.txt /home/wasadmin/scripts/newContainers/todo.txt >> /home/wasadmin/scripts/newContainers/todo.txt
cat /home/wasadmin/scripts/newContainers/to_process.txt > /home/wasadmin/scripts/newContainers/todo.txt
cat /home/wasadmin/scripts/newContainers/newCont.txt >> /home/wasadmin/scripts/newContainers/todo.txt
sort -u /home/wasadmin/scripts/newContainers/todo.txt > /home/wasadmin/scripts/newContainers/to_process.txt

#Copy "to_process" back
cat /home/wasadmin/scripts/newContainers/to_process.txt > /mnt/282787svnsrv/to_process.txt

And this is the cron entry to have this process every 20 minutes:

*/20 * * * * /home/wasadmin/scripts/newContainers/newContainers.sh 2>&1 > /dev/null

Advertisement