Cloning Directories in Ruby using Hard Links
Hard links exist under windows 7 so you can clone huge directories or files without taking up any extra disk space. Both the original and the copy are equal and apps can't tell the difference between them - because we are using true hard links (not shortcuts or symbolic links).
The dos command for cloning a file is simply
mklink /H original clone
If you delete the original file, the clone is still there and indistinguishable from the original file. You can clone as many times as you like and no extra disk space is used.
I'm not aware of a way of recursively hard linking directories (files as hard links, sub directories as real directories) in windows or linux using the standard commands, so here is a recursive directory cloning utility written in Ruby. It uses the FileUtils.ln method to do the file cloning. This works under both windows 7 and linux.
# Copy a directory recursively creating hardlinks for files and real dirs for directories
# Andy Bulka, Reilly Beacom
# version 1.5
require 'fileutils'
require 'optparse'
def ln_r source, target, options = {:verbose => true, :report => true, :countsize => false, options[:deletetarget] => true}
verbose = options[:verbose]
puts "ln_r copying and hard linking from #{source} to #{target}" if verbose
puts "..." if verbose
raise "source not a directory" if not File.directory?(source)
# Add trailing slash
source = File.join(source, "")
target = File.join(target, "")
# Ensure target dir exists before we start and delete destination files
FileUtils.mkdir_p target
FileUtils.rm_r Dir.glob(File.join(target, '/*')) if options[:deletetarget]
total_file_sizes = 0
Dir.glob(File.join(source, '**/*')).each do | source_path |
target_path = source_path.gsub Regexp.new("^" + source), target
if File.file? source_path
FileUtils.mkdir_p File.dirname(target_path)
FileUtils.ln source_path, target_path
total_file_sizes += File.size(source_path) if options[:countsize]
puts "created hard link #{target_path} (source: #{source_path}" if verbose
else
FileUtils.mkdir_p target_path
puts "created directory " + target_path if verbose
end
end
puts "Done copying/linking." if verbose
def number_with_delimiter(number, delimiter=",")
number.to_s.gsub(/(\d)(?=(\d\d\d)+(?!\d))/, "\\1#{delimiter}")
end
puts "Bytes saved by linking: #{number_with_delimiter(total_file_sizes/1024000)} Mb" if options[:countsize]
if options[:report]
puts
puts "---- RESULT: SOURCE DIRECTORY"
puts Dir.glob(File.join(source, '/**/*'))
puts "---- TARGET DIRECTORY"
puts Dir.glob(File.join(target, '/**/*'))
puts "---- RESULT END"
puts
end
end
# This hash will hold all of the options
# parsed from the command-line by
# OptionParser.
options = {}
optparse = OptionParser.new do|opts|
# Set a banner, displayed at the top
# of the help screen.
opts.banner = "Usage: ln_r.rb [options] source_dir target_dir"
# Define the options, and what they do
options[:verbose] = false
opts.on( '-v', '--verbose', 'Output more information' ) do
options[:verbose] = true
end
options[:test] = false
opts.on( '-t', '--test', 'Run test copy on test data dirs - Andy only' ) do
options[:test] = true
end
options[:report] = false
opts.on( '-r', '--report', 'Display directory of source and target dirs after finish' ) do
options[:report] = true
end
options[:countsize] = false
opts.on( '-c', '--countsize', 'Display bytes saved by using hard linking' ) do
options[:countsize] = true
end
options[:deletetarget] = true
opts.on( '-d', '--dontdeletetarget', 'Dont rm -r * target directory first' ) do
options[:deletetarget] = false
end
#options[:logfile] = nil
#opts.on( '-l', '--logfile FILE', 'Write log to FILE' ) do|file|
# options[:logfile] = file
#end
# This displays the help screen, all programs are
# assumed to have this option.
opts.on( '-h', '--help', 'Display this screen' ) do
puts opts
exit
end
end
# Parse the command-line. Remember there are two forms
# of the parse method. The 'parse' method simply parses
# ARGV, while the 'parse!' method parses ARGV and removes
# any options found there, as well as any parameters for
# the options. What's left is the list of files to resize.
optparse.parse!
puts "Being verbose" if options[:verbose]
#puts "Logging to file #{options[:logfile]}" if options[:logfile]
if options[:test]
ln_r "LinkTests1/dirA", "LinkTests1/dirB", options
ln_r "LinkTests2/dirA", "LinkTests2/dirB/MyCopy/Fred", options
exit
end
if ARGV.length < 2
puts optparse.help
exit
end
ln_r ARGV[0], ARGV[1], options
You can invoke it using:
ruby ln_r.rb [options] dir1 dir2 ...
-v, --verbose Output more information
-t, --test Run test copy on test data dirs - Andy only
-r, --report Display directory of source and target dirs after finish
-c, --countsize Display bytes saved by using hard linking
-d, --dontdeletetarget Dont rm -r * target directory first
-h, --help Display this screen
Update Feb 2013
Its true that you can achieve the above in linux with
cp -lr from to
where -r means recursive copy and -l mean use linking.
Under windows, the COPY isn't so smart and so the above ruby script may be of help. Alternatively you can use a port of the linux cp command under windows - and it seems to work OK. See Port of the most important GNU utilities to Windows
Comments:
cp -al dir1 dir2
"copy [cp] with archive [a] (recursive) and link [l]"