tristan/puppet-filebucket-util
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: tristan:master
Branches Tags
tristan:master
tristan:feature-delete
tristan:feature-specify-bucket
tristan:feature-filter-list-by-time
tristan:feature-restore-to-alt-filepath
tristan:feature-backup
tristan:feature-restore-by-file
tristan:feature-restore
tristan:entry-hash
..
compare: tristan:feature-restore-to-alt-filepath
Branches Tags
tristan:master
tristan:feature-delete
tristan:feature-specify-bucket
tristan:feature-filter-list-by-time
tristan:feature-restore-to-alt-filepath
tristan:feature-backup
tristan:feature-restore-by-file
tristan:feature-restore
tristan:entry-hash

No commits in common. "master" and "feature-restore-to-alt-filepath" have entirely different histories.
master ... feature-re

2 changed files with 34 additions and 183 deletions
Show Stats Expand all files Collapse all files

3
README.md
View File

@ -26,6 +26,3 @@ This will allow you to restore a file on disk to the specific version that is st
### Get ### Get
The utility will allow you to get the contents of a backup up file and have it output to the terminal. The utility will allow you to get the contents of a backup up file and have it output to the terminal.
### Delete
The utility allows you to delete entries/backups in the bucket if you choose to do so.

214
bucket-tool
View File

@ -11,63 +11,47 @@ require 'digest'
# BEGIN: Helper Functions # BEGIN: Helper Functions
def usage def usage
puts <<EOT puts "#{__FILE__} [ACTION <arg>] [<flags>]
#{__FILE__} [ACTION <arg>] [<flags>]
Description: Description:
This utlity is meant to be used to interact with & manage the filebucket in the same capacity as This utlity is meant to be used to interact with & manage the filebucket on P4 nodes due to the
the builtin filebucket utility. However, it does not handle any puppet tie ins. utility for that `puppet filebucket -l <action>` being nonfunctional.
I created this due to the puppet filebucket utility in my companies P4 envrionments not This implements the same functionality (minus the puppet tie-in) and will allow the user to
functioning search the filebucket and restore from it.
Actions: Actions:
search <term> : Search for bucket entries matching a portion of the filepath search <term> : Search for bucket entries matching a portion of the filepath
list : List all Bucket entries list : List all Bucket entries
list-files : List all files/paths that have been backed up to the bucket list-files : List all files/paths that have been backed up to the bucket
get <entry-hash> : Get the content of a specific entry (by hash) get <entry-hash> : Get the content of a specific entry (by hash)
restore <value> : Restore previous state of file stored in bucket. Value can be hash or filename/filepath restore <value> : Restore previous state of file stored in bucket. Value can be hash or filename/filepath
: Note: To restore to an alternate path you will need to provide the path via the -o flag : Note: To restore to an alternate path you will need to provide the path via the -o flag
: :
backup <file> : Backup the file to the bucket backup <file> : Backup the file to the bucket (will work relatively unless full path is provided)
delete <hash> : Delete the entry from the bucket
Restore Flags: Restore Flags:
-o | --output-file <file> : Used to provide an alternate restoral path for the restore function -o | --output-file <file> : Used to provide an alternate restoral path for the restore function
Global Flags: Global Flags:
-d | --debug : Set debug flag -d | --debug : Set debug flag
-h | --help : This help message -h | --help : This help message
-b | --bucket <bucket-dir> : User specified bucket directory
Listing Filter Flags:
--from-date <DATE> : Filter listings from starting after this point (FORMAT: YYYY-MM-DD HH:MM:SS)
: EX:
: 1. --from-date 2023-05-10
: 2. --from-date "2023-05-10 13:23"
:
--to-date <DATE> : Filter listings from ending at this point (FORMAT: YYYY-MM-DD HH:MM:SS)
: EX:
: 1. --to-date 2024-05-10
: 2. --to-date "2024-05-10 05:27"
:
Info Format Flags: Info Format Flags:
-i | --inline : Set the info format to inline (MTIME : HASH : FILENAME) -i | --inline : Set the info format to inline (MTIME : HASH : FILENAME)
-l | --long : Set the info format to long -l | --long : Set the info format to long
: Entry [HASH]: : Entry [HASH]:
: Paths: path1,path2,...,pathn : Paths: path1,path2,...,pathn
: MTIME: YYYY-MM-DD HH:MM:SS -#### : MTIME: YYYY-MM-DD HH:MM:SS -####
: :
-c | --csv : Set the info format to csv ( MTIME,HASH,FILENAME1[;FILENAMEn] ) -c | --csv : Set the info format to csv ( MTIME,HASH,FILENAME1[;FILENAMEn] )
Author: Author:
Name: Tristan Ancelet Name: Tristan Ancelet
Email: tristan.ancelet@acumera.com Email: tristan.ancelet@acumera.com
Phone (Work) #: +1 (337) 965-1855 Phone (Work) #: +1 (337) 965-1855
EOT "
end end
def log (message, level = 0) def log (message, level = 0)
@ -210,13 +194,10 @@ $CONFIG[:search_term]=""
$CONFIG[:log_file]="" $CONFIG[:log_file]=""
$CONFIG[:alt_filepath]="" $CONFIG[:alt_filepath]=""
$CONFIG[:info_format]="inline" $CONFIG[:info_format]="inline"
$CONFIG[:from_time]=Time.at(0) # Beginning of epoch time
$CONFIG[:to_time]=Time.now # Today
File.open('/etc/hostname') do |file| File.open('/etc/hostname') do |file|
$HOSTNAME=file.read().strip() $HOSTNAME=file.read().strip()
end end
FLAG_REGEX=/^\-+\S+/ FLAG_REGEX=/^\-+\S+/
DATE_REGEX=/^(?<year>[0-9]{4})-(?<month>[0-9]{1,2})-(?<day>[0-9]{1,2})[[:space:]]*((?<hour>[0-9]{1,2}):(?<minute>[0-9]{1,2}):(?<second>[0-9]{1,2}))?$/
# END: Variables # END: Variables
@ -236,7 +217,7 @@ if not (ARGV & ['-d', '--debug']).empty?
end end
i=0 i=0
case ARGV[i] case ARGV[i]
when 'search', 'get', 'restore', 'backup', 'delete' when 'search', 'get', 'restore', 'backup'
$CONFIG[:action]=ARGV[i] $CONFIG[:action]=ARGV[i]
log "$CONFIG[:action] was set to #{ARGV[i]}" log "$CONFIG[:action] was set to #{ARGV[i]}"
log "user provided search action ARGV[i.next] == #{ARGV[i.next]}" log "user provided search action ARGV[i.next] == #{ARGV[i.next]}"
@ -278,53 +259,14 @@ while i < ARGV.count
when '-o', '--output-file' when '-o', '--output-file'
log "user provided ARGV[i.next] == #{ARGV[i.next]}" log "user provided ARGV[i.next] == #{ARGV[i.next]}"
if ARGV[i.next] != "" and not ARGV[i.next] =~ FLAG_REGEX if ARGV[i.next] != "" and not ARGV[i.next] =~ FLAG_REGEX
$CONFIG[:alt_filepath]=File.expand_path(ARGV[i.next]) $CONFIG[:alt_filepath]=ARGV[i.next]
log "search_term was set to #{$CONFIG[:alt_filepath]}" log "search_term was set to #{ARGV[i.next]}"
i+=2 i+=2
else else
puts "Flag[#{ARGV[i]}] : Argument[#{ARGV[i.next]}] : Either the argument was not provided or it was a flag" puts "Flag[#{ARGV[i]}] : Argument[#{ARGV[i.next]}] : Either the argument was not provided or it was a flag"
usage usage
exit exit
end end
when '-d', '--debug'
log "#{ARGV[i]} as specified, and the $CONFIG[:debug] flag was already enabled"
i+=1
when '-b', '--bucket'
log "user provided ARGV[i.next] == #{ARGV[i.next]}"
if ARGV[i.next] != "" and not ARGV[i.next] =~ FLAG_REGEX
$CONFIG[:bucket_dir]=ARGV[i.next]
log "$CONFIG[:bucket_dir] was set to #{ARGV[i.next]}"
i+=2
else
puts "Flag[#{ARGV[i]}] : Argument[#{ARGV[i.next]}] : Either the argument was not provided or it was a flag"
usage
exit
end
when '--to-date', '--from-date'
if ARGV[i.next] != "" and ARGV[i.next] =~ DATE_REGEX
date_matches = DATE_REGEX.match(ARGV[i.next]).named_captures.each_value.select{|val| not val.nil?}.map(&:to_i)
case ARGV[i]
when '--to-date'
$CONFIG[:to_time]=Time.new(*date_matches)
log "$CONFIG[:to_time] was set to #{$CONFIG[:to_time]}}"
when '--from-date'
$CONFIG[:from_time]=Time.new(*date_matches)
log "$CONFIG[:from_time] was set to #{$CONFIG[:from_time]}}"
end
i+=2
else
puts "Flag[#{ARGV[i]}] : Argument[#{ARGV[i.next]}] : Either the argument was not provided or it was a flag"
usage
exit
end
when FLAG_REGEX
# Catch all to prevent user from specifying a non-accounted for flag
puts "#{ARGV[i]} is not a valid flag."
usage
exit
else else
i+=1 i+=1
@ -337,11 +279,6 @@ if $CONFIG[:action] == ""
puts "Action was not provided" puts "Action was not provided"
end end
if not Dir.exist? $CONFIG[:bucket_dir]
puts "BucketDir[#{$CONFIG[:bucket_dir]}] Does not exist. Please check to make sure configuration is correct"
exit
end
case $CONFIG[:action] case $CONFIG[:action]
when 'search', 'get' when 'search', 'get'
if $CONFIG[:search_term] == "" if $CONFIG[:search_term] == ""
@ -354,8 +291,6 @@ case $CONFIG[:action]
if not File.exist? $CONFIG[:search_term] if not File.exist? $CONFIG[:search_term]
puts "File #{$CONFIG[:search_term]} does not exist. Please check to make sure that there are not typos and attempt the run again." puts "File #{$CONFIG[:search_term]} does not exist. Please check to make sure that there are not typos and attempt the run again."
exit exit
else
$CONFIG[:search_term] = File.expand_path($CONFIG[:search_term])
end end
end end
@ -417,54 +352,6 @@ class BucketEntry
end end
end end
def delete
returncode = true
log "User has chosen to delete this BucketEntry"
Dir.chdir(@entry_dir){
log "Changed to #{@entry_dir} to delete children files"
Dir.children(@entry_dir).each{|file|
log "Deleting #{file}"
if File.unlink(file)
log "#{file} deleted"
else
puts "There was an issue deleting #{File.expand_path(file)}"
exit
end
}
}
log "Deleting #{@entry_dir}"
if Dir.delete(@entry_dir)
log "Deleted #{@entry_dir}"
else
puts "There was an issue when attempting to delete #{@entry_dir}"
end
dir = @entry_dir
log "Beginning to delete trailing dirs unless one has children other than those that make up #{File.dirname(@entry_dir)}"
for i in 1..8
dir = File.dirname(dir)
log "Beginning to delete #{dir}"
children = Dir.children(dir)
log "Dir[#{dir}] children found to be #{children.join(',')}"
if children.empty?
if Dir.delete(dir)
log "Deleted #{dir}"
else
puts "There was an issue when attempting to delete #{dir}"
exit
end
else
log "#{dir} showed to contain another child directory. Not deleting and breaking loop"
break
end
end
returncode
end
end end
class Bucket class Bucket
@ -490,12 +377,8 @@ class Bucket
Dir["#{@bucketdir}/**/paths"].each.map{|path| File.dirname(path)}.each do |directory| Dir["#{@bucketdir}/**/paths"].each.map{|path| File.dirname(path)}.each do |directory|
log "\"#{directory}\" was grabbed from bucket directory. Making new BucketEntry" log "\"#{directory}\" was grabbed from bucket directory. Making new BucketEntry"
entry = BucketEntry.new(directory) entry = BucketEntry.new(directory)
if entry.mtime <= $CONFIG[:to_time] and entry.mtime >= $CONFIG[:from_time] @entries[entry.hash]=entry
@entries[entry.hash]=entry log "BucketEntry[#{entry.hash}] was added to @entries Size=#{@entries.count()}"
log "BucketEntry[#{entry.hash}] was added to @entries Size=#{@entries.count()}"
else
log "Entry[#{entry.hash}] was filtered out by the user provided time constraints"
end
end end
log "Bucket[#{@bucketdir}] was loaded" log "Bucket[#{@bucketdir}] was loaded"
end end
@ -568,10 +451,6 @@ def get_entry_by_file (bucket, filenames)
end end
end end
if $CONFIG[:alt_filepath] != ""
return entry, $CONFIG[:alt_filepath]
end
if filename[0] != '/' if filename[0] != '/'
filename = "/#{filename}" filename = "/#{filename}"
end end
@ -582,12 +461,6 @@ end
def get_entry_by_hash (bucket) def get_entry_by_hash (bucket)
if bucket.entries.has_key? $CONFIG[:search_term] if bucket.entries.has_key? $CONFIG[:search_term]
entry = bucket.entries[$CONFIG[:search_term]] entry = bucket.entries[$CONFIG[:search_term]]
if $CONFIG[:alt_filepath] != ""
log "$CONFIG[:alt_filepath] was set. Skipping prompts asking for filepaths"
return entry, $CONFIG[:alt_filepath]
end
filepath = "" filepath = ""
if entry.filepaths.count == 1 if entry.filepaths.count == 1
filepath = entry.filepaths[0] filepath = entry.filepaths[0]
@ -615,6 +488,10 @@ def restore_entry (bucket)
entry, filepath = get_entry_by_hash bucket entry, filepath = get_entry_by_hash bucket
end end
if $CONFIG[:alt_filepath] != ""
filepath=$CONFIG[:alt_filepath]
end
if get_verification "Are you sure you want to overwrite #{filepath}?" if get_verification "Are you sure you want to overwrite #{filepath}?"
File.open(filepath,'w') do |file| File.open(filepath,'w') do |file|
file.write(entry.content) file.write(entry.content)
@ -661,7 +538,7 @@ def backup_file (bucket)
log "Created preceeding directorys to hash directory" log "Created preceeding directorys to hash directory"
dir=$CONFIG[:bucket_dir] dir=$CONFIG[:bucket_dir]
hash.chars[0,8].each{|char| hash.chars[0,8].map{|char|
dir+="/#{char}" dir+="/#{char}"
log "Checking to make sure that #{dir} doesn't exist" log "Checking to make sure that #{dir} doesn't exist"
if not Dir.exist? dir if not Dir.exist? dir
@ -706,26 +583,6 @@ def backup_file (bucket)
puts "File #{$CONFIG[:search_term]} was backed up to #{entry_dir}" puts "File #{$CONFIG[:search_term]} was backed up to #{entry_dir}"
end end
def delete_entry (bucket)
if bucket.entries.has_key? $CONFIG[:search_term]
entry = bucket.entries[$CONFIG[:search_term]]
else
puts "BucketEntry[#{$CONFIG[:search_term]}] Does not exist. Please make sure you provided the correct hash value"
exit
end
puts "Corresponding Entry: #{entry.info}"
if get_verification "Are you sure you want to delete BucketEntry[#{entry.hash}]? "
if get_verification "This cannot be undone. Are you sure you want to continue?: "
if entry.delete
puts "Ok. BucketEntry[#{entry.hash}] Has been deleted"
end
end
end
end
# END: Work Functions # END: Work Functions
@ -752,9 +609,6 @@ if __FILE__ == $0
when 'backup' when 'backup'
backup_file bucket backup_file bucket
when 'delete'
delete_entry bucket
end end
end end