debian系统redis-dump安装

1. Ruby 环境

Redis-dump 是一个 Ruby 工具，需先安装 Ruby 和 RubyGems。

安装命令：

sudo apt update
sudo apt install ruby-full build-essential

[root@a29d39f5fd10:/opt/redis-dump/bin# apt install ruby-full build-essential
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ruby-full is already the newest version (1:3.1).
build-essential is already the newest version (12.9).
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.

2. 编译工具和开发库

某些 Ruby Gem 可能需要编译本地扩展，需安装构建工具和依赖库：

sudo apt install libssl-dev libreadline-dev zlib1g-dev

root@a29d39f5fd10:/opt/redis-dump/bin#  apt install libssl-dev libreadline-dev zlib1g-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libssl-dev is already the newest version (3.0.16-1~deb12u1).
libreadline-dev is already the newest version (8.2-1.3).
zlib1g-dev is already the newest version (1:1.2.13.dfsg-1).
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
root@a29d39f5fd10:/opt/redis-dump/bin#  apt install libssl-dev libreadline-dev zlib1g-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libssl-dev is already the newest version (3.0.16-1~deb12u1).
libreadline-dev is already the newest version (8.2-1.3).
zlib1g-dev is already the newest version (1:1.2.13.dfsg-1).
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
root@a29d39f5fd10:/opt/redis-dump/bin#  apt install libssl-dev libreadline-dev zlib1g-dev -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libssl-dev is already the newest version (3.0.16-1~deb12u1).
libreadline-dev is already the newest version (8.2-1.3).
zlib1g-dev is already the newest version (1:1.2.13.dfsg-1).
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.

作用：

libssl-dev: SSL/TLS 支持。
libreadline-dev: 增强命令行交互功能。
zlib1g-dev: 压缩库支持。

3. 安装 redis-dump

直接通过 RubyGems 安装：

gem install redis-dump


root@a29d39f5fd10:/opt/redis-dump/bin# gem install redis-dump
Fetching redis-dump-0.6.1.gem
Successfully installed redis-dump-0.6.1
Parsing documentation for redis-dump-0.6.1
Installing ri documentation for redis-dump-0.6.1
Done installing documentation for redis-dump after 0 seconds
1 gem installed

验证安装

redis-dump -v

root@a29d39f5fd10:/opt/redis-dump/bin# redis-dump -v
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- redis (LoadError)from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb:7:in `<top (required)>'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/exe/redis-dump:18:in `<top (required)>'from /usr/local/bin/redis-dump:25:in `load'from /usr/local/bin/redis-dump:25:in `<main>'

安装redis

root@a29d39f5fd10:/opt/redis-dump/bin# gem install redis
Fetching redis-5.4.0.gem
Fetching connection_pool-2.5.3.gem
Fetching redis-client-0.24.0.gem
Successfully installed connection_pool-2.5.3
Successfully installed redis-client-0.24.0
Successfully installed redis-5.4.0
Parsing documentation for connection_pool-2.5.3
Installing ri documentation for connection_pool-2.5.3
Parsing documentation for redis-client-0.24.0
Installing ri documentation for redis-client-0.24.0
Parsing documentation for redis-5.4.0
Installing ri documentation for redis-5.4.0
Done installing documentation for connection_pool, redis-client, redis after 0 seconds
3 gems installed

再次验证

root@a29d39f5fd10:/opt/redis-dump/bin# redis-dump --version
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- yajl (LoadError)
Did you mean?  yamlfrom <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb:8:in `<top (required)>'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/exe/redis-dump:18:in `<top (required)>'from /usr/local/bin/redis-dump:25:in `load'from /usr/local/bin/redis-dump:25:in `<main>'

安装yajl

root@a29d39f5fd10:/opt/redis-dump/bin# gem install yajl-ruby
Fetching yajl-ruby-1.4.3.gem
Building native extensions. This could take a while...
Successfully installed yajl-ruby-1.4.3
Parsing documentation for yajl-ruby-1.4.3
Installing ri documentation for yajl-ruby-1.4.3
Done installing documentation for yajl-ruby after 0 seconds
1 gem installed

再次验证

root@a29d39f5fd10:/opt/redis-dump/bin# redis-dump --version
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- uri/redis (LoadError)from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb:11:in `<top (required)>'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/exe/redis-dump:18:in `<top (required)>'from /usr/local/bin/redis-dump:25:in `load'from /usr/local/bin/redis-dump:25:in `<main>'

修复uri/redis错误

出现 LoadError: cannot load such file – uri/redis 错误是因为 redis-dump 的代码中存在错误的 require 语句，错误地尝试加载 uri/redis（该库不存在）。以下是解决方法：

定位错误文件

root@a29d39f5fd10:/opt/redis-dump/bin# find /var/lib/gems -name "dump.rb"  # 查找 redis-dump 的 dump.rb 文件
/var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb

查看错误文件内容

root@a29d39f5fd10:/opt/redis-dump/bin# vi /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb
bash: vi: command not found

root@a29d39f5fd10:/opt/redis-dump/bin# cat /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rbunless defined?(RD_HOME)RD_HOME = File.expand_path(File.join(File.dirname(__FILE__), '..', '..') )
endrequire 'redis'
require 'yajl'
require 'base64'require 'uri/redis'require_relative "dump/version"class Redisclass Dumpunless defined?(Redis::Dump::VALID_TYPES)VALID_TYPES = ['string', 'set', 'list', 'zset', 'hash', 'none'].freezeend@host = '127.0.0.1'@port = 6379@debug = false@encoder = Yajl::Encoder.new@parser = Yajl::Parser.new@chunk_size = 10000@with_optimizations = true@with_base64 = falseclass << selfattr_accessor :debug, :encoder, :parser, :safe, :host, :port, :password, :timeout, :chunk_size, :with_optimizations, :with_base64def le(msg)STDERR.puts "#%.4f: %s" % [Time.now.utc.to_f, msg]enddef ld(msg)STDERR.puts "#%.4f: %s" % [Time.now.utc.to_f, msg] if debugenddef memory_usage`ps -o rss= -p #{Process.pid}`.to_i # in kbenddef check_utf8=(check)if check == false@parser = Yajl::Parser.new(:check_utf8 => false)endendendattr_accessor :dbs, :uriattr_reader :redis_connectionsdef initialize(dbs=nil, uri="redis://#{Redis::Dump.host}:#{Redis::Dump.port}")@redis_connections = {}@uri = uriunless dbs.nil?@dbs = Range === dbs ? dbs : (dbs..dbs)@dbs = (@dbs.first.to_i..@dbs.last.to_i) # enforce integers@dbs.to_a.each { |db| redis(db) } # open_all_connectionsendenddef redis(db)redis_connections["#{uri}/#{db}"] ||= connect("#{uri}/#{db}")enddef connect(this_uri)self.class.ld 'CONNECT: ' << this_uriopts = {:url => this_uri}opts[:password] = Redis::Dump.password if Redis::Dump.passwordopts[:timeout] = Redis::Dump.timeout if Redis::Dump.timeoutRedis.new **optsenddef each_database@redis_connections.keys.sort.each do |redis_uri|yield redis_connections[redis_uri]endenddef dump filter=nilfilter ||= '*'entries = []each_database do |redis|chunk_entries = []Redis::Dump.ld "[db#{redis.connection[:db]}] Memory before: #{Redis::Dump.memory_usage}kb"dump_keys = redis.keys(filter)dump_keys_size = dump_keys.sizeRedis::Dump.ld "[db#{redis.connection[:db]}] Dumping #{dump_keys_size} keys: #{dump_keys.join(', ')}"dump_keys.each_with_index do |key,idx|entry, idxplus = key, idx+1if block_given?chunk_entries << entryprocess_chunk idx, dump_keys_size do |count|Redis::Dump.ld " dumping #{chunk_entries.size} (#{count}) from #{redis.connection[:id]}"output_buffer = []chunk_entries = chunk_entries.select do |key|type = Redis::Dump.type(redis, key)if self.class.with_optimizations && type == 'string'trueelseoutput_buffer.push self.class.encoder.encode(Redis::Dump.dump(redis, key, type))falseendendunless output_buffer.empty?yield output_bufferendunless chunk_entries.empty?yield Redis::Dump.dump_strings(redis, chunk_entries) { |obj| self.class.encoder.encode(obj) }endoutput_buffer.clearchunk_entries.clearendelseentries << self.class.encoder.encode(Redis::Dump.dump(redis, entry))endendRedis::Dump.ld "[db#{redis.connection[:db]}] Memory after: #{Redis::Dump.memory_usage}kb"endentriesenddef process_chunk idx, total_sizeidxplus = idx+1yield idxplus if (idxplus % self.class.chunk_size).zero? || idxplus >= total_sizeendprivate :process_chunkdef report filter='*'values = []total_size, dbs = 0, {}each_database do |redis|chunk_entries = []dump_keys = redis.keys(filter)dump_keys_size = dump_keys.sizedump_keys.each_with_index do |key,idx|entry, idxplus = Redis::Dump.report(redis, key), idx+1chunk_entries << entryprocess_chunk idx, dump_keys_size do |count|Redis::Dump.ld " reporting on #{chunk_entries.size} (#{idxplus}) from #{redis.connection[:id]}"chunk_entries.each do |e|puts record if obj.global.verbose >= 1dbs[e['db']] ||= 0dbs[e['db']] += e['size']total_size += e['size']endchunk_entries.clearendendendputs dbs.keys.sort.collect { |db| "  db#{db}: #{dbs[db].to_bytes}" }puts "total: #{total_size.to_bytes}"valuesenddef load(string_or_stream, &each_record)count = 0Redis::Dump.ld " LOAD SOURCE: #{string_or_stream}"Redis::Dump.parser.parse string_or_stream do |obj|unless @dbs.member?(obj["db"].to_i)Redis::Dump.ld "db out of range: #{obj["db"]}"nextendthis_redis = redis(obj["db"])Redis::Dump.ld "load[#{this_redis.hash}, #{obj}]"if each_record.nil?if Redis::Dump.safe && this_redis.exists(obj['key'])Redis::Dump.ld " record exists (no change)"nextendbeginval, type = obj['value'], obj['type']Redis::Dump.ld " > load `#{val}`"if Redis::Dump.with_base64 && type === 'string'Redis::Dump.ld " > load+decode64 for `#{val}`"val = Base64.decode64 valendret = Redis::Dump.set_value this_redis, obj['key'], type, val, obj['ttl']rescue => exRedis::Dump.le '(key: %s) %s' % [obj['key'], ex.message]endelseeach_record.call objendcount += 1endcountendmodule ClassMethodsdef type(this_redis, key)type = this_redis.type keyraise TypeError, "Unknown type: #{type}" if !VALID_TYPES.member?(type)typeenddef report(this_redis, key)info = { 'db' => this_redis.connection[:db], 'key' => key }info['type'] = type(this_redis, key)info['size'] = stringify(this_redis, key, info['type'], info['value']).sizeinfo['bytes'] = info['size'].to_bytesinfoenddef dump(this_redis, key, type=nil)type ||= type(this_redis, key)info = { 'db' => this_redis.connection[:db], 'key' => key }info['ttl'] = this_redis.ttl keyinfo['type'] = typestringified = stringify(this_redis, key, info['type'], info['value'])info['value'] = value(this_redis, key, info['type'])info['size'] = stringified.sizeif Redis::Dump.with_base64 && type === 'string'info['value'] = Base64.encode64(info['value'])endinfoenddef dump_strings(this_redis, keys)vals = this_redis.mget *keysidx = -1keys.collect { |key|idx += 1val = vals[idx].to_sinfo = {'db'    => this_redis.connection[:db],'key'   => key,'ttl'   => this_redis.ttl(key),'type'  => 'string','value' => Redis::Dump.with_base64 ? Base64.encode64(val) : val,'size'  => vals[idx].to_s.size}block_given? ? yield(info) : info}enddef set_value(this_redis, key, type, value, expire=nil)t ||= typesend("set_value_#{t}", this_redis, key, value)this_redis.expire key, expire if expire.to_s.to_i > 0enddef value(this_redis, key, t=nil)send("value_#{t}", this_redis, key)enddef stringify(this_redis, key, t=nil, v=nil)send("stringify_#{t}", this_redis, key, v)enddef set_value_hash(this_redis, key, hash)hash.keys.each { |k|  this_redis.hset key, k, hash[k] }enddef set_value_list(this_redis, key, list)list.each { |value|  this_redis.rpush key, value }enddef set_value_set(this_redis, key, set)set.each { |value|  this_redis.sadd? key, value }enddef set_value_zset(this_redis, key, zset)zset.each { |pair|  this_redis.zadd key, pair[1].to_f, pair[0] }enddef set_value_string(this_redis, key, str)this_redis.set key, strenddef set_value_none(this_redis, key, str)# ignoreenddef value_string(this_redis, key)  this_redis.get key                                                       enddef value_list  (this_redis, key)  this_redis.lrange key, 0, -1                                             enddef value_set   (this_redis, key)  this_redis.smembers key                                                  enddef value_zset  (this_redis, key)  this_redis.zrange(key, 0, -1, :with_scores => true).flatten.tuple        enddef value_hash  (this_redis, key)  this_redis.hgetall(key)                                                  enddef value_none  (this_redis, key)  ''                                                                       enddef stringify_string(this_redis, key, v=nil)  (v || value_string(this_redis, key))                          enddef stringify_list  (this_redis, key, v=nil)  (v || value_list(this_redis, key)).join                       enddef stringify_set   (this_redis, key, v=nil)  (v || value_set(this_redis, key)).join                        enddef stringify_zset  (this_redis, key, v=nil)  (v || value_zset(this_redis, key)).flatten.compact.join       enddef stringify_hash  (this_redis, key, v=nil)  (v || value_hash(this_redis, key)).to_a.flatten.compact.join  enddef stringify_none  (this_redis, key, v=nil)  (v || '')                                                     endendextend Redis::Dump::ClassMethodsclass Problem < RuntimeErrordef initialize(*args)@args = args.flatten.compactenddef message() @args && @args.first endendend
endclass Arraydef chunk(number_of_chunks)chunks = (1..number_of_chunks).collect { [] }chunks.each do |a_chunk|a_chunk << self.shift if self.any?endchunksendalias / chunkdef tuple(tuple_size=2)tuples = (1..(size/tuple_size)).collect { [] }tuples.each_with_index do |a_tuple,idx|tuple_size.times { a_tuple << self.shift } if self.any?endtuplesend
endclass Numericdef to_ms(self*1000).to_iend# TODO: Use 1024?def to_bytesargs = case self.abs.to_iwhen (1000)..(1000**2)'%3.2f%s' % [(self / 1000.to_f).to_s, 'KB']when (1000**2)..(1000**3)'%3.2f%s' % [(self / (1000**2).to_f).to_s, 'MB']when (1000**3)..(1000**4)'%3.2f%s' % [(self / (1000**3).to_f).to_s, 'GB']when (1000**4)..(1000**6)'%3.2f%s' % [(self / (1000**4).to_f).to_s, 'TB']else[self, 'B'].joinendend
end

编辑文件内容

root@a29d39f5fd10:/opt/redis-dump/bin#  nano /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb
bash: nano: command not found

安装nano

root@a29d39f5fd10:/opt/redis-dump/bin# apt-get install -y nano
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:hunspell
The following NEW packages will be installed:nano
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
Need to get 680 kB of archives.
After this operation, 2916 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main arm64 nano arm64 7.2-1+deb12u1 [680 kB]
Fetched 680 kB in 1s (475 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package nano.
(Reading database ... 42773 files and directories currently installed.)
Preparing to unpack .../nano_7.2-1+deb12u1_arm64.deb ...
Unpacking nano (7.2-1+deb12u1) ...
Setting up nano (7.2-1+deb12u1) ...
update-alternatives: using /bin/nano to provide /usr/bin/editor (editor) in auto mode
update-alternatives: using /bin/nano to provide /usr/bin/pico (pico) in auto mode
root@a29d39f5fd10:/opt/redis-dump/bin# nano /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb

再次编辑文件

root@a29d39f5fd10:/opt/redis-dump/bin# nano /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/lib/redis/dump.rb

将require ‘uri/redis’ 修改位require ‘uri’

保存并退出编辑器（Ctrl+O → Enter → Ctrl+X）

再次验证

root@a29d39f5fd10:/opt/redis-dump/bin# redis-dump --version
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- drydock (LoadError)from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'from /var/lib/gems/3.1.0/gems/redis-dump-0.6.1/exe/redis-dump:19:in `<top (required)>'from /usr/local/bin/redis-dump:25:in `load'from /usr/local/bin/redis-dump:25:in `<main>'

安装drydock

root@a29d39f5fd10:/opt/redis-dump/bin# gem install drydock
Fetching drydock-0.6.9.gem
Successfully installed drydock-0.6.9
Parsing documentation for drydock-0.6.9
Installing ri documentation for drydock-0.6.9
Done installing documentation for drydock after 0 seconds
1 gem installed