Geek asks and answers

Batch Rename Keys

I have a fairly large database (5 dB about a million keys each), and each key has an environment namespace in it. For instance:"datamine::production::crosswalk==foobar"

I need to synchronize the development environment with this data copied from an RDB snapshot.

So what I'm trying to do is batch rename each key, change the namespace from datamine::productionto datamine::development. Is there a good way to achieve this?

What I tried so far

  • redis-cli command keys "datamine::production*"sent to sed, then back to redis-cli. It takes forever, and for some reason, bombs on many keys (combining several into one line, sporadically). I would prefer a better option.

  • Search / replace Perl in a .rdb file. My local redis-server apartment refuses to load the modified RDB.

+3
source share
2 answers

Decision:

Ok, here I wrote a script to solve this problem. To do this, you need a stone "Radish". Hope someone finds this helpful ...

#!/usr/bin/env ruby

# A script to translate the current redis database into a namespace for another environment
# GWI Redis keys are namespaced as "datamine::production", "datamine::development", etc.
# This script connects to redis and translates these key names in-place.
#
# This script does not use Rails, but needs the "redis" gem available
require 'Benchmark'
require 'Redis'

FROM_NAMESPACE = "production"
TO_NAMESPACE = "development"
NAMESPACE_PREFIX = "datamine::"
REDIS_SERVER = "localhost"
REDIS_PORT   = "6379"
REDIS_DBS = [0,1,2,3,4,5]

redis = Redis.new(host: REDIS_SERVER, port: REDIS_PORT, timeout: 30)

REDIS_DBS.each do |redis_db|
  redis.select(redis_db)
  puts "Translating db ##{redis_db}..."
  seconds = Benchmark.realtime do
    dbsize = redis.dbsize.to_f
    inc_threshold = (dbsize/100.0).round
    i = 0
    old_keys = redis.keys("#{NAMESPACE_PREFIX}#{FROM_NAMESPACE}*")
    old_keys.each do |old_key|
      new_key = old_key.gsub(FROM_NAMESPACE, TO_NAMESPACE)
      redis.rename(old_key, new_key)
      print "#{((i/dbsize)*100.0).round}% complete\r" if (i % inc_threshold == 0) # on whole # % only
      i += 1
    end
  end
  puts "\nDone. It took #{seconds} seconds"
end
+3
source

I have a working solution:

EVAL "local old_prefix_len = string.len(ARGV[1])
local keys = redis.call('keys', ARGV[1] .. '*')
for i = 1, #keys do
    local old_key = keys[i]
    local new_key = ARGV[2] .. string.sub(old_key, old_prefix_len + 1)
    redis.call('rename', old_key, new_key)
end" 0 "datamine::production::" "datamine::development::"

The last two parameters are the old prefix and the new prefix.

+1
source

More articles:


All Articles