Nexus Weblogging
ChinaonRails
You are here ChinaonRails > 敏捷 > > 遗留数据处理

martin


0.273%

disconnected
登录后回复主题 | 跳到回复(7)

遗留数据处理

... by martin ... 20 月 15 天前 ... 451 次点击

发布测试版本,需要导入遗留的数据,刚开始,直接新建两个connection,一个是旧的数据库sqlserver,一个是新的数据库mysql,但是在保存mysql数据的时候:报无法找到sqlserver的表,原来数据在获取的时候才去取sqlserver的数据,并没有完全取出来。于是乎想到前两天网上看到的Magic Multi Connection:http://magicmodels.rubyforge.org/magic_multi_connections/

安装试用一下:
gem install magic_multi_connections


代码只是做了稍微的改动,就可以用了。

require 'rubygems'
require_gem 'activerecord'
require_gem 'magic_multi_connections'

ActiveRecord::Base.establish_connection(:adapter=>'sqlserver',:host=>'localhost',:database=>'hbpcvs',:username=>'sa')

class Member < ActiveRecord::Base
self.table_name = "t_rm_vip_info"
self.primary_key = "card_flowno"
end

class MemberScore < ActiveRecord::Base
self.table_name="t_memtotalscore"
self.primary_key="membercode"
end

scores = {}
MemberScore.find(:all).each{|ms| scores[ms.membercode]=ms.remainderscore}

print "Fetch member's amount:",members.size,"\n"

class UserInfo < ActiveRecord::Base
establish_connection(:adapter=>'mysql',:host=>'localhost',:database=>'romh',:username=>'root')
end

Member.find(:all).each do |m|
u = UserInfo.new
m.attribute_names.each{|ar| u.send("#{ar}=",m.send("#{ar}"))if u.attribute_present?(ar)}
u.card_no = m.id
#其他的属性
#...
#额外另一张表用户的积分
u.socres=scores[m.id].nil? ? 0:scores[m.id]
u.save!
end

成功。


但是唯一可惜的是效率的问题,一万多条数据跑了好几分钟有空再考虑优化,现在得继续忙着干活去.

1 - 2007-7-13 14:16
bd7lx 深圳
Ar 的extension扩展
http://agilewebdevelopment.com/plugins/ar_extensions

http://www.igvita.com/blog/2007/07/11/efficient-updates-data-import-in-rails/

ar-extension够快的处理导入数据插件
2 - 2-3 12:04
bd7lx 深圳
http://www.jobwd.com/article/show/31

Super-fast bulk data imports in Rails with ar-extensions (Sat Jan 19 2008)
7 - 11 小时 48 分钟前 - 9:57
bd7lx 深圳
ActiveRecord create_or_update based on natural-key

效率应该不高

http://www.pathf.com/blogs/2009/01/activerecord-create_or_update-based-on-natural-key/

natural key也叫business key
看完之后有话想说?那就帮楼主加盖一层吧!

在回复之前你需要先进行登录
电子邮件或昵称
密码
© 2007 A Jesse Cai Production   -   About   -   京ICP备07020911号
a site powered by Project Babel