開発
Small tip for migrating from Trac to Redmine
denvazh
How come?
We were using Trac for only one project in the company. (*Unfortunately) After sometime it was necessary to migrate all the project contents to the Redmine system. (*Fortunately) Redmine provides nice migration script called migrate_from_trac, thus it is very easy to migrate. (*Unfortunately, again) What seems easy, however, might be a bit harder when one would actually try using it, but I will return to it later.
Migrating
Redmine uses Rails, thus it is necessary to setup database and add connection information to the config/database.yml.
If is also required to install necessary database connectors for both Redmine and Trac databases. For example, Trac uses sqlite ( by default ),
so if it necessary to migrate from sqlite database, then one can install necessary connectors with:
$: gem install sqlite-ruby or $: gem install sqlite3-ruby
Finally, to migrate project it is necessary to run rake script with migrate_from_trac option. To migrate settings to specific database environment use RAILS_ENV environment variable. For example, to migrate to production database:
$: rake redmine:migrate_from_trac RAILS_ENV="production" Trac directory []: /home/projects/myproject Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: Database encoding [UTF-8]: Target project identifier []: myproject Deleting data Migrating components.............................. Migrating milestones.............. Migrating custom fields....... Migrating tickets................................. Migrating wiki........... Components: 10/11 Milestones: 25/25 Tickets: 131/131 Ticket files: 203/203 Custom values: 15/15 Wiki edits: 1022/1022
Troubles?
In my case, I had trouble when migration scripts was unable to import entries with not correct timestamp.
$: rake generate_session_store $: rake redmine:load_default_data RAILS_ENV="production" $: rake redmine:migrate_from_trac RAILS_ENV="production" WARNING: a new project will be added to Redmine during this process. Are you sure you want to continue ? [y/N] y Trac directory []: /home/projects/myproject Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: sqlite3 Trac database encoding [UTF-8]: Target project identifier []: myproject Migrating components.... Migrating milestones...... Migrating custom fields... Migrating tickets............................. Migrating wiki.rake aborted! PGError: ERROR: timestamp out of range: "41334730-01-26 05:38:55.492266" LINE 1: ...otected", "parent_id", "wiki_id", "title") VALUES('41334730-... ^ : INSERT INTO "wiki_pages" ("created_on", "protected", "parent_id", "wiki_id", "title") VALUES('41334730-01-26 05:38:55.492266', 'f', NULL, 2, 'DissGlossar') RETURNING "id" Tasks: TOP => redmine:migrate_from_trac (See full trace by running task with --trace)
After digging for a bit more time ( for instance running with suggested option –trace ) I noticed, that timestamp entries in sqlite database was stored as 16-digit long value, whereas postgres expected 10-digit long timestamp. Fixing content in the sqlite database is quite a hassle, because it involves changes in all entries for tickets and wiki, thus I created a patch to improve migrate_from_trac rake script in way, that every table that had timestamp field would be fixed during migration. This way data would be extracted from sqlite database as it is, then if timestamp value is 16-digit long, then it will reduce it to 10-digit ( basically drop last 6 zeros ) format.
To use this patch
- Copy and paste code below to the file migrate_from_trac.patch
- Put file to redmine project directory in “lib/tasks”
- From redmine root run command: cd lib/tasks && patch < migrate_from_trac.patch
- After patch was applied, run migration script again
Patch that allows to workaround migration of 16-digit format timestamp to database, that expects 10-digit long format:
--- migrate_from_trac.rake 2012-03-26 11:11:52.000000000 +0900 +++ migrate_from_trac.rake 2012-03-26 11:56:15.000000000 +0900 @@ -22,6 +22,27 @@ namespace :redmine do desc 'Trac migration script' task :migrate_from_trac => :environment do + + module FixTime + # Workaround for import from sqlite3 + # Fixing issue with timestamp length ( Trac - 16, Redmine - 10) + class TrueTime + attr_accessor :timestamp + + def initialize(input_timestamp) + @timestamp = fix_timestamp(input_timestamp) + end + + def fix_timestamp(timestamp) + if (timestamp && timestamp > 9999999999) + new_time = timestamp/1000000 + return new_time.ceil + else + return timestamp + end + end # fix_timestamp + end # class + end module TracMigrate TICKET_MAP = [] @@ -69,6 +90,7 @@ 'developer' => developer_role } + class ::Time class << self alias :real_now :now @@ -93,7 +115,9 @@ # If this attribute is set a milestone has a defined target timepoint def due if read_attribute(:due) && read_attribute(:due) > 0 - Time.at(read_attribute(:due)).to_date + duetime = FixTime::TrueTime.new(read_attribute(:due)) + due = duetime.timestamp + Time.at(due).to_date else nil end @@ -101,7 +125,9 @@ # This is the real timepoint at which the milestone has finished. def completed if read_attribute(:completed) && read_attribute(:completed) > 0 - Time.at(read_attribute(:completed)).to_date + completed = FixTime::TrueTime.new(read_attribute(:completed)) + completed = completed.timestamp + Time.at(completed).to_date else nil end @@ -121,7 +147,10 @@ set_table_name :attachment set_inheritance_column :none - def time; Time.at(read_attribute(:time)) end + def time + time = FixTime::TrueTime.new(read_attribute(:time)) + Time.at(time.timestamp) + end def original_filename filename @@ -182,14 +211,24 @@ read_attribute(:description).blank? ? summary : read_attribute(:description) end - def time; Time.at(read_attribute(:time)) end - def changetime; Time.at(read_attribute(:changetime)) end + def time + time = FixTime::TrueTime.new(read_attribute(:time)) + Time.at(time.timestamp) + end + + def changetime + time = FixTime::TrueTime.new(read_attribute(:changetime)) + Time.at(time.timestamp) + end end class TracTicketChange < ActiveRecord::Base set_table_name :ticket_change - def time; Time.at(read_attribute(:time)) end + def time + time = FixTime::TrueTime.new(read_attribute(:time)) + Time.at(time.timestamp) + end end TRAC_WIKI_PAGES = %w(InterMapTxt InterTrac InterWiki RecentChanges SandBox TracAccessibility TracAdmin TracBackup TracBrowser TracCgi TracChangeset \ @@ -214,7 +253,10 @@ super.select {|column| column.name.to_s != 'readonly'} end - def time; Time.at(read_attribute(:time)) end + def time + time = FixTime::TrueTime.new(read_attribute(:time)) + Time.at(time.timestamp) + end end class TracPermission < ActiveRecord::Base
Now migration should go without and further problems. Let’s try again.
$: rake redmine:migrate_from_trac RAILS_ENV="production" WARNING: a new project will be added to Redmine during this process. Are you sure you want to continue ? [y/N] y Trac directory []: /home/projects/myproject Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: sqlite3 Trac database encoding [UTF-8]: Target project identifier []: myproject Migrating components.... Migrating milestones...... Migrating custom fields... Migrating tickets............................. Migrating wiki.......................................................................................................................................................................................................................................................................................................................... Components: 0/4 Milestones: 0/6 Tickets: 29/29 Ticket files: 1/1 Custom values: 84/84 Wiki edits: 314/314 Wiki files: 21/22
I solved it by converting all time columns like:
UPDATE ticket SET time = time / 1000000.0 where time > 9999999999;